Getting started with Flock

Installing Flock

Flock can be installed using Spack as follows.

spack install mochi-flock +bedrock +mpi +python

The +bedrock variant will enable Bedrock support, which will be useful for spinning up a Flock provider without having to write code. +mpi is used to enable bootstrapping from an MPI communicator. +python enables Flock’s Python binding. The spack info mochi-flock command can be used to show the list of variants available.

In the following sections, the code can be compiled and linked against the flock-server and flock-client libraries, which can be found either by calling find_package(flock) in CMake, or pkg-config --libs --cflags flock-server (respectively flock-client) with PkgConfig.

What is Flock?

Flock is a group management microservice for Mochi. It provides capabilities for:

  • Group Formation: Creating groups of distributed processes or providers

  • Membership Discovery: Discovering other members of a group

  • Dynamic Membership: Supporting processes joining and leaving groups

  • Multiple Bootstrap Methods: Initializing groups via self, view files, MPI, join, or file-based methods

  • Multiple Backends: Static or centralized group management implementations (for now)

Flock is particularly useful if you want your service to provide a simple way to be discovered by clients.

Instantiating a Flock provider

Flock adopts the typical Mochi microservice architecture (used for instance in the Margo microservice template), with a server library providing the microservice’s provider implementation, and a client library providing access to its capabilities (e.g., querying group membership). A provider is what manages a group, hence the first thing we need to do is instantiate a provider.

Since we have enabled Bedrock support, let’s take advantage of that and write a bedrock-config.json file for Bedrock to use (if you are not familiar with Bedrock, I highly recommend you to read the Bedrock section. Using Bedrock will save you development time since it allows you to bootstrap a Mochi service using a JSON file instead of writing code).

{
    "libraries": [
        "libflock-bedrock-module.so"
    ],
    "providers": [
        {
            "type": "flock",
            "name": "my_flock_provider",
            "provider_id": 42,
            "config": {
                "bootstrap": "self",
                "group": {
                    "type": "static",
                    "config": {}
                },
                "file": "mygroup.flock"
            }
        }
    ]
}

We can now give this config file to Bedrock as follows.

$ bedrock na+sm -c bedrock-config.json -v trace
...
[trace] [flock] Provider registered with ID 42
...
[info] Bedrock daemon now running at na+sm://12345-0

We now have a Flock provider running, with a provider id of 42, managing a group using the “self” bootstrap method and a “static” backend. The group view will be persisted to the file “mygroup.flock”.

If you need to create a provider in C (either because you don’t want to use Bedrock or because you want your provider to be embedded into an existing application), the following code shows how to do that.

/*
 * (C) 2024 The University of Chicago
 *
 * See COPYRIGHT in top-level directory.
 */
#include <assert.h>
#include <stdio.h>
#include <margo.h>
#include <flock/flock-server.h>
#include <flock/flock-bootstrap.h>

int main(int argc, char** argv)
{
    (void)argc;
    (void)argv;

    /* Initialize Margo */
    margo_instance_id mid = margo_init("na+sm", MARGO_SERVER_MODE, 0, 0);
    assert(mid);

    margo_set_log_level(mid, MARGO_LOG_INFO);

    /* Get and print server address */
    hg_addr_t my_address;
    margo_addr_self(mid, &my_address);
    char addr_str[128];
    size_t addr_str_size = 128;
    margo_addr_to_string(mid, addr_str, &addr_str_size, my_address);
    margo_addr_free(mid, my_address);
    margo_info(mid, "Server running at address %s, with provider id 42", addr_str);

    /* Initialize provider arguments and group view */
    struct flock_provider_args args = FLOCK_PROVIDER_ARGS_INIT;
    flock_group_view_t initial_view = FLOCK_GROUP_VIEW_INITIALIZER;
    args.initial_view = &initial_view;

    /* Initialize the view from self (single-member group) */
    flock_group_view_init_from_self(mid, 42, &initial_view);

    /* Configuration for static backend */
    const char* config = "{ \"group\":{ \"type\":\"static\", \"config\":{} } }";

    /* Register the provider */
    flock_provider_register(mid, 42, config, &args, FLOCK_PROVIDER_IGNORE);

    /* Wait for finalize */
    margo_wait_for_finalize(mid);

    return 0;
}

The key steps in this code are:

  1. Initialize Margo in server mode

  2. Initialize the provider arguments structure

  3. Create an initial group view using flock_group_view_init_from_self for a single-member group

  4. Specify the configuration (static backend in this case)

  5. Register the provider with flock_provider_register

Interacting with the group via the client interface

Now we can use the client library to create a client object, create a group handle, and interact with our group.

/*
 * (C) 2024 The University of Chicago
 *
 * See COPYRIGHT in top-level directory.
 */
#include <stdio.h>
#include <stdlib.h>
#include <margo.h>
#include <assert.h>
#include <flock/flock-client.h>
#include <flock/flock-group.h>

#define FATAL(...) \
    do { \
        margo_critical(__VA_ARGS__); \
        exit(-1); \
    } while(0)

int main(int argc, char** argv)
{
    if(argc != 3) {
        fprintf(stderr, "Usage: %s <server address> <provider id>\n", argv[0]);
        exit(-1);
    }

    flock_return_t ret;
    hg_return_t hret;
    const char* svr_addr_str = argv[1];
    uint16_t    provider_id  = atoi(argv[2]);

    /* Initialize Margo in client mode */
    margo_instance_id mid = margo_init("na+sm", MARGO_CLIENT_MODE, 0, 0);
    assert(mid);

    margo_set_log_level(mid, MARGO_LOG_INFO);

    /* Lookup server address */
    hg_addr_t svr_addr;
    hret = margo_addr_lookup(mid, svr_addr_str, &svr_addr);
    if(hret != HG_SUCCESS) {
        FATAL(mid, "margo_addr_lookup failed for address %s", svr_addr_str);
    }

    /* Create Flock client */
    flock_client_t flock_clt;
    margo_info(mid, "Creating FLOCK client");
    ret = flock_client_init(mid, ABT_POOL_NULL, &flock_clt);
    if(ret != FLOCK_SUCCESS) {
        FATAL(mid, "flock_client_init failed (ret = %d)", ret);
    }

    /* Create group handle */
    flock_group_handle_t flock_gh;
    margo_info(mid, "Creating group handle for provider id %d", (int)provider_id);
    ret = flock_group_handle_create(flock_clt, svr_addr, provider_id, true, &flock_gh);
    if(ret != FLOCK_SUCCESS) {
        FATAL(mid, "flock_group_handle_create failed (ret = %d)", ret);
    }

    /* Use the group handle (see later tutorials) */
    margo_info(mid, "Group handle created successfully");

    /* Release group handle */
    margo_info(mid, "Releasing group handle");
    ret = flock_group_handle_release(flock_gh);
    if(ret != FLOCK_SUCCESS) {
        FATAL(mid, "flock_group_handle_release failed (ret = %d)", ret);
    }

    /* Finalize client */
    margo_info(mid, "Finalizing client");
    ret = flock_client_finalize(flock_clt);
    if(ret != FLOCK_SUCCESS) {
        FATAL(mid, "flock_client_finalize failed (ret = %d)", ret);
    }

    /* Free address and finalize */
    hret = margo_addr_free(mid, svr_addr);
    if(hret != HG_SUCCESS) {
        FATAL(mid, "Could not free address (margo_addr_free returned %d)", hret);
    }

    margo_finalize(mid);

    return 0;
}

The client is created using flock_client_init. We then create a flock_group_handle_t. This handle is the object that will let us interact with the group.

The group handle is created using flock_group_handle_create, which takes:

  • The client object

  • The server address

  • The provider ID

  • A boolean indicating whether to refresh the group view automatically

  • A pointer to store the resulting handle

flock_group_handle_release should be called to destroy the group handle. flock_client_finalize is then used to finalize the client.

This program can be called as follows (changing the address as needed).

$ ./client na+sm://12345-0 42

Bootstrap methods

Flock supports multiple bootstrap methods for initializing groups:

  • self: Creates a single-member group with just the current process

  • view: Initializes from a provided group view structure

  • mpi: Uses MPI to bootstrap a group from all MPI processes

  • join: Joins an existing group by contacting a member

  • file: Loads group membership from a file

The bootstrap method is specified in the configuration when creating a provider. Later tutorials will cover each of these methods in detail.

Backend types

Flock currently provides two backend implementations:

  • static: For groups with fixed membership that doesn’t change after initialization

  • centralized: For groups with dynamic membership, using a centralized coordination approach (first member is the authority on who is currently a member, and cannot leave the group)

The backend type is specified in the “group” section of the configuration. Later tutorials will cover the differences between these backends and when to use each.