Using Argobots pools with Margo RPCs
In the previous tutorial, we saw that the alpha_provider_register
function is taking an ABT_pool
argument that is passed down to
MARGO_REGISTER_PROVIDER
.
Argobots pools are a good way to assign resources (typically cores) to particular providers. In the following example, we rewrite the server code in such a way that the Alpha provider gets its own execution stream.
server.c (show/hide)
#include <assert.h>
#include <stdio.h>
#include <margo.h>
#include <alpha-server.h>
static void finalize_xstream_cb(void* data);
int main(int argc, char** argv)
{
margo_instance_id mid = margo_init("tcp", MARGO_SERVER_MODE, 0, 0);
assert(mid);
margo_set_log_level(mid, MARGO_LOG_INFO);
hg_addr_t my_address;
margo_addr_self(mid, &my_address);
char addr_str[128];
size_t addr_str_size = 128;
margo_addr_to_string(mid, addr_str, &addr_str_size, my_address);
margo_addr_free(mid,my_address);
margo_info(mid, "Server running at address %s, with provider id 42", addr_str);
ABT_pool pool;
ABT_pool_create_basic(
ABT_POOL_FIFO,
ABT_POOL_ACCESS_SPSC,
ABT_TRUE,
&pool);
ABT_xstream xstream;
ABT_xstream_create_basic(
ABT_SCHED_DEFAULT,
1,
&pool,
ABT_SCHED_CONFIG_NULL,
&xstream);
alpha_provider_register(mid, 42, pool, ALPHA_PROVIDER_IGNORE);
margo_push_finalize_callback(mid, finalize_xstream_cb, (void*)xstream);
margo_wait_for_finalize(mid);
return 0;
}
static void finalize_xstream_cb(void* data) {
ABT_xstream xstream = (ABT_xstream)data;
ABT_xstream_join(xstream);
ABT_xstream_free(&xstream);
}
After initializing Margo (which initializes Argobots), we create an Argobots
pool and an execution stream that will execute work (ULTs and tasklets) from
this pool. We use ABT_POOL_ACCESS_MPSC
as access type to indicate
that there will be multiple producers of work units (in particular, the ES
running the Mercury progress loop) and a single consumer of work units
(the ES we are about to create).
ABT_xstream_create_basic
is then used to create the ES.
Because Margo is initializing and finalizing Argobots, we need a way to
destroy this ES before Margo finalizes Argobots. Hence with use
margo_push_finalize_callback
to add a callback that will be
called upon finalizing Margo. This callback joins the ES and destroys it.
We pass the newly created pool to the alpha_provider_register
function, which will make the Alpha provider use this pool to execute
its RPC handlers.
For more elaborate assignments of resources to providers, please see the Argobots section of this documentation.