Getting started with Yokan
Installing Yokan
Yokan can be installed using Spack as follows.
spack install mochi-yokan +bedrock
The +bedrock
variant will enable Bedrock support, which will be useful
for spinning up a Yokan server without having to write code.
The spack info mochi-yokan
command can be used to show the list of variants
available. Many variants refer to database backend types. They are disabled by
default. In this tutorial we will use the map backend, which is natively available
in Yokan, but feel free to try these tutorials with other backends!
In the following sections, the code can be compiled and linked against the yokan-server,
yokan-client, and yokan-admin libraries, which can be found either by calling
find_package(yokan)
in CMake, or pkg-config --libs --cflags yokan-server
(respectively yokan-client
and yokan-admin
) with PkgConfig.
Instantiating a Yokan provider
Yokan adopts the typical Mochi microservice architecture (used for instance in the Margo microservice template), with a server library providing the microservice’s provider implementation, a client library providing access to its capabilities (e.g., putting and getting key/value pairs), and an admin library providing control over the providers (creating and destroying databases). Hence the first thing we need to do is instantiate a provider.
Since we have enabled Bedrock support, let’s take advantage of that and write a config.json file for Bedrock to use (if you are not familiar with Bedrock, I highly recommand you to read the Bedrock section. Using Bedrock will save you development time since it allows you to bootstrap a Mochi service using a JSON file instead of writing code).
{
"libraries": {
"yokan": "libyokan-bedrock-module.so"
},
"providers": [
{
"name" : "my_yokan_provider",
"type" : "yokan",
"provider_id" : 42,
"pool" : "__primary__",
"config" : {},
"dependencies" : {}
}
]
}
We can now give this config file to Bedrock as follows.
$ bedrock na+sm -c config.json
[2021-10-14 10:16:17.529] [info] [yokan] YOKAN provider registration done
[2021-10-14 10:16:17.530] [info] Bedrock daemon now running at na+sm://8551-0
We now have a Yokan provider running, with a provider id of 42.
If you need to create a provider in C (either because you don’t want to use Bedrock or because you want your provider to be embedded into an existing application), the following code shows how to do that.
server.c (show/hide)
#include <assert.h>
#include <stdio.h>
#include <margo.h>
#include <yokan/server.h>
int main(int argc, char** argv)
{
margo_instance_id mid = margo_init("na+sm", MARGO_SERVER_MODE, 0, 0);
assert(mid);
hg_addr_t my_address;
margo_addr_self(mid, &my_address);
char addr_str[128];
size_t addr_str_size = 128;
margo_addr_to_string(mid, addr_str, &addr_str_size, my_address);
margo_addr_free(mid,my_address);
margo_set_log_level(mid, MARGO_LOG_INFO);
margo_info(mid, "Server running at address %s", addr_str);
yk_return_t ret = yk_provider_register(mid, 42, NULL, YOKAN_PROVIDER_IGNORE);
assert(ret == YOKAN_SUCCESS);
margo_wait_for_finalize(mid);
return 0;
}
Using the admin library to create a database
Now that we have our server running, let’s write some code using the admin library to create a database.
admin.c (show/hide)
#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
#include <margo.h>
#include <yokan/admin.h>
int main(int argc, char** argv)
{
if(argc != 3) {
fprintf(stderr, "Usage: %s <address> <provider id>\n", argv[0]);
exit(-1);
}
margo_instance_id mid = margo_init("na+sm", MARGO_CLIENT_MODE, 0, 0);
assert(mid);
uint16_t provider_id = atoi(argv[2]);
hg_addr_t server_addr = HG_ADDR_NULL;
hg_return_t hret = margo_addr_lookup(mid, argv[1], &server_addr);
assert(hret == HG_SUCCESS);
yk_return_t ret;
yk_admin_t admin = YOKAN_ADMIN_NULL;
yk_database_id_t db_id;
ret = yk_admin_init(mid, &admin);
assert(ret == YOKAN_SUCCESS);
ret = yk_open_database(admin, server_addr, provider_id, NULL, "map", "{}", &db_id);
assert(ret == YOKAN_SUCCESS);
char db_id_str[37];
yk_database_id_to_string(db_id, db_id_str);
printf("Database id is %s (take note of it!)\n", db_id_str);
ret = yk_admin_finalize(admin);
assert(ret == YOKAN_SUCCESS);
margo_finalize(mid);
return 0;
}
After the typical Margo initialization, we lookup the address of
the server from the string address provided in argv[1]
.
We then create a yk_admin_t
object, which we initialize
with yk_admin_init
.
By using yk_open_database
,
the admin sends an RPC to the provider requesting it to open
a database of type map
, which is an in-memory key/value
store implemented using C++ std::map
. Since this database
is in memory, this open operation will actually create it.
The yk_open_database
function’s last parameter is a pointer
to the returned yk_database_id_t
referencing the database.
These identifiers are unique and can be serialized into a 37-byte
null-terminated string using yk_database_id_to_string
.
We print this identifier before calling yk_admin_finalize
to finalize the admin.
This code, once compiled, can be called as follows (changing the address as needed).
$ ./admin na+sm://8972-0 42
Database id is 2e7ca988-9681-43f7-9ef3-ff4c941fbefd (take note of it!)
Interacting with the database via the client interface
Now let the fun start. We can use the client library to create a client object, create a database handle, and start interacting with our database.
client.c (show/hide)
#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <margo.h>
#include <yokan/client.h>
#include <yokan/database.h>
int main(int argc, char** argv)
{
if(argc != 4) {
fprintf(stderr, "Usage: %s <address> <provider id> <database id>\n", argv[0]);
exit(-1);
}
margo_instance_id mid = margo_init("na+sm", MARGO_CLIENT_MODE, 0, 0);
assert(mid);
uint16_t provider_id = atoi(argv[2]);
hg_addr_t server_addr = HG_ADDR_NULL;
hg_return_t hret = margo_addr_lookup(mid, argv[1], &server_addr);
assert(hret == HG_SUCCESS);
yk_return_t ret;
yk_client_t client = YOKAN_CLIENT_NULL;
yk_database_id_t db_id;
ret = yk_client_init(mid, &client);
assert(ret == YOKAN_SUCCESS);
yk_database_id_from_string(argv[3], &db_id);
yk_database_handle_t db_handle = YOKAN_DATABASE_HANDLE_NULL;
ret = yk_database_handle_create(
client, server_addr, provider_id, db_id, &db_handle);
assert(ret == YOKAN_SUCCESS);
const char* key = "matthieu";
const char* value_in = "dorier";
ret = yk_put(db_handle, YOKAN_MODE_DEFAULT,
key, strlen(key), value_in, strlen(value_in));
assert(ret == YOKAN_SUCCESS);
char value_out[128];
size_t value_out_size = 128;
ret = yk_get(db_handle, YOKAN_MODE_DEFAULT,
key, strlen(key), value_out, &value_out_size);
assert(ret == YOKAN_SUCCESS);
assert(strcmp(value_in, value_out) == 0);
ret = yk_database_handle_release(db_handle);
assert(ret == YOKAN_SUCCESS);
ret = yk_client_finalize(client);
assert(ret == YOKAN_SUCCESS);
margo_finalize(mid);
return 0;
}
The client is created using yk_client_init
. We then convert
the string database id into a yk_database_id_t
using
yk_database_id_from_string
, and create a yk_database_handle_t
by passing it to yk_database_handle_create
. This handle
is the object that will let us interact with the database.
As an example of using the client API, we show the use of
yk_put
and yk_get
to respectively put and
get a key/value pair from the database. These functions will be
detailed more in the next tutorial.
yk_database_handle_release
should be called to destroy
the database handle. yk_client_finalize
is then used
to finalize the client.
This program can be called as follows (changing the address and the database id as needed).
$ ./client na+sm://8972-0 42 2e7ca988-9681-43f7-9ef3-ff4c941fbefd