Exercise 2: a proper phonebook Mochi component

Important

This exercise (as well as Exercise 3) does not need the margo-tutorial-exercises repository used in Exercise 1.

Important

If you come from Exercise 1, make sure to deactivate the Spack environment before starting this exercise (we will build a new one, which we will also use in Exercise 3). This can be done with spack env deactivate in any terminal in which you had activated the environment. You can check whether an environment is active using spack env status.

In this exercise, we will use the Margo microservice template to develop a proper phonebook microservice.

Click on the green “Use this template” button and select “Create a new repository”. Give the repository a name (e.g. “phonebook”). Put the repository on “private” if you wish, then click on “Create repository from template”.

Click on Settings > Actions > General, and set the Workflow permissions to “Read and write permissions”, then click “Save” in the Workflow Permissions section (there are save buttons for each section that will save only the modifications made to that section).

Go back to the root of the code (on Github in your browser), and edit initial-setup.json. Change “alpha” to a service name of your choosing (this name will also be used to prefix your API functions, so choose something short, e.g. YP, for yellow pages. In the following, we will assume this is the name you used). Change “resource” to the name of the resource we are going to manage, here “phonebook”. Click on the green “Commit changes” button.

Wait a little and refresh the page. You might see a brown dot indicating a workflow action is in progress. Or you might get a 404, which means the workflow completed: as part of the GitHub workflow that sets up your code, it will delete initial-setup.json.

Note

Other github workflows will run to test your code and upload a coverage report to codecov.io whenever you push commits to GitHub. These workflows will not work properly if you have made the repository private, so you may receive emails from GitHub about some failed workflows. Simply ignore them.

Clone your code in the mochi-tutorial directory on your machine, then create a Spack environment and build the code like you did in Exercise 1 using the spack.yaml file at the root of your new project (spack env create ex2 spack.yaml, spack env activate ex2, spack install). Create a build directory, cd into it, and build the code (cmake .. -DENABLE_TESTS=ON). You may want to use the flag -DENABLE_TESTS=ON when calling cmake to make sure that the tests are also built. You can run tests using make test.

Note

The cmake will download Catch2 as part of the build process, make will build it.

It is now time to edit the code to make it do what we want. The template provides the implementation of two RPCs, “hello” and “sum”. You will recognize the latter as the same as provided in Exercise 1. This can be helpful to reproduce what you have coded in Exercise 1 in the context of this new exercise.

include/YP/YP-client.h contains the functions required to create and free a YP_client object, which will be used to register the RPCs and contact servers. There is nothing to modify in this file.

include/YP/YP-phonebook.h declares a YP_phonebook_handle_t type, which represents a phonebook managed by a remote server. This file is also where the client interface will be defined. Go ahead and add declarations for a YP_insert and a YP_lookup functions, following what you did in Ex 1.

src/client.h contains the definition of the YP_client structure. Go ahead and add two hg_id_t RPC ids to represent the insert and lookup.

Before looking further into the client implementation, open src/types.h and add the type definitions for our RPCs (insert_in_t, insert_out_t, lookup_in_t, lookup_out_t, etc.). Take the example of the sum_* structures for that.

src/client.c contains the implementation of the client-side functions. In YP_client_init, add the registration of our two new RPCs.

Still in src/client.c and using code from Ex 1 (or by copying and adapting the content of the YP_compute_sum function), implement the client-side function that sends the insert and lookup RPCs to the server.

At this point, feel free to compile your code to make sure it builds fine. You won’t be able to test it yet since there is no server-side implementation of our RPCs, so let’s focus on the server library next.

include/YP/YP-backend.h contains the definition of a backend, i.e. the interface that a phonebook implementation must satisfy. Add the proper insert and lookup function pointers to this structure.

src/dummy/dummy-backend.c contains a “dummy” implementation of such a backend. Copy the phonebook.h file from the Ex 1 to the dummy folder, and include it in dummy-backend.c. Add a phonebook_t phonebook field to the dummy_context structure, then edit (1) dummy_create_phonebook and dummy_open_phonebook to add set this field (using phonebook_new()), and (2) dummy_close_phonebook and dummy_destroy_phonebook to call phonebook_delete().

Still in the same file, add the implementation of the dummy_insert and dummy_lookup functions and add the function pointers in the static YP_backend_impl dummy_backend structure definition.

src/provider.h contains the state of a phonebook provider. Edit the YP_provider structure to add the hg_id_t for our two new RPCs, just like you did for the client.

src/provider.c contains the implementation of the provider functions. Find the YP_provider_register function and add the registration of your new RPCs by taking the existing RPCs as examples. Such registration involves (1) calling MARGO_REGISTER_PROVIDER with the appropriate arguments, (2) calling margo_register_data to associate the provider with the RPC, and (3) setting the RPC ID in the provider structure.

Still in src/provider.c, find the YP_finalize_provider function and add the calls necessary to deregister the two new RPCs.

Note

You will first need to add declarations of your new RPCs, at the beginning of the file, where DECLARE_MARGO_RPC_HANDLER is used.

We can now implement the functions that will handle the RPCs. In the same file, find the YP_sum_ult function, copy it (including the DEFINE_MARGO_RPC_HANDLER line that follows it) and edit it to transform it into a YP_insert_ult function, then do the same with a YP_lookup_ult function.

At this point, you can make sure your code builds fine. Your microservice is ready! If you have time, feel free to look into the tests folder, in particular the test-client.c file, and edit it (replacing calls to the “sum” RPC) to try out your new functionalities.

In practice, the next steps at this point would be to (1) add more tests, (2) remove everything related to the “hello” and “sum” RPCs (because obviously a phonebook is not a calculator), and (3) implement more complex backends by copying the code of the “dummy” backend and changing it to use external libraries or more complicated implementations.