Interoperability

Thallium and PyMargo are respectively C++ and Python wrappers for Margo, hence, the three are interoperable. The table bellow summarizes the mapping between Margo constructs and Thallium and PyMargo classes.

Margo	Thallium	PyMargo
margo_instance_id	engine	Engine
hg_addr_t	endpoint	Address
hg_bulk_t	bulk	Bulk
hg_handle_t (sender)	callable_remote_procedure	CallableRemoteFunction
hg_handle_t (receiver)	request	Handle
margo_request	async_response	Request

From Margo to Thallium

The Thallium engine and endpoint classes provide constructors that will create instances from their Margo counterparts. To construct a Thallium bulk from a Margo hg_bulk_t, you can use engine::wrap(). It is also possible to retrieve the underlying Margo constructs from Thallium. The engine::get_margo_instance(), endpoint::get_addr(), and bulk::get_bulk() functions return the object’s internal margo_instance_id, hg_addr_t, and hg_bulk_t respectively.

There is no conversion mechanism (but generally no need for them) between hg_handle_t and callable_remote_procedure or request, and between async_response and margo_request.

Serialization mechanisms differ between Margo and Thallium. Margo uses either Mercury’s boost preprocessor macros to automatically define serialization function or rely on user-provided hg_proc functions, while Thallium relies on the Cereal library. Hence it is not possible to, for instance, define an RPC using MARGO_REGISTER on the client side, and with thallium::engine::define() on the server side, and vice versa.

Important

If you are provided with a margo_instance_id and wrap it in a Thallium engine to define some RPC functions in C++, you will need your engine instance to remain alive after having defined the RPC. This is because the RPC function retains a weak pointer to the engine’s internal structure and will use it when it is invoked. This also applies when creating an instance of a provider: while Thallium’s provider class constructor takes an engine argument, it only keeps a weak reference to it.

From Margo to PyMargo

PyMargo uses pybind11 to convert Margo structures and functions into Python. More specifically, native C handles are exposed as Python capsules. Higher-level classes then wrap these capsules to provide an object-oriented interface for them. For instance, the Address class is a pure-python class wrapping a _pymargo.hg_addr_t capsule, itself exposing an hg_addr_t native handle.

High-level Python classes generally have functions to get the internal capsule of an object, should they be necessary to pass to C/C++ functions. For instance, Address has the hg_addr property, Engine has an mid property, and so on.

If you have a Mochi library developped in C or C++ and want to make a Python binding for it that is compatible with PyMargo, we recommend that you look at how Yokan does it, in its python folder.

Serialization in PyMargo relies on Python’s pickle module, hence it is not possible to register an RPC with MARGO_REGISTER or engine::define() on one side, and expect Python to correctly work on the other side.

Important

If you use Thallium as an intermediary between PyMargo and a C++ library, you can get a margo_instance_id from a PyMargo Engine, then create a Thallium engine from it, but don’t forget to keep this instance alive.