Bulk transfers

In the following, we will see how to use bulk (RDMA) transfers in PyMargo. Bellow are the example server and client codes we will use.

Server

import sys
from pymargo.core import Engine, Handle
from pymargo.bulk import Bulk
import pymargo.bulk

class Receiver:

    def __init__(self, engine):
        self.engine = engine

    def do_bulk_transfer(self, handle: Handle, remote_bulk: Bulk, bulk_size: int):
        local_buffer = bytes(bulk_size)
        local_bulk = self.engine.create_bulk(local_buffer, pymargo.bulk.write_only)
        self.engine.transfer(
            op=pymargo.bulk.pull,
            origin_addr=handle.address,
            origin_handle=remote_bulk,
            origin_offset=0,
            local_handle=local_bulk,
            local_offset=0,
            size=bulk_size)
        print(local_buffer)
        handle.respond()

if __name__ == "__main__":
    with Engine("tcp") as engine:
        receiver = Receiver(engine)
        engine.register("do_bulk_transfer", receiver.do_bulk_transfer)
        print(f"Service running at {engine.address}")
        engine.enable_remote_shutdown()
        engine.wait_for_finalize()

Client

import sys
from pymargo.core import Engine
import pymargo

if __name__ == "__main__":
    with Engine("tcp", mode=pymargo.client) as engine:
        do_bulk_transfer = engine.register("do_bulk_transfer")
        address = engine.lookup(sys.argv[1])
        buffer = b"This is a bytes buffer"
        local_bulk = engine.create_bulk(buffer, pymargo.bulk.read_only)
        do_bulk_transfer.on(address)(local_bulk, len(buffer))
        address.shutdown()

The above server registers a do_bulk_transfer method of a Receiver instance (we use such a class just for the convenience of keeping the engine as an instance variable. This also shows how a code can be structured around a class that provides RPC functionalities). This do_bulk_transfer method takes a Bulk object among other arguments. This object represents a handle to a region of memory exposed by the client. The do_bulk_transfer function first creates a bytes buffer of the same size to use as target for a PULL operation. It creates a local Bulk object using the engine’s create_bulk function, then uses transfer to pull the data from the client’s memory to the server’s buffer. Note that this transfer function can transfer to and from any part of the origin and local buffers by specifying different offsets.

On the client side, we also create a Bulk object from a local buffer, and pass it as argument to the RPC.

Note

In the code above we have used bytes objects as buffers. The create_bulk function can however work with any object that satisfies the buffer protocol, such as bytearray, array.array, numpy arrays, etc., provided that the buffer’s memory is contiguous.