C++ API Documentation

warabi::AsyncRequest

class AsyncRequest

AsyncRequest objects are used to keep track of on-going asynchronous operations.

Public Functions

AsyncRequest(): Default constructor. Will create a non-valid AsyncRequest.

AsyncRequest(const AsyncRequest &other): Copy constructor.

AsyncRequest(AsyncRequest &&other): Move constructor.

AsyncRequest &operator=(const AsyncRequest &other): Copy-assignment operator.

AsyncRequest &operator=(AsyncRequest &&other): Move-assignment operator.

~AsyncRequest(): Destructor.

void wait() const: Wait for the request to complete.

bool completed() const: Test if the request has completed, without blocking.

operator bool() const: Checks if the object is valid.

warabi::Backend

class Backend

Interface for target backends. To build a new backend, implement a class MyBackend that inherits from Backend, and put WARABI_REGISTER_BACKEND(mybackend, MyBackend); in a cpp file that includes your backend class’ header file.

Your backend class should also have two static functions to respectively create and open a target:

std::unique_ptr<Backend> create(const json& config) std::unique_ptr<Backend> attach(const json& config)

Public Functions

Backend() = default: Constructor.

Backend(Backend&&) = default: Move-constructor.

Backend(const Backend&) = default: Copy-constructor.

Backend &operator=(Backend&&) = default: Move-assignment operator.

Backend &operator=(const Backend&) = default: Copy-assignment operator.

virtual ~Backend() = default: Destructor.

inline const std::string &name() const: Return the name of backend.

virtual std::string getConfig() const = 0: Returns a JSON-formatted configuration string.

virtual Result<std::unique_ptr<WritableRegion>> create(size_t size) = 0

Create a region of a given size and return an std::unique_ptr to a WritableRegion, or nullptr if the operation failed.

Parameters: size – Size of the region to create.
Returns: std::unique_ptr<WritableRegion>.

virtual Result<std::unique_ptr<WritableRegion>> write(const RegionID &region, bool persist) = 0: Request access to a particular region for writing. If the region does not exist, returns a nullptr.

virtual Result<std::unique_ptr<ReadableRegion>> read(const RegionID &region) = 0: Request access to a particular region for reading. If the region does not exist, returns a nullptr.

virtual Result<bool> erase(const RegionID &region) = 0

See also

TopicHandle::erase

virtual Result<bool> destroy() = 0

Destroys the underlying target.

Returns: a Result<bool> instance indicating whether the database was successfully destroyed.

virtual Result<std::unique_ptr<MigrationHandle>> startMigration(bool removeSource) = 0

Create a MigrationHandle to start a migration of the target to another provider.

Parameters: removeSource – Whether to delete the source target after completion.

warabi::Client

class Client

The Client object is the main object used to establish a connection with a Warabi service.

Public Functions

Client(): Default constructor.

Client(margo_instance_id mid)

Constructor using a margo instance id.

Parameters: mid – Margo instance id.

Client(const thallium::engine &engine)

Constructor.

Parameters: engine – Thallium engine.

Client(const Client&): Copy constructor.

Client(Client&&): Move constructor.

Client &operator=(const Client&): Copy-assignment operator.

Client &operator=(Client&&): Move-assignment operator.

~Client(): Destructor.

const thallium::engine &engine() const: Returns the thallium engine used by the client.

TargetHandle makeTargetHandle(const std::string &address, uint16_t provider_id) const

Creates a handle to a remote target and returns. You may set “check” to false if you know for sure that the corresponding target exists, which will avoid one RPC.

Parameters

address – Address of the provider holding the database.
provider_id – Provider id.

Returns

a TargetHandle instance.

operator bool() const: Checks that the Client instance is valid.

std::string getConfig() const

Get internal configuration as a JSON-formatted string.

Returns: configuration string.

warabi::Exception

class Exception : public std::exception

warabi::MigrationHandle

class MigrationHandle

A MigrationHandle is an abstract class representing an object that one can request from a target using target.startMigration(). Its roles are (1) to lock all accesses to the target until its destruction (acting like a lock guard), (2) to provide a list of files that need to be migrated, (3) to cleanup any temporary files used during migration upon destruction, and (4) to mark the target as migrated.

Public Functions

virtual ~MigrationHandle() = default: Destructor.

virtual std::string getRoot() const = 0: Get the path relative to which the files returned by getFiles are located.

virtual std::list<std::string> getFiles() const = 0: Get a list of files to migrate. The file names must be relative to the root.

virtual void cancel() = 0: Mark the migration as canceled.

warabi::MigrationOptions

Warning

doxygenclass: Cannot find class “warabi::MigrationOptions” in doxygen xml output for project “warabi” from directory: /home/docs/checkouts/readthedocs.org/user_builds/mochi/checkouts/latest/docs/source/warabi/doxygen

warabi::Provider

class Provider

A Provider is an object that can receive RPCs and dispatch them to specific targets.

Public Functions

Provider(const tl::engine &engine, uint16_t provider_id, const std::string &config, const tl::pool &pool = tl::pool(), remi_client_t remi_cl = nullptr, remi_provider_t remi_pr = nullptr)

Constructor.

Parameters

engine – Thallium engine to use to receive RPCs.
provider_id – Provider id.
config – JSON-formatted configuration.
pool – Argobots pool to use to handle RPCs.
remi_cl – Remi client for migration.
remi_pr – Remi provider for migration.

Provider(margo_instance_id mid, uint16_t provider_id, const std::string &config, const tl::pool &pool = tl::pool(), remi_client_t remi_cl = nullptr, remi_provider_t remi_pr = nullptr)

Constructor.

Parameters

mid – Margo instance id to use to receive RPCs.
provider_id – Provider id.
config – JSON-formatted configuration.
pool – Argobots pool to use to handle RPCs.
remi_cl – Remi client for migration.
remi_pr – Remi provider for migration.

Provider(const Provider&) = delete: Copy-constructor is deleted.

Provider(Provider&&): Move-constructor.

Provider &operator=(const Provider&) = delete: Copy-assignment operator is deleted.

Provider &operator=(Provider&&) = delete: Move-assignment operator is deleted.

~Provider(): Destructor.

std::string getConfig() const

Return a JSON-formatted configuration of the provider.

Returns: JSON formatted string.

operator bool() const: Checks whether the Provider instance is valid.

void migrateTarget(const std::string &address, uint16_t provider_id, const std::string &options)

Migrate the internal target to a destination provider. The options argument should be a JSON string with the following optional keys:

“new_root” (string): path where to place the target in the destination (defaults to the same path as in the source);
”transfer_size” (int): size of individual transfers used by REMI (defaults to using a single transfer for the full target);
”merge_config” (object): the content of this field will be merged with the target’s configuration (defaults to an empty object);
”remove_source” (bool): whether to remove the target in the source provider (defaults to true);

Parameters

address –
provider_id –
options –

warabi::ReadableRegion

class ReadableRegion : public warabi::Region 

Public Functions

virtual Result<bool> read(const std::vector<std::pair<size_t, size_t>> &regionOffsetSizes, thallium::bulk data, const thallium::endpoint &address, size_t bulkOffset) = 0

See also

TopicHandle::read

virtual Result<bool> read(const std::vector<std::pair<size_t, size_t>> &regionOffsetSizes, void *data) = 0

See also

TopicHandle::read

warabi::Region

class Region

Abstract class representing a handle to a region in a given Backend. Each Backend implementation will typically have implementations for WritableRegion and ReadableRegion.

Subclassed by warabi::ReadableRegion, warabi::WritableRegion

Public Functions

virtual Result<RegionID> getRegionID() = 0: Return the RegionID of the region.

warabi::Result

template<typename T> class Result

The Result object is a generic object used to hold and send back the result of an RPC. It contains three fields:

success must be set to true if the request succeeded, false otherwise
error must be set to an error string if an error occured
value must be set to the result of the request if it succeeded

This class is specialized for two types: bool and std::string. If bool is used, both the value and the success fields will be managed by the same underlying variable. If std::string is used, both the value and the error fields will be managed by the same underlying variable.

Template Parameters: T – Type of the result.

Public Functions

inline bool &success(): Whether the request succeeded.

inline const bool &success() const: Whether the request succeeded.

inline std::string &error(): Error string if the request failed.

inline const std::string &error() const: Error string if the request failed.

inline T &value(): Value if the request succeeded.

inline const T &value() const: Value if the request succeeded.

inline T &valueOrThrow() &: Value if the request succeeded, throws otherwise.

inline const T &valueOrThrow() const &: Value if the request succeeded, throws otherwise.

inline T &&valueOrThrow() &&: Value if the request succeeded, throws otherwise.

template<typename F> inline decltype(auto) andThen(F &&f) const &: Execute a function on the value if the value is present, otherwise throws an Exception.

template<typename F> inline decltype(auto) andThen(F &&f) &&: Execute a function on the value if the value is present, otherwise throws an Exception.

inline void check() const: Throw an Exception if the Result contains an error.

template<typename Archive> inline void serialize(Archive &a)

Serialization function for Thallium.

Template Parameters: Archive – Archive type.
Parameters: a – Archive instance.

warabi::Result<bool>

Warning

doxygenclass: Cannot find class “warabi::Result<bool>” in doxygen xml output for project “warabi” from directory: /home/docs/checkouts/readthedocs.org/user_builds/mochi/checkouts/latest/docs/source/warabi/doxygen

warabi::Result<std::string>

Warning

doxygenclass: Cannot find class “warabi::Result<std::string>” in doxygen xml output for project “warabi” from directory: /home/docs/checkouts/readthedocs.org/user_builds/mochi/checkouts/latest/docs/source/warabi/doxygen

warabi::TargetFactory

class TargetFactory

The TargetFactory contains functions to create or open targets.

Public Static Functions

static Result<std::unique_ptr<Backend>> createTarget(const std::string &backend_name, const thallium::engine &engine, const json &config)

Creates a target and returns a unique_ptr to the created instance.

Parameters

backend_name – Name of the backend to use.
engine – Thallium engine.
config – Configuration object to pass to the backend’s create function.

Returns

a unique_ptr to the created Target.

static Result<std::unique_ptr<Backend>> recoverTarget(const std::string &backend_name, const thallium::engine &engine, const json &config, const std::vector<std::string> &filenames)

Recovers a target after migration.

Parameters

backend_name – Name of the backend to use.
engine – Thallium engine.
config – Configuration object to pass to the backend’s create function.
filenames – List of files that were migrated.

Returns

a unique_ptr to the recovered Target.

static Result<bool> validateConfig(const std::string &backend_name, const json &config)

Validate that the JSON configuration has the expected schema.

Parameters

backend_name – Name of the backend to use.
config – Configuration object to validate.

Returns

true if the JSON configuration is valid, false otherwise.

static inline TargetFactory &instance(): Get a singleton instance.

warabi::TargetHandle

class TargetHandle

A TargetHandle object is a handle for a remote target on a server. It enables invoking the target’s functionalities.

Public Functions

TargetHandle(): Constructor. The resulting TargetHandle handle will be invalid.

TargetHandle(const TargetHandle&): Copy-constructor.

TargetHandle(TargetHandle&&): Move-constructor.

TargetHandle &operator=(const TargetHandle&): Copy-assignment operator.

TargetHandle &operator=(TargetHandle&&): Move-assignment operator.

~TargetHandle(): Destructor.

Client client() const: Returns the client this target has been opened with.

operator bool() const: Checks if the TargetHandle instance is valid.

void create(RegionID *region, size_t size, AsyncRequest *req = nullptr) const

Create a region in the target, with the given size.

Parameters

region – [out] Created region ID.
size – [in] Size of the region to create.
req – [out] Optional request to make the call asynchronous.

void write(const RegionID &region, size_t regionOffset, const char *data, size_t size, bool persist = false, AsyncRequest *req = nullptr) const

Write data in a region.

Parameters

region – [in] Region to write into.
regionOffset – [in] Offset in the region at which to start writing.
data – [in] Pointer to the data to write.
size – [in] Size to write.
persist – [in] Whether to also persist to data.
req – [out] Optional request to make the call asynchronous.

void write(const RegionID &region, const std::vector<std::pair<size_t, size_t>> &regionOffsetSizes, const char *data, bool persist = false, AsyncRequest *req = nullptr) const

Write data in multiple non-contiguous segments of a region.

Note: the local data pointer is assumed to be contiguous. Its size is computed from the sum of the sizes in the regionOffsetSizes vector.

Parameters

region – [in] Region to write into.
regionOffsetSizes – [in] Offset/size pairs in the region at which write.
data – [in] Pointer to the data to write.
persist – [in] Whether to also persist to data.
req – [out] Optional request to make the call asynchronous.

void write(const RegionID &region, size_t regionOffset, thallium::bulk data, const std::string &address, size_t bulkOffset, size_t size, bool persist = false, AsyncRequest *req = nullptr) const

Write data in a region.

Note: if address is an empty string, it is assumed that the data comes from the calling process.

Parameters

region – [in] Region to write into.
regionOffset – [in] Offset in the region at which to start writing.
data – [in] Bulk handle from which to pull the data.
address – [in] Address of the process in which the data is.
bulkOffset – [in] Offset at which the data starts in the bulk handle.
size – [in] Size to write.
persist – [in] Whether to also persist to data.
req – [out] Optional request to make the call asynchronous.

void write(const RegionID &region, const std::vector<std::pair<size_t, size_t>> &regionOffsetSizes, thallium::bulk data, const std::string &address, size_t bulkOffset, bool persist = false, AsyncRequest *req = nullptr) const

Write data in multiple non-contiguous segments of a region.

Note: if address is an empty string, it is assumed that the data comes from the calling process.

Parameters

region – [in] Region to write into.
regionOffsetSizes – [in] Offset/size pairs in the region at which write.
data – [in] Bulk handle from which to pull the data.
address – [in] Address of the process in which the data is.
bulkOffset – [in] Offset at which the data starts in the bulk handle.
size – [in] Size to write.
persist – [in] Whether to also persist to data.
req – [out] Optional request to make the call asynchronous.

void persist(const RegionID &region, size_t offset, size_t size, AsyncRequest *req = nullptr) const

Persist a segment of the specified region.

Parameters

region – [in] Region to persist.
offset – [in] Offset from which to persist.
size – [in] Size to persist.
req – [out] Optional request to make the call asynchronous.

void persist(const RegionID &region, const std::vector<std::pair<size_t, size_t>> &regionOffsetSizes, AsyncRequest *req = nullptr) const

Persist non-contiguous segments of the specified region.

Parameters

region – [in] Region to persist.
regionOffsetSizes – [in] Offset/size pairs in the region to persist.
size – [in] Size to persist.
req – [out] Optional request to make the call asynchronous.

void createAndWrite(RegionID *region, const char *data, size_t size, bool persist = false, AsyncRequest *req = nullptr) const: Combines create and write.

void createAndWrite(RegionID *region, thallium::bulk data, const std::string &address, size_t bulkOffset, size_t size, bool persist = false, AsyncRequest *req = nullptr) const: Combines create and write.

void read(const RegionID &region, size_t regionOffset, char *data, size_t size, AsyncRequest *req = nullptr) const

Read part of a region into the provided local memory buffer.

Parameters

region – [in] Region to read.
regionOffset – [in] Offset at which to read.
data – [in] Buffer into which to read.
size – [in] Size to read.
req – [out] Optional request to make the call asynchronous.

void read(const RegionID &region, const std::vector<std::pair<size_t, size_t>> &regionOffsetSizes, char *data, AsyncRequest *req = nullptr) const

Read non-contiguous part of a region into the provided non-contiguous, local memory buffer.

Note: the size to read is computed as the sum of the sizes in the regionOffsetSizes parameter.

Parameters

region – [in] Region to read.
regionOffsetSizes – [in] Offset/size pairs in the region to read.
data – [in] Buffer into which to read.
req – [out] Optional request to make the call asynchronous.

void read(const RegionID &region, size_t regionOffset, thallium::bulk data, const std::string &address, size_t bulkOffset, size_t size, AsyncRequest *req = nullptr) const

Read part of a region into the provided local memory buffer.

Parameters

region – [in] Region to read.
regionOffset – [in] Offset at which to read.
data – [in] Bulk handle into which to push the data.
address – [in] Address of the process owning the bulk handle.
bulkOffset – [in] Offset at which to push in the provided bulk handle.
size – [in] Size to read.
req – [out] Optional request to make the call asynchronous.

void read(const RegionID &region, const std::vector<std::pair<size_t, size_t>> &regionOffsetSizes, thallium::bulk data, const std::string &address, size_t bulkOffset, AsyncRequest *req = nullptr) const

Read part of a region into the provided local memory buffer.

Note: the size to read is computed as the sum of the sizes in the regionOffsetSizes parameter.

Parameters

region – [in] Region to read.
regionOffsetSizes – [in] Offset/size pairs in the region to read.
data – [in] Bulk handle into which to push the data.
address – [in] Address of the process owning the bulk handle.
bulkOffset – [in] Offset at which to push in the provided bulk handle.
req – [out] Optional request to make the call asynchronous.

void erase(const RegionID &region, AsyncRequest *req = nullptr) const

Erase a region.

Parameters

region – [in] Region to erase.
req – [out] Optional request to make the call asynchronous.

void setEagerWriteThreshold(size_t size): Set the threshold for eager writes (default is 2048).

void setEagerReadThreshold(size_t size): Set the threshold for eager reads (default is 2048).

warabi::TransferManager

class TransferManager

Interface for transfer managers. To build a new TransferManager, implement a class MyType that inherits from TransferManager, and put WARABI_REGISTER_TRANSFER_MANAGER(mytype, MyType); in a cpp file that includes your class’ header file.

Your class should also have a static functions to create a TransferManager:

std::unique_ptr<TransferManager> create(const json& config)

Public Functions

TransferManager() = default: Constructor.

TransferManager(TransferManager&&) = default: Move-constructor.

TransferManager(const TransferManager&) = default: Copy-constructor.

TransferManager &operator=(TransferManager&&) = default: Move-assignment operator.

TransferManager &operator=(const TransferManager&) = default: Copy-assignment operator.

virtual ~TransferManager() = default: Destructor.

inline const std::string &name() const: Return the name of class.

virtual std::string getConfig() const = 0: Returns a JSON-formatted configuration string.

virtual Result<bool> pull(WritableRegion &region, const std::vector<std::pair<size_t, size_t>> &regionOffsetSizes, thallium::bulk data, thallium::endpoint address, size_t bulkOffset, bool persist) = 0

Pull data from the bulk handle and into the region wrapped by the WritableRegion object.

Parameters

region – [in] Region to pull into.
regionOffsetSizes – [in] ranges in the region to pull into.
data – [in] Remote bulk to pull from.
address – [in] Address ot the remote process.
bulkOffset – [in] Offset in the bulk handle.
persist – [in] Whether to persist the data.

Returns

a Result<bool> indicating the result of the operation.

virtual Result<bool> push(ReadableRegion &region, const std::vector<std::pair<size_t, size_t>> &regionOffsetSizes, thallium::bulk data, thallium::endpoint address, size_t bulkOffset) = 0

Push data from the ReadableRegion and to the remote bulk handle.

Parameters

region – [in] Region to push from.
regionOffsetSizes – [in] ranges in the region to pull into.
data – [in] Remote bulk to push to.
address – [in] Address ot the remote process.
bulkOffset – [in] Offset in the bulk handle.

Returns

a Result<bool> indicating the result of the operation.

warabi::TransferManagerFactory

class TransferManagerFactory

The TransferManagerFactory contains functions to create TransferManagers.

Public Static Functions

static Result<bool> validateConfig(const std::string &name, const json &config)

Validate that the JSON configuration has the expected schema.

Parameters

name – Type of transfer manager to use.
config – Configuration object to validate.

Returns

true if the JSON configuration is valid, false otherwise.

static Result<std::unique_ptr<TransferManager>> createTransferManager(const std::string &name, const thallium::engine &engine, const json &config)

Creates a TransferManager and returns a unique_ptr to the created instance.

Parameters

name – Name of the type to use.
engine – Thallium engine.
config – Configuration object to pass to the create function.

Returns

a unique_ptr to the created TransferManager.

warabi::WritableRegion

class WritableRegion : public warabi::Region 

Public Functions

virtual Result<bool> write(const std::vector<std::pair<size_t, size_t>> &regionOffsetSizes, thallium::bulk data, const thallium::endpoint &address, size_t bulkOffset, bool persist) = 0

See also

TopicHandle::write

virtual Result<bool> write(const std::vector<std::pair<size_t, size_t>> &regionOffsetSizes, const void *data, bool persist) = 0

See also

TopicHandle::write

virtual Result<bool> persist(const std::vector<std::pair<size_t, size_t>> &regionOffsetSizes) = 0

See also

TopicHandle::persist