C++ API Documentation

yokan::BasicUserMem

Warning

doxygenclass: Cannot find class “yokan::BasicUserMem” in doxygen xml output for project “yokan” from directory: /home/docs/checkouts/readthedocs.org/user_builds/mochi/checkouts/latest/docs/source/yokan/doxygen

yokan::BitField

Warning

doxygenclass: Cannot find class “yokan::BitField” in doxygen xml output for project “yokan” from directory: /home/docs/checkouts/readthedocs.org/user_builds/mochi/checkouts/latest/docs/source/yokan/doxygen

yokan::Client

class Client

yokan::Collection

class Collection

yokan::Database

class Database

yokan::DatabaseFactory

class DatabaseFactory

The DatabaseFactory is used by the provider to build key/value store instances of various types.

Public Static Functions

static inline Status makeDatabase(const std::string &backendType, const std::string &jsonConfig, DatabaseInterface **kvs)

Create a DatabaseInterface object of a specified type and return a pointer to it. It is the respondibility of the user to call delete on this pointer.

If the function fails, it will return a Status error.

Parameters

  • backendType – Name of the backend.

  • jsonConfig – Json-formatted configuration string.

  • kvs – Created DatabaseInterface pointer.

Returns

Status.

static inline bool hasBackendType(const std::string &backendType)

Check if the backend type is available in the factory.

Parameters

backendType – Name of the backend.

Returns

whether the backend type is available.

yokan::DatabaseInterface

class DatabaseInterface

Abstract embedded database object.

Public Functions

virtual std::string type() const = 0

Get the name of the backend (e.g. “map”).

Returns

The name of the backend.

virtual std::string config() const = 0

Get the internal configuration as a JSON-formatted string.

Returns

Backend configuration.

virtual void destroy() = 0

Destroy the resources (files, etc.) associated with the database.

inline virtual bool supportsMode(int32_t mode) const

Check if the backend supports the specified mode.

Parameters

mode – Mode.

Returns

true/false.

virtual bool isSorted() const = 0

Check if the backend is sorted (list functions will return keys in some defined order, either alphabetical or custom).

Returns

true/false.

inline virtual Status count(int32_t mode, uint64_t *c) const

Get the number of key/value pairs stored.

Parameters

  • mode – Mode.

  • c – Result.

Returns

Status.

inline virtual Status exists(int32_t mode, const UserMem &keys, const BasicUserMem<size_t> &ksizes, BitField &b) const

Check if the provided keys exist. The keys are packed into a single buffer. ksizes provides a pointer to the memory holding the key sizes. The number of keys is conveyed by ksizes.size and b.size, which should be equal (otherwise Status::InvalidArg is returned).

Parameters

  • mode – Mode.

  • keys – Packed keys.

  • ksizes – Packed key sizes.

  • b – Memory segment holding booleans indicating whether each key exists.

Returns

Status.

inline virtual Status length(int32_t mode, const UserMem &keys, const BasicUserMem<size_t> &ksizes, BasicUserMem<size_t> &vsizes) const

Get the size of values associated with the keys. The keys are packed into a single buffer. ksizes provides a pointer to the memory holding the key sizes. vsizes provides a pointer to the memory where the value sizes need to be stored. The number of keys is conveyed by ksizes.size and vsizes.size, which should be equal (otherwise Status::InvalidArg is returned).

Parameters

  • mode – Mode.

  • keys – Packed keys.

  • ksizes – Packed key sizes

  • b – Memory segment to store value sizes.

Returns

Status.

inline virtual Status put(int32_t mode, const UserMem &keys, const BasicUserMem<size_t> &ksizes, const UserMem &vals, const BasicUserMem<size_t> &vsizes)

Put multiple key/value pairs into the database. The keys, ksizes, values, and vsizes are packed into user-provided memory segments. The number of key/value pairs is conveyed by ksizes.size and vsizes.size, which should be equal.

Parameters

  • mode – [in] Mode.

  • keys – [in] Keys to put.

  • ksizes – [in] Key sizes.

  • vals – [in] Values to put.

  • vsizes – [in] Value sizes.

Returns

Status.

inline virtual Status get(int32_t mode, bool packed, const UserMem &keys, const BasicUserMem<size_t> &ksizes, UserMem &vals, BasicUserMem<size_t> &vsizes)

Get values associated of keys. vsizes is used both as input (to know where to place data in vals and how much is available to each value) and as output (to store the actual size of each value).

This function expects (and will not check) that

  • ksizes.size == vsizes.size

  • the sum of ksizes <= keys.size

  • the sum of vsizes <= vals.size

Note: this function is not const because it can potentially call erase() if a YOKAN_MODE_CONSUME is specified, for instance.

Parameters

  • mode – Mode.

  • packed – whether data is packed.

  • keys – Keys to get.

  • ksizes – Size of the keys.

  • vals – Values to get.

  • vsizes – Size of the values.

Returns

Status.

inline virtual Status fetch(int32_t mode, const UserMem &keys, const BasicUserMem<size_t> &ksizes, const FetchCallback &func)

Get values associated of keys and pass them successively to the provided callback function.

Note: this function is not const because it can potentially call erase() if a YOKAN_MODE_CONSUME is specified, for instance.

Parameters

  • mode – Mode.

  • packed – whether data is packed.

  • keys – Keys to get.

  • ksizes – Size of the keys.

  • func – Function to call on the value.

Returns

Status.

inline virtual Status erase(int32_t mode, const UserMem &keys, const BasicUserMem<size_t> &ksizes)

Erase a set of key/value pairs. Keys are packed into a single buffer. The number of keys is conveyed by ksizes.size.

Parameters

  • mode – [in] Mode.

  • keys – [in] Keys to erase.

  • ksizes – [in] Size of the keys.

Returns

Status.

inline virtual Status listKeys(int32_t mode, bool packed, const UserMem &fromKey, const std::shared_ptr<KeyValueFilter> &filter, UserMem &keys, BasicUserMem<size_t> &keySizes) const

This version of listKeys uses a single contiguous buffer to hold all the keys. Their size is stored in the keySizes user-allocated buffer. After a successful call, keySizes.size holds the number of keys read. The function will try to read up to keySizes.size keys.

keySizes is considered an input and an output. As input, it provides the size that should be used for each keys in the keys buffer. As an output, it stores the actual size of each key.

Parameters

  • mode – [in] Mode.

  • packed – [in] Whether data is packed.

  • fromKey – [in] Starting key.

  • filter – [in] Key filter.

  • keys – [out] Resulting keys.

  • keySizes – [inout] Resulting key sizes.

Returns

Status.

inline virtual Status listKeyValues(int32_t mode, bool packed, const UserMem &fromKey, const std::shared_ptr<KeyValueFilter> &filter, UserMem &keys, BasicUserMem<size_t> &keySizes, UserMem &vals, BasicUserMem<size_t> &valSizes) const

Same as listKeys but also returns the values.

inline virtual Status iter(int32_t mode, uint64_t max, const UserMem &fromKey, const std::shared_ptr<KeyValueFilter> &filter, bool ignore_values, const IterCallback &func) const

Iterate through the key/value pairs, calling the provided function on each key/value pair.

Parameters

  • mode – Mode.

  • max – Max number of key/values pairs to list (0 to list everything).

  • fromKey – Starting key.

  • filter – Key filter.

  • func – Function to call on each key.

Returns

Status.

inline virtual Status collCreate(int32_t mode, const char *name)

Create a collection in the underlying database.

Parameters
Returns

Status

inline virtual Status collDrop(int32_t mode, const char *name)

Erase a collection from the underlying database.

Parameters
Returns

Status

inline virtual Status collExists(int32_t mode, const char *name, bool *flag) const

Check if a collection exists in the underlying database.

Parameters

  • mode – Mode

  • name – Collection name

  • flag – Set to true if the collection exists

Returns

Status

inline virtual Status collLastID(int32_t mode, const char *name, yk_id_t *id) const

Get the last id in the collection (i.e. the id that the next document stored will have).

Parameters

  • mode – Mode

  • name – Collection name

  • id – Last id

Returns

Status

inline virtual Status collSize(int32_t mode, const char *name, size_t *size) const

Get the collection size (may be different from LastID if some documents have been erased).

Parameters

  • mode – Mode

  • name – Collection name

  • size – Size

Returns

Status

inline virtual Status docSize(const char *collection, int32_t mode, const BasicUserMem<yk_id_t> &ids, BasicUserMem<size_t> &sizes) const

Get the size of document associated with ids.

Parameters

  • collection – Name of the collection.

  • mode – Mode.

  • ids – Buffer storing ids of the documents.

  • sizes – Buffer in which to put sizes of the documents.

Returns

Status.

inline virtual Status docStore(const char *collection, int32_t mode, const UserMem &documents, const BasicUserMem<size_t> &sizes, BasicUserMem<yk_id_t> &ids)

Store multiple documents into the database.

Parameters

  • collection – [in] Name of the collection.

  • mode – [in] Mode.

  • documents – [in] Buffer containing documents to put.

  • size – [in] Buffer containing the sizes of documents.

  • ids – [in] Buffer to store resulting document ids.

Returns

Status.

inline virtual Status docUpdate(const char *collection, int32_t mode, const BasicUserMem<yk_id_t> &ids, const UserMem &documents, const BasicUserMem<size_t> &sizes)

Update multiple documents in the database.

Parameters

  • collection – [in] Name of the collection.

  • mode – [in] Mode.

  • ids – [in] Buffer containing document ids.

  • documents – [in] Buffer containing documents to put.

  • size – [in] Buffer containing the sizes of documents.

Returns

Status.

inline virtual Status docLoad(const char *collection, int32_t mode, bool packed, const BasicUserMem<yk_id_t> &ids, UserMem &documents, BasicUserMem<size_t> &sizes)

Load documents associated of ids. sizes is used both as input (to know where to place data in documents and how much is available to each document) and as output (to store the actual size of each value).

Parameters

  • collection – Name of the collection.

  • mode – Mode.

  • packed – whether data is packed.

  • ids – Buffer containing document ids.

  • documents – Documents to get.

  • sizes – Size of the documents.

Returns

Status.

inline virtual Status docFetch(const char *collection, int32_t mode, const BasicUserMem<yk_id_t> &ids, const DocFetchCallback &func)

Get the documents associated withe the ids and pass them successively to the provided callback function.

Note: this function is not const because it can potentially call erase() if a YOKAN_MODE_CONSUME is specified, for instance.

Parameters

  • collection – Collection name.

  • mode – Mode.

  • ids – Ids.

  • func – Function to call on the documents.

Returns

Status.

inline virtual Status docErase(const char *collection, int32_t mode, const BasicUserMem<yk_id_t> &ids)

Erase a set of documents.

Parameters

  • collection – [in] Collection name.

  • mode – [in] Mode.

  • ids – [in] Ids of the documents to erase.

Returns

Status.

inline virtual Status docList(const char *collection, int32_t mode, bool packed, yk_id_t from_id, const std::shared_ptr<DocFilter> &filter, BasicUserMem<yk_id_t> &ids, UserMem &documents, BasicUserMem<size_t> &doc_sizes) const

List documents from the collection.

Parameters

  • collection – [in] Collection

  • mode – [in] Mode

  • packed – [in] Whether data is packed in the document buffer

  • from_id – [in] Starting document id

  • filter – [in] Filter

  • ids – [out] Resulting ids

  • documents – [out] Resulting documents

  • doc_sizes – [inout] Buffer sizes / Resulting document sizes

Returns

Status.

inline virtual Status docIter(const char *collection, int32_t mode, uint64_t max, yk_id_t from_id, const std::shared_ptr<DocFilter> &filter, const DocIterCallback &func) const

Iterate through the documents, calling the provided function on each id/document pair.

Parameters

  • collection – Collection

  • mode – Mode.

  • max – Max number of documents to list (0 to list everything).

  • from_id – starting ID.

  • filter – Document filter.

  • func – Function to call on each key.

Returns

Status.

inline virtual Status startMigration(std::unique_ptr<MigrationHandle> &mh)

Set the provided unique_ptr to point to a MigrationHandle that can be used by the provider to retrieve the files used by the database.

yokan::DocFilter

class DocFilter

Abstract class representing a document filter.

Public Functions

virtual bool check(const char *collection, yk_id_t id, const void *doc, size_t docsize) const = 0

Checks whether the document passes the filter.

virtual size_t docSizeFrom(const char *collection, const void *val, size_t vsize) const = 0

Compute the new document size from the provided document after the filter is applied, or an upper bound of the document size. This function will only be applied to documents that pass the check function (i.e. you can assume that check has already been called and returned true).

virtual size_t docCopy(const char *collection, void *dst, size_t max_dst_size, const void *val, size_t vsize) const = 0

Copy the document to the target destination. This copy may be implemented differently depending on the mode, and may alter the content of the document. This function should return the size actually copied.

inline virtual bool shouldStop(const char *collection, const void *doc, size_t size) const

Some filters may be able to tell Yokan that no more documents will be accepted after a certain document. In such a case, the filter can implement the shouldStop function to optimize iterations. Note that this function will be called only if check returns false.

Parameters

  • collection – Collection name

  • doc – Document

  • size – Document size

Returns

whether iteration can stop after this key.

yokan::DocumentStoreMixin

template<typename DB>
class DocumentStoreMixin : public DB

This class allows any Database backend that provides key/value storage functionalities to offer document storage on top. The way to use it is by making your backend implementation class T inherite from DocumentStoreMixin<DatabaseInterface> instead of DatabaseInterface.

yokan::Exception

class Exception : public std::exception

yokan::FilterFactory

class FilterFactory

This class is used to register new filters and for the provider to create filters depending on the mode passed.

yokan::KeyValueFilter

class KeyValueFilter

Abstract class to represent a key/value filter.

Public Functions

virtual bool requiresValue() const = 0

Returns whether the filter requires the value to be loaded when calling check. If not, some backends may chose to call check with a nullptr value and vsize of 0 to avoid loading a value unnecessarily.

virtual bool check(const void *key, size_t ksize, const void *val, size_t vsize) const = 0

Checks whether a key (or key/value pair) passes the filter.

virtual size_t keySizeFrom(const void *key, size_t ksize) const = 0

Compute the new key size from the provided key after the filter is applied, or an upper bound of the key size. This function will only be applied to keys that pass the check function (i.e. you can assume that check has already been called and returned true).

virtual size_t valSizeFrom(const void *val, size_t vsize) const = 0

Compute the new value size from the provided value after the filter is applied, or an upper bound of the value size. This function will only be applied to keys that pass the check function (i.e. you can assume that check has already been called and returned true).

virtual size_t keyCopy(void *dst, size_t max_dst_size, const void *key, size_t ksize) const = 0

Copy the key to the target destination. This copy may be implemented differently depending on the mode, and may alter the content of the key. This function should return the size actually copied.

virtual size_t valCopy(void *dst, size_t max_dst_size, const void *val, size_t vsize) const = 0

Copy the value to the target destination. This copy may be implemented differently depending on the mode, and may alter the content of the value. This function should return the size actually copied.

inline virtual bool shouldStop(const void *key, size_t ksize, const void *val, size_t vsize) const

Some filters may be able to tell Yokan that no more keys will be accepted after a certain key (e.g. the prefix filter). In such a case, the filter can implement the shouldStop function to optimize iterations. Note that this function will be called only if check returns false.

Parameters

  • key – Key

  • ksize – Key size

  • val – Value

  • vsize – Value size

Returns

whether iteration can stop after this key.

yokan::KeyWatcher

class KeyWatcher

The KeyWatcher class is a help class used in backends and implementing a convenient way of watching and being notified for modification in keys. Threads expecting a modification of a key should first add the key using KeyWatcher::addKey. They then have to unlock any lock/mutex that could prevent a writer from adding the key. Finally, they call KeyWatcher::waitKey to wait on the key they have just added. A writer thread can call KeyWatcher::notifyKey to wake up only the threads blocked waiting for a partucular key.

Imporant: the memory address and size of the key passed to waitKey should be the same as that passed to addKey.

yokan::MigrationHandle

class MigrationHandle

A MigrationHandle is an abstract class representing an object that one can request from a database using database.startMigration(). Its roles are (1) to lock all accesses to the database until its destruction (acting like a lock guard), (2) to provide a list of files that need to be migrated, (3) to cleanup any temporary files used during migration upon destruction, and (4) to mark the database as migrated.

Public Functions

virtual ~MigrationHandle() = default

Destructor.

virtual std::string getRoot() const = 0

Get the path relative to which the files returned by getFiles are located.

virtual std::list<std::string> getFiles() const = 0

Get a list of files to migrate. The file names must be relative to the root.

virtual void cancel() = 0

Mark the migration as canceled.

yokan::Provider

class Provider

yokan::ScopedMutex

Warning

doxygenclass: Cannot find class “yokan::ScopedMutex” in doxygen xml output for project “yokan” from directory: /home/docs/checkouts/readthedocs.org/user_builds/mochi/checkouts/latest/docs/source/yokan/doxygen

yokan::ScopedReadLock

Warning

doxygenclass: Cannot find class “yokan::ScopedReadLock” in doxygen xml output for project “yokan” from directory: /home/docs/checkouts/readthedocs.org/user_builds/mochi/checkouts/latest/docs/source/yokan/doxygen

yokan::ScopedWriteLock

Warning

doxygenclass: Cannot find class “yokan::ScopedWriteLock” in doxygen xml output for project “yokan” from directory: /home/docs/checkouts/readthedocs.org/user_builds/mochi/checkouts/latest/docs/source/yokan/doxygen