Database migration

Yokan supports migrating a database from one provider to another using REMI (REsource MIgration component). This is useful for relocating databases, balancing load, or moving data between storage tiers.

Important

Database migration requires:

Yokan compiled with the +remi variant
A REMI receiver provider on the destination
A REMI sender provider on the source
The destination provider initialized with an empty configuration: "{}"

What is database migration?

Database migration physically transfers the database files from a source provider to a destination provider. After migration:

The destination provider takes ownership of the database
The source provider’s database becomes invalid
All key/value pairs and collections are transferred
The database can continue being accessed at the new location

Migration uses REMI to transfer the database files efficiently over the network.

Setting up for migration

Compiling Yokan with REMI support:

spack install mochi-yokan +remi

Provider setup for migration:

The source and destination providers must be configured with REMI support:

#include <remi/remi-server.h>
#include <remi/remi-client.h>

// Register a REMI provider (needed on destination)
remi_provider_t remi_provider;
remi_provider_register(
    mid, ABT_IO_INSTANCE_NULL,
    provider_id, ABT_POOL_NULL, &remi_provider);

// Create a REMI client (needed on source)
remi_client_t remi_client;
remi_client_init(mid, ABT_IO_INSTANCE_NULL, &remi_client);

// Source provider: needs REMI client
struct yk_provider_args args1 = YOKAN_PROVIDER_ARGS_INIT;
args1.remi.client = remi_client;
args1.remi.provider = REMI_PROVIDER_NULL;
yk_provider_register(mid, 1, config, &args1, &source_provider);

// Destination provider: needs REMI provider and EMPTY config
struct yk_provider_args args2 = YOKAN_PROVIDER_ARGS_INIT;
args2.remi.client = REMI_CLIENT_NULL;
args2.remi.provider = remi_provider;
yk_provider_register(mid, 2, "{}", &args2, &dest_provider);
//                            ^^^^
//                            MUST be empty!

The destination provider must be initialized with an empty configuration "{}" because it will receive its database through migration.

Migration API

The migration function is called from the source provider:

yk_return_t yk_provider_migrate_database(
    yk_provider_t provider,                        // Source provider
    const char* dest_addr,                         // Destination address
    uint16_t dest_provider_id,                     // Destination provider ID
    const struct yk_migration_options* options     // Migration options
);

Migration options structure:

struct yk_migration_options {
    const char* new_root;      // New path for database on destination
    const char* extra_config;  // Extra JSON config for destination
    size_t      xfer_size;     // Transfer block size (0 = default)
};

Migration example

Here’s a complete example showing database migration:

#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <margo.h>
#include <remi/remi-server.h>
#include <remi/remi-client.h>
#include <yokan/server.h>
#include <yokan/client.h>
#include <yokan/database.h>

int main(int argc, char** argv)
{
    // Initialize Margo
    margo_instance_id mid = margo_init("ofi+tcp", MARGO_SERVER_MODE, 0, 0);
    assert(mid);

    // Get our own address
    hg_addr_t addr;
    hg_return_t hret = margo_addr_self(mid, &addr);
    assert(hret == HG_SUCCESS);

    char addr_str[128];
    hg_size_t bufsize = 128;
    hret = margo_addr_to_string(mid, addr_str, &bufsize, addr);
    assert(hret == HG_SUCCESS);

    // Register a REMI provider (needed for destination)
    remi_provider_t remi_provider;
    int ret = remi_provider_register(
            mid, ABT_IO_INSTANCE_NULL,
            3, ABT_POOL_NULL, &remi_provider);
    assert(ret == REMI_SUCCESS);

    // Create a REMI client (needed for source)
    remi_client_t remi_client;
    ret = remi_client_init(mid, ABT_IO_INSTANCE_NULL, &remi_client);
    assert(ret == REMI_SUCCESS);

    // Register source Yokan provider with a map database
    yk_provider_t provider1;
    struct yk_provider_args args1 = YOKAN_PROVIDER_ARGS_INIT;
    args1.remi.client = remi_client;
    args1.remi.provider = REMI_PROVIDER_NULL;
    const char* config1 = "{ \"database\": { \"type\": \"map\" } }";
    yk_return_t yret = yk_provider_register(
            mid, 1, config1, &args1, &provider1);
    assert(yret == YOKAN_SUCCESS);

    // Register destination Yokan provider with EMPTY config
    yk_provider_t provider2;
    struct yk_provider_args args2 = YOKAN_PROVIDER_ARGS_INIT;
    args2.remi.client = REMI_CLIENT_NULL;
    args2.remi.provider = remi_provider;
    yret = yk_provider_register(
            mid, 2, "{}", &args2, &provider2);
    //          ^^^^ MUST be empty!
    assert(yret == YOKAN_SUCCESS);

    // Create Yokan client
    yk_client_t client;
    yret = yk_client_init(mid, &client);
    assert(yret == YOKAN_SUCCESS);

    // Get handle to source database
    yk_database_handle_t dbh1;
    yret = yk_database_handle_create(
            client, addr, 1, true, &dbh1);
    assert(yret == YOKAN_SUCCESS);

    // Populate source database with some data
    printf("Populating source database...\n");
    for(int i = 0; i < 10; i++) {
        char key[16], value[16];
        sprintf(key, "key%05d", i);
        sprintf(value, "value%05d", i);
        yret = yk_put(dbh1, 0, key, strlen(key), value, strlen(value));
        assert(yret == YOKAN_SUCCESS);
    }
    printf("  Inserted 10 key/value pairs\n");

    // Set up migration options
    struct yk_migration_options options;
    options.new_root = "/tmp/migrated-database";
    options.extra_config = "{}";
    options.xfer_size = 0;  // Use default

    // Migrate database from provider 1 to provider 2
    printf("\nMigrating database from provider 1 to provider 2...\n");
    yret = yk_provider_migrate_database(
            provider1, addr_str, 2, &options);
    assert(yret == YOKAN_SUCCESS);
    printf("  Migration successful!\n");

    // Try to access source database - should get error now
    printf("\nVerifying source database is now invalid...\n");
    yret = yk_put(dbh1, 0, "test", 4, "data", 4);
    assert(yret == YOKAN_ERR_INVALID_DATABASE);
    printf("  Source database correctly returns INVALID_DATABASE\n");

    // Release old handle
    yk_database_handle_release(dbh1);

    // Get handle to destination database
    yk_database_handle_t dbh2;
    yret = yk_database_handle_create(
            client, addr, 2, true, &dbh2);
    assert(yret == YOKAN_SUCCESS);

    // Verify data was migrated
    printf("\nVerifying migrated data at destination...\n");
    for(int i = 0; i < 10; i++) {
        char key[16], value[16], expected[16];
        sprintf(key, "key%05d", i);
        sprintf(expected, "value%05d", i);
        size_t vsize = 16;
        yret = yk_get(dbh2, 0, key, strlen(key), value, &vsize);
        assert(yret == YOKAN_SUCCESS);
        value[vsize] = '\0';
        assert(strcmp(value, expected) == 0);
    }
    printf("  All 10 key/value pairs successfully retrieved\n");

    // Clean up
    yk_database_handle_release(dbh2);
    yk_client_finalize(client);
    remi_client_finalize(remi_client);
    margo_addr_free(mid, addr);
    margo_finalize(mid);

    printf("\nMigration example completed successfully!\n");
    return 0;
}

This example:

Sets up both source and destination providers with REMI support
Creates a database on the source provider and populates it
Migrates the database to the destination provider
Verifies the source provider’s database is now invalid
Verifies the destination provider now has the migrated data

Migration behavior

After migration:

The source provider returns YOKAN_ERR_INVALID_DATABASE for operations
The destination provider can now access the migrated database
All key/value pairs are transferred
All collections and documents are transferred
The database files are physically moved or copied to the new location

Migration options:

new_root: Specifies the filesystem path where the database will be stored on the destination. This is important when the destination needs the database in a specific location (e.g., on a particular disk or mount point).
extra_config: Additional JSON configuration to apply at the destination. For example, you might change backend parameters while migrating.
xfer_size: Controls the transfer block size for REMI. Setting this to 0 uses REMI’s default. Larger sizes may improve throughput for large databases.

Backend compatibility

Migration support depends on the backend’s ability to have its files transferred. In-memory backends without persistence can be migrated but they will first dump their data into a temporary file, and send that file. Some backends may return YOKAN_ERR_OP_UNSUPPORTED when migration is attempted if migration is not suported..

Using migration with Bedrock

When using Bedrock to manage Yokan providers, the setup is similar but Bedrock handles provider registration. The key requirements remain:

Compile Yokan with +remi
Register a “remi_sender” and/or a “remi_receiver” in your Bedrock configuration, depending on what you intend to support
Ensure the destination Yokan provider has an empty database configuration
Pass REMI sender/receiver as dependencies to the respective Yokan providers

Example Bedrock configuration:

{
    "libraries": [
        "libyokan-bedrock-module.so",
        "libremi-bedrock-module.so"
    ],
    "providers": [
        {
            "name": "sender",
            "type": "remi_sender",
            "provider_id": 1
        },
        {
            "name": "receiver",
            "type": "remi_receiver",
            "provider_id": 2
        },
        {
            "name": "yokan_source",
            "type": "yokan",
            "provider_id": 3,
            "config": {
                "database": {
                    "type": "map"
                }
            },
            "dependencies": {
                "remi_sender": "sender"
            }
        },
        {
            "name": "yokan_dest",
            "type": "yokan",
            "provider_id": 4,
            "config": {},
            "dependencies": {
                "remi_receiver": "receiver"
            }
        }
    ]
}

Error handling

Common errors during migration:

YOKAN_ERR_OP_UNSUPPORTED: Backend doesn’t support migration
YOKAN_ERR_INVALID_DATABASE: Source database is invalid or already migrated
REMI errors: File transfer failures, permission issues
Network errors: Connection failures to destination

After a failed migration attempt, the source database typically remains valid and can be retried.

Limitations

Migration is an all-or-nothing operation (no partial migration)
No built-in support for incremental migration
Source database becomes invalid after migration
Not all backends support migration