Runtime configuration (C++)
While Bedrock configurations are typically provided at startup via JSON files, Bedrock also provides a powerful C++ API for manipulating configurations at runtime. This allows you to:
Add and remove Argobots pools and execution streams
Load modules dynamically
Register new providers
Change provider pools
Migrate provider state between services
Snapshot and restore provider state
This tutorial covers how to use these capabilities from C++ code.
Client and ServiceHandle
To manipulate a Bedrock service at runtime, you need to create a bedrock::Client
and obtain a bedrock::ServiceHandle to the target service.
/*
* (C) 2024 The University of Chicago
*
* See COPYRIGHT in top-level directory.
*/
#include <bedrock/Client.hpp>
#include <bedrock/ServiceHandle.hpp>
#include <iostream>
int main(int argc, char** argv) {
if(argc != 2) {
std::cerr << "Usage: " << argv[0] << " <server_address>" << std::endl;
return 1;
}
try {
// Initialize Thallium engine
thallium::engine engine("na+sm", THALLIUM_CLIENT_MODE);
// Create Bedrock client
bedrock::Client client(engine);
// Create service handle
// Provider ID 0 is the default Bedrock provider
bedrock::ServiceHandle service = client.makeServiceHandle(argv[1], 0);
std::cout << "Connected to Bedrock service" << std::endl;
// Get configuration
std::string config;
service.getConfig(&config);
std::cout << "Current configuration:" << std::endl;
std::cout << config << std::endl;
} catch(const bedrock::Exception& ex) {
std::cerr << "Bedrock error: " << ex.what() << std::endl;
return 1;
}
return 0;
}
The ServiceHandle is your interface for all runtime configuration operations.
Loading modules at runtime
You can load new Bedrock modules (shared libraries) into a running service:
/*
* (C) 2024 The University of Chicago
*
* See COPYRIGHT in top-level directory.
*/
#include <bedrock/Client.hpp>
#include <bedrock/ServiceHandle.hpp>
#include <iostream>
int main(int argc, char** argv) {
if(argc != 3) {
std::cerr << "Usage: " << argv[0] << " <server_address> <module_path>" << std::endl;
return 1;
}
const char* server_addr = argv[1];
const char* module_path = argv[2];
try {
thallium::engine engine("na+sm", THALLIUM_CLIENT_MODE);
bedrock::Client client(engine);
bedrock::ServiceHandle service = client.makeServiceHandle(server_addr, 0);
std::cout << "Loading module: " << module_path << std::endl;
// Load the module
service.loadModule(module_path);
std::cout << "Module loaded successfully" << std::endl;
} catch(const bedrock::Exception& ex) {
std::cerr << "Error: " << ex.what() << std::endl;
return 1;
}
return 0;
}
This is useful for dynamically extending a service’s capabilities without requiring a restart.
Managing pools and execution streams
You can add and remove Argobots pools and execution streams at runtime.
Adding a pool
/*
* (C) 2024 The University of Chicago
*
* See COPYRIGHT in top-level directory.
*/
#include <bedrock/Client.hpp>
#include <bedrock/ServiceHandle.hpp>
#include <iostream>
int main(int argc, char** argv) {
if(argc != 2) {
std::cerr << "Usage: " << argv[0] << " <server_address>" << std::endl;
return 1;
}
try {
thallium::engine engine("na+sm", THALLIUM_CLIENT_MODE);
bedrock::Client client(engine);
bedrock::ServiceHandle service = client.makeServiceHandle(argv[1], 0);
// Define pool configuration as JSON
std::string pool_config = R"(
{
"name": "my_dynamic_pool",
"kind": "fifo_wait",
"access": "mpmc"
}
)";
std::cout << "Adding Argobots pool..." << std::endl;
// Add the pool
service.addPool(pool_config);
std::cout << "Pool added successfully" << std::endl;
// Verify by querying configuration
std::string config_str;
service.getConfig(&config_str);
auto config = nlohmann::json::parse(config_str);
auto& pools = config["margo"]["argobots"]["pools"];
std::cout << "Current pools:" << std::endl;
for(auto& pool : pools) {
std::cout << " - " << pool["name"] << std::endl;
}
} catch(const bedrock::Exception& ex) {
std::cerr << "Error: " << ex.what() << std::endl;
return 1;
}
return 0;
}
The pool configuration is provided as a JSON string. After adding a pool, it becomes available for use by providers and execution streams.
Adding an execution stream
/*
* (C) 2024 The University of Chicago
*
* See COPYRIGHT in top-level directory.
*/
#include <bedrock/Client.hpp>
#include <bedrock/ServiceHandle.hpp>
#include <iostream>
int main(int argc, char** argv) {
if(argc != 2) {
std::cerr << "Usage: " << argv[0] << " <server_address>" << std::endl;
return 1;
}
try {
thallium::engine engine("na+sm", THALLIUM_CLIENT_MODE);
bedrock::Client client(engine);
bedrock::ServiceHandle service = client.makeServiceHandle(argv[1], 0);
// First, add a pool for the xstream to use
std::string pool_config = R"(
{
"name": "xstream_pool",
"kind": "fifo_wait",
"access": "mpmc"
}
)";
service.addPool(pool_config);
std::cout << "Created pool for execution stream" << std::endl;
// Define execution stream configuration
std::string xstream_config = R"(
{
"name": "my_dynamic_xstream",
"cpubind": 1,
"scheduler": {
"type": "basic_wait",
"pools": ["xstream_pool"]
}
}
)";
std::cout << "Adding execution stream..." << std::endl;
// Add the execution stream
service.addXstream(xstream_config);
std::cout << "Execution stream added successfully" << std::endl;
} catch(const bedrock::Exception& ex) {
std::cerr << "Error: " << ex.what() << std::endl;
return 1;
}
return 0;
}
Removing pools and execution streams
// Remove a pool (must not be in use)
service.removePool("my_dynamic_pool");
// Remove an execution stream
service.removeXstream("my_dynamic_xstream");
Warning
You cannot remove a pool or execution stream that is currently in use by a provider or scheduler. Ensure all dependents have been removed first.
Adding providers at runtime
One of the most powerful features is the ability to register new providers dynamically:
/*
* (C) 2024 The University of Chicago
*
* See COPYRIGHT in top-level directory.
*/
#include <bedrock/Client.hpp>
#include <bedrock/ServiceHandle.hpp>
#include <iostream>
int main(int argc, char** argv) {
if(argc != 2) {
std::cerr << "Usage: " << argv[0] << " <server_address>" << std::endl;
return 1;
}
try {
thallium::engine engine("na+sm", THALLIUM_CLIENT_MODE);
bedrock::Client client(engine);
bedrock::ServiceHandle service = client.makeServiceHandle(argv[1], 0);
// Define provider description
std::string provider_desc = R"(
{
"name": "my_dynamic_provider",
"type": "yokan",
"provider_id": 100,
"dependencies": {
"pool": "__primary__"
},
"config": {
"database": {
"type": "map"
}
}
}
)";
std::cout << "Adding provider..." << std::endl;
uint16_t provider_id;
service.addProvider(provider_desc, &provider_id);
std::cout << "Provider added with ID: " << provider_id << std::endl;
// Verify by querying configuration
std::string config_str;
service.getConfig(&config_str);
auto config = nlohmann::json::parse(config_str);
auto& providers = config["providers"];
std::cout << "\nCurrent providers:" << std::endl;
for(auto& provider : providers) {
std::cout << " - " << provider["name"]
<< " (type=" << provider["type"]
<< ", id=" << provider["provider_id"] << ")" << std::endl;
}
} catch(const bedrock::Exception& ex) {
std::cerr << "Error: " << ex.what() << std::endl;
return 1;
}
return 0;
}
The provider description follows the same format as in the JSON configuration file. Dependencies can reference existing pools, execution streams, and other providers.
Changing provider pools
You can change which Argobots pool a provider uses for handling RPCs:
/*
* (C) 2024 The University of Chicago
*
* See COPYRIGHT in top-level directory.
*/
#include <bedrock/Client.hpp>
#include <bedrock/ServiceHandle.hpp>
#include <iostream>
int main(int argc, char** argv) {
if(argc != 4) {
std::cerr << "Usage: " << argv[0] << " <server_address> <provider_name> <new_pool>" << std::endl;
return 1;
}
const char* server_addr = argv[1];
const char* provider_name = argv[2];
const char* new_pool = argv[3];
try {
thallium::engine engine("na+sm", THALLIUM_CLIENT_MODE);
bedrock::Client client(engine);
bedrock::ServiceHandle service = client.makeServiceHandle(server_addr, 0);
std::cout << "Changing pool for provider '" << provider_name
<< "' to '" << new_pool << "'" << std::endl;
// Change the provider's pool
service.changeProviderPool(provider_name, new_pool);
std::cout << "Pool changed successfully" << std::endl;
} catch(const bedrock::Exception& ex) {
std::cerr << "Error: " << ex.what() << std::endl;
return 1;
}
return 0;
}
This is useful for dynamically adjusting resource allocation and load balancing.
Note
Not all providers support changing their pool at runtime. Check the provider’s documentation to see if this feature is supported.
Provider migration
Bedrock supports migrating provider state from one service to another:
/*
* (C) 2024 The University of Chicago
*
* See COPYRIGHT in top-level directory.
*/
#include <bedrock/Client.hpp>
#include <bedrock/ServiceHandle.hpp>
#include <iostream>
int main(int argc, char** argv) {
if(argc != 5) {
std::cerr << "Usage: " << argv[0]
<< " <source_address> <provider_name> <dest_address> <dest_provider_id>"
<< std::endl;
return 1;
}
const char* source_addr = argv[1];
const char* provider_name = argv[2];
const char* dest_addr = argv[3];
uint16_t dest_provider_id = std::atoi(argv[4]);
try {
thallium::engine engine("na+sm", THALLIUM_CLIENT_MODE);
bedrock::Client client(engine);
bedrock::ServiceHandle service = client.makeServiceHandle(source_addr, 0);
std::cout << "Migrating provider '" << provider_name << "'" << std::endl;
std::cout << " From: " << source_addr << std::endl;
std::cout << " To: " << dest_addr << " (provider ID " << dest_provider_id << ")" << std::endl;
// Migration configuration (provider-specific)
std::string migration_config = "{}";
// Migrate provider state
// remove_source=false means keep the source after migration
service.migrateProvider(
provider_name,
dest_addr,
dest_provider_id,
migration_config,
false // remove_source
);
std::cout << "Migration completed successfully" << std::endl;
} catch(const bedrock::Exception& ex) {
std::cerr << "Error: " << ex.what() << std::endl;
return 1;
}
return 0;
}
Migration is useful for:
Load balancing across services
Draining a service before maintenance
Elastic scaling of services
The migration_config is provider-specific and controls how migration
is performed. The remove_source flag determines whether the source
provider’s state is deleted after migration.
Note
Provider migration requires that:
The destination address has a provider of the same type
The provider supports migration (implements the
migratemethod)The migration configuration is valid for the provider type
Snapshotting provider state
You can snapshot a provider’s state to persistent storage:
/*
* (C) 2024 The University of Chicago
*
* See COPYRIGHT in top-level directory.
*/
#include <bedrock/Client.hpp>
#include <bedrock/ServiceHandle.hpp>
#include <iostream>
int main(int argc, char** argv) {
if(argc != 4) {
std::cerr << "Usage: " << argv[0]
<< " <server_address> <provider_name> <snapshot_path>"
<< std::endl;
return 1;
}
const char* server_addr = argv[1];
const char* provider_name = argv[2];
const char* snapshot_path = argv[3];
try {
thallium::engine engine("na+sm", THALLIUM_CLIENT_MODE);
bedrock::Client client(engine);
bedrock::ServiceHandle service = client.makeServiceHandle(server_addr, 0);
std::cout << "Snapshotting provider '" << provider_name << "'" << std::endl;
std::cout << " Destination: " << snapshot_path << std::endl;
// Snapshot configuration (provider-specific)
std::string snapshot_config = "{}";
// Create snapshot
// remove_source=false means keep the provider running after snapshot
service.snapshotProvider(
provider_name,
snapshot_path,
snapshot_config,
false // remove_source
);
std::cout << "Snapshot created successfully" << std::endl;
} catch(const bedrock::Exception& ex) {
std::cerr << "Error: " << ex.what() << std::endl;
return 1;
}
return 0;
}
Snapshots can be used for:
Checkpointing service state
Backing up data before shutdown
Cloning provider state
Restoring from snapshots
/*
* (C) 2024 The University of Chicago
*
* See COPYRIGHT in top-level directory.
*/
#include <bedrock/Client.hpp>
#include <bedrock/ServiceHandle.hpp>
#include <iostream>
int main(int argc, char** argv) {
if(argc != 4) {
std::cerr << "Usage: " << argv[0]
<< " <server_address> <provider_name> <snapshot_path>"
<< std::endl;
return 1;
}
const char* server_addr = argv[1];
const char* provider_name = argv[2];
const char* snapshot_path = argv[3];
try {
thallium::engine engine("na+sm", THALLIUM_CLIENT_MODE);
bedrock::Client client(engine);
bedrock::ServiceHandle service = client.makeServiceHandle(server_addr, 0);
std::cout << "Restoring provider '" << provider_name << "'" << std::endl;
std::cout << " Source: " << snapshot_path << std::endl;
// Restore configuration (provider-specific)
std::string restore_config = "{}";
// Restore from snapshot
service.restoreProvider(
provider_name,
snapshot_path,
restore_config
);
std::cout << "Provider restored successfully" << std::endl;
} catch(const bedrock::Exception& ex) {
std::cerr << "Error: " << ex.what() << std::endl;
return 1;
}
return 0;
}
Asynchronous operations
All runtime configuration operations support asynchronous execution using
the bedrock::AsyncRequest class:
/*
* (C) 2024 The University of Chicago
*
* See COPYRIGHT in top-level directory.
*/
#include <bedrock/Client.hpp>
#include <bedrock/ServiceHandle.hpp>
#include <bedrock/AsyncRequest.hpp>
#include <iostream>
int main(int argc, char** argv) {
if(argc != 2) {
std::cerr << "Usage: " << argv[0] << " <server_address>" << std::endl;
return 1;
}
try {
thallium::engine engine("na+sm", THALLIUM_CLIENT_MODE);
bedrock::Client client(engine);
bedrock::ServiceHandle service = client.makeServiceHandle(argv[1], 0);
// Create an async request object
bedrock::AsyncRequest request;
// Define provider configuration
std::string provider_desc = R"(
{
"name": "async_provider",
"type": "yokan",
"provider_id": 101,
"config": {
"database": {"type": "map"}
}
}
)";
std::cout << "Issuing asynchronous addProvider request..." << std::endl;
// Issue async request (non-blocking)
uint16_t provider_id;
service.addProvider(provider_desc, &provider_id, &request);
std::cout << "Request issued, continuing other work..." << std::endl;
// Do other work here while the request is being processed
// ...
std::cout << "Waiting for request to complete..." << std::endl;
// Wait for the request to complete
request.wait();
std::cout << "Provider added asynchronously with ID: " << provider_id << std::endl;
} catch(const bedrock::Exception& ex) {
std::cerr << "Error: " << ex.what() << std::endl;
return 1;
}
return 0;
}
Asynchronous operations allow your code to continue working while the configuration change is being applied.
Querying configuration
You can retrieve the current configuration of a service:
/*
* (C) 2024 The University of Chicago
*
* See COPYRIGHT in top-level directory.
*/
#include <bedrock/Client.hpp>
#include <bedrock/ServiceHandle.hpp>
#include <nlohmann/json.hpp>
#include <iostream>
int main(int argc, char** argv) {
if(argc != 2) {
std::cerr << "Usage: " << argv[0] << " <server_address>" << std::endl;
return 1;
}
try {
thallium::engine engine("na+sm", THALLIUM_CLIENT_MODE);
bedrock::Client client(engine);
bedrock::ServiceHandle service = client.makeServiceHandle(argv[1], 0);
// Get the complete configuration
std::string config_str;
service.getConfig(&config_str);
nlohmann::json config = nlohmann::json::parse(config_str);
std::cout << "=== Complete Configuration ===" << std::endl;
std::cout << config.dump(2) << std::endl;
// Extract specific information
std::cout << "\n=== Providers ===" << std::endl;
if(config.contains("providers")) {
for(auto& provider : config["providers"]) {
std::cout << " - " << provider["name"]
<< " (type=" << provider["type"]
<< ", id=" << provider["provider_id"] << ")" << std::endl;
}
}
// Extract pool information
std::cout << "\n=== Argobots Pools ===" << std::endl;
if(config.contains("margo") &&
config["margo"].contains("argobots") &&
config["margo"]["argobots"].contains("pools")) {
for(auto& pool : config["margo"]["argobots"]["pools"]) {
std::cout << " - " << pool["name"]
<< " (kind=" << pool["kind"] << ")" << std::endl;
}
}
} catch(const bedrock::Exception& ex) {
std::cerr << "Error: " << ex.what() << std::endl;
return 1;
}
return 0;
}
This returns the complete configuration as a JSON object, which you can parse and inspect.
Using Jx9 for selective queries
For more complex queries, use Jx9 scripts to extract specific information:
/*
* (C) 2024 The University of Chicago
*
* See COPYRIGHT in top-level directory.
*/
#include <bedrock/Client.hpp>
#include <bedrock/ServiceHandle.hpp>
#include <nlohmann/json.hpp>
#include <iostream>
int main(int argc, char** argv) {
if(argc != 2) {
std::cerr << "Usage: " << argv[0] << " <server_address>" << std::endl;
return 1;
}
try {
thallium::engine engine("na+sm", THALLIUM_CLIENT_MODE);
bedrock::Client client(engine);
bedrock::ServiceHandle service = client.makeServiceHandle(argv[1], 0);
// Jx9 script to extract provider names
std::string jx9_script = R"(
$result = [];
foreach($__config__['providers'] as $provider) {
$result[] = $provider['name'];
}
return $result;
)";
std::cout << "Querying provider names with Jx9..." << std::endl;
// Execute the Jx9 query
std::string result;
service.queryConfig(jx9_script, &result);
// Parse the result
auto provider_names = nlohmann::json::parse(result);
std::cout << "Provider names:" << std::endl;
for(auto& name : provider_names) {
std::cout << " - " << name << std::endl;
}
// Another Jx9 query to count providers by type
std::string count_script = R"(
$counts = [];
foreach($__config__['providers'] as $provider) {
$type = $provider['type'];
if(!array_key_exists($type, $counts)) {
$counts[$type] = 0;
}
$counts[$type] += 1;
}
return $counts;
)";
service.queryConfig(count_script, &result);
auto provider_counts = nlohmann::json::parse(result);
std::cout << "\nProviders by type:" << std::endl;
for(auto& [type, count] : provider_counts.items()) {
std::cout << " - " << type << ": " << count << std::endl;
}
} catch(const bedrock::Exception& ex) {
std::cerr << "Error: " << ex.what() << std::endl;
return 1;
}
return 0;
}
Complete example
Here’s a complete example that demonstrates multiple runtime operations:
/*
* (C) 2024 The University of Chicago
*
* See COPYRIGHT in top-level directory.
*/
#include <bedrock/Client.hpp>
#include <bedrock/ServiceHandle.hpp>
#include <bedrock/AsyncRequest.hpp>
#include <nlohmann/json.hpp>
#include <iostream>
int main(int argc, char** argv) {
if(argc != 2) {
std::cerr << "Usage: " << argv[0] << " <server_address>" << std::endl;
return 1;
}
try {
thallium::engine engine("na+sm", THALLIUM_CLIENT_MODE);
bedrock::Client client(engine);
bedrock::ServiceHandle service = client.makeServiceHandle(argv[1], 0);
std::cout << "=== Runtime Configuration Example ===" << std::endl;
// 1. Query initial configuration
std::cout << "\n1. Querying initial configuration..." << std::endl;
std::string config_str;
service.getConfig(&config_str);
auto config = nlohmann::json::parse(config_str);
size_t initial_providers = config["providers"].size();
std::cout << " Initial providers: " << initial_providers << std::endl;
// 2. Add a new pool
std::cout << "\n2. Adding new Argobots pool..." << std::endl;
std::string pool_config = R"(
{
"name": "runtime_pool",
"kind": "fifo_wait",
"access": "mpmc"
}
)";
service.addPool(pool_config);
std::cout << " Pool 'runtime_pool' added" << std::endl;
// 3. Add a new execution stream
std::cout << "\n3. Adding new execution stream..." << std::endl;
std::string xstream_config = R"(
{
"name": "runtime_xstream",
"scheduler": {
"type": "basic_wait",
"pools": ["runtime_pool"]
}
}
)";
service.addXstream(xstream_config);
std::cout << " Execution stream 'runtime_xstream' added" << std::endl;
// 4. Add a provider using the new pool
std::cout << "\n4. Adding provider with new pool..." << std::endl;
std::string provider_desc = R"(
{
"name": "runtime_provider",
"type": "yokan",
"provider_id": 200,
"dependencies": {
"pool": "runtime_pool"
},
"config": {
"database": {"type": "map"}
}
}
)";
uint16_t provider_id;
bedrock::AsyncRequest request;
service.addProvider(provider_desc, &provider_id, &request);
request.wait();
std::cout << " Provider added with ID: " << provider_id << std::endl;
// 5. Query updated configuration
std::cout << "\n5. Querying updated configuration..." << std::endl;
service.getConfig(&config_str);
config = nlohmann::json::parse(config_str);
size_t final_providers = config["providers"].size();
std::cout << " Final providers: " << final_providers << std::endl;
// 6. Use Jx9 to list all pools
std::cout << "\n6. Listing all pools via Jx9..." << std::endl;
std::string jx9_script = R"(
$result = [];
if(array_key_exists('margo', $__config__) &&
array_key_exists('argobots', $__config__['margo']) &&
array_key_exists('pools', $__config__['margo']['argobots'])) {
foreach($__config__['margo']['argobots']['pools'] as $pool) {
$result[] = $pool['name'];
}
}
return $result;
)";
std::string result;
service.queryConfig(jx9_script, &result);
auto pools = nlohmann::json::parse(result);
std::cout << " Pools:" << std::endl;
for(auto& pool : pools) {
std::cout << " - " << pool << std::endl;
}
std::cout << "\n=== Runtime configuration operations completed ===" << std::endl;
} catch(const bedrock::Exception& ex) {
std::cerr << "Error: " << ex.what() << std::endl;
return 1;
}
return 0;
}
Compilation
To compile code using the Bedrock client API, link against the bedrock::client
CMake target:
find_package(bedrock REQUIRED)
add_executable(my_client my_client.cpp)
target_link_libraries(my_client bedrock::client)
Or use pkg-config:
$ g++ -std=c++17 my_client.cpp -o my_client \\
$(pkg-config --cflags --libs bedrock-client)
Error handling
The Bedrock client API throws bedrock::Exception on errors:
#include <bedrock/Exception.hpp>
try {
service.addProvider(description);
} catch (const bedrock::Exception& ex) {
std::cerr << "Bedrock error: " << ex.what() << std::endl;
}
Always wrap Bedrock operations in try-catch blocks for robust error handling.