---

EloqStore

EloqStore is a high-performance hybrid-tier key-value storage engine that combines object storage (S3-compatible) with local NVMe SSDs to deliver exceptional write throughput and sub-millisecond read latency. Built in C++ with a focus on production-grade reliability and performance.

EloqStore serves as the foundational storage layer for EloqData's database products (EloqKV, EloqDoc, EloqSQL, enabling SSD-based workloads to achieve memory-like latency characteristics while maintaining durability and cost efficiency.

For example, EloqKV on EloqStore delivers 4x higher disk access throughput compared to Redis's memory access, with 10x cost reduction, making it a viable drop-in replacement for Redis.

📋 Requirements

EloqStore uses io_uring for high-performance asynchronous I/O operations. As a result, it requires:

• Minimum OS: Ubuntu 24.04 or later (or equivalent Linux kernel 6.8+)
• Kernel: Linux kernel 6.8+ (required for full io_uring support)

🏗️ Architecture

EloqStore implements a multi-tier storage architecture:

• Hot Tier: In-memory B-tree index with non-leaf nodes cached for O(log n) point lookups
• Warm Tier: Local NVMe SSD for frequently accessed data with single I/O point reads
• Cold Tier: Object storage (S3-compatible) as the primary durable storage backend

The engine uses a copy-on-write (COW) B-tree structure that enables lock-free reads during batch writes, eliminating read-write contention and ensuring consistent tail latency.

✨ Core Features

• Batch Write Optimization: Copy-on-write B-tree enables high-throughput batch writes without blocking concurrent reads. MVCC-based design eliminates lock contention and provides predictable write amplification.

• Point Read Tail Latency Optimization: In-memory B-tree non-leaf nodes ensure exactly one disk I/O per point read on cold data, delivering consistent P99 latency with deterministic I/O patterns.

• Object Storage as Primary Storage: S3-compatible object storage backend provides 11 9's durability, unlimited scalability, and high availability with intelligent caching for low-latency access.

• Zero-Copy Snapshots: Copy-on-write semantics enable O(1) snapshot creation without data duplication, supporting point-in-time recovery and consistent backups.

• Agent Branching: Instant branch creation for isolated data views in AI/ML workloads, enabling experimentation and multi-tenant isolation without data duplication.

🚀 Deployment Models

Self-Hosted

Deploy on-premises or in your cloud infrastructure with full control over data and operations. EloqStore is available under dual license: BSL 2.0 or AGPL v3. Complete source code access.

Cloud & Enterprise

Managed service EloqCloud offering with:

• Serverless architecture (no infrastructure management)
• High performance and fast scaling
• Enterprise-grade security and compliance

🌐 Ecosystem

EloqStore exposes a C++ API for direct integration. EloqStore powers different kinds of database products including:

• Redis/Valkey Protocol: Drop-in replacement for Redis with persistent storage
• MongoDB Wire Protocol: Document database with MongoDB API compatibility
• SQL Interface: Relational database with MySQL API compatibility
• Vector Search: Native vector indexing and similarity search capabilities

🔨 Compile

Install Dependencies

For Ubuntu 24.04, you can install all required dependencies using the provided script:

bash scripts/install_dependency_ubuntu2404.sh

This script installs all necessary dependencies including:

• Build tools (CMake, GCC, Ninja)
• System libraries (Boost, glog, jsoncpp, liburing, zstd, etc.)
• AWS SDK C++ (S3)
• Testing framework (Catch2)
• Additional libraries (Abseil, gRPC, etc.)

Note: This script requires sudo privileges and may take several minutes to complete.

Debug Mode

mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Debug
cmake --build . -j8
cd ..

Release Mode

mkdir Release
cd Release
cmake .. -DCMAKE_BUILD_TYPE=Release
cmake --build . -j8
cd ..

🧪 Testing

Run Unit Tests

EloqStore requires an S3-compatible object storage backend for testing. MinIO is recommended for local development and testing.

1. Download and start MinIO:

# Download MinIO
wget https://dl.min.io/server/minio/release/linux-amd64/minio
chmod +x minio

# Start MinIO server (runs on port 9000 by default)
./minio server /tmp/minio-data --address :9900 --console-address :9901

2. Run unit tests:

ctest --test-dir build/tests/

Note: Ensure MinIO is running before executing the tests. The tests will connect to MinIO running on 127.0.0.1:9900 by default.

Benchmark

# An example to run eloqstore with 10GB data, with each record 1K.
# load
./build/benchmark/simple_bench --kvoptions=./benchmark/opts_append.ini --workload=write-read --kv_size=1024 --batch_size=20000 --max_key=10000000 --read_per_part=4 --partitions=1 --load
# run
./build/benchmark/simple_bench --kvoptions=./benchmark/opts_append.ini --workload=write-read --kv_size=1024 --batch_size=20000 --max_key=10000000 --read_per_part=4 --partitions=1

Install Format Tool

bash scripts/format.sh

This script will execute when necessary:

• Install code format tools
• Format the code

The first time this script is executed, it will install:

• libtinfo5
• clang-format(18.1.8)

Note: The main branch has code format checks; please run this script before committing the MR.

🤝 Contributing

We welcome contributions from the developer community!

1. Check CONTRIBUTING.md for development guidelines
2. Review GitHub Issues for planned features
3. Join our Discord for discussions
4. Submit PRs for bug fixes, features, or documentation improvements