[RFC] Persistent Storage Architecture for Stratum V2 Share Accounting

[average-gary]

Overview

Currently, Stratum V2 share accounting in SRI is purely in-memory, with share accounting logic contained within the channels-sv2 crate. This RFC proposes different architectural approaches to add persistent storage capabilities for share accounting data, block discovery records, and mining analytics while maintaining the existing clean separation of concerns.

Current Baseline (origin/main on 8e4cb26)

What exists today:

• Share accounting logic in protocols/v2/channels-sv2/src/server/share_accounting.rs
• Pure in-memory ShareAccounting struct with no persistence
• Share validation, duplicate detection, and batch acknowledgment logic
• Integration points in StandardChannel and ExtendedChannel
• No storage backend infrastructure
• Data is lost on pool restart/crash and when flushed

Current ShareAccounting capabilities:

• Tracks per-channel statistics (shares accepted, work sum, best difficulty)
• Duplicate share detection via in-memory HashSet
• Batch acknowledgment triggering based on configurable batch sizes
• Chain tip management for share validation

Limitations of current approach:

• ❌ No persistence
• ❌ No historical analytics or reporting
• ❌ No block discovery tracking beyond logs
• ❌ No audit trail for shares or pool performance analysis

Proposed Storage Architecture

Option 1: Trait-Based Storage Integration

Description: Add storage traits to the channels-sv2 crate with feature-gated integration directly in ShareAccounting.

Pros:

• Minimal disruption to existing share accounting logic
• Zero-cost abstraction when storage disabled
• Direct integration at the point of share processing
• Maintains existing API compatibility

Cons:

• Couples storage concerns to protocol crate
• May bloat channels-sv2 over time
• SRI team maintains storage trait definitions

Implementation approach:

  // In channels-sv2/src/server/share_accounting.rs

  #[cfg(feature = "storage")]
  pub mod storage {
      use async_trait::async_trait;

      #[async_trait]
      pub trait ShareAccountingStorage: Send + Sync {
          async fn store_share_record(&mut self, record: &ShareRecord) -> StorageResult<()>;
          async fn store_block_record(&mut self, record: &BlockRecord) -> StorageResult<()>;
          async fn is_share_hash_seen(&self, channel_id: &str, share_hash: &str) -> StorageResult<bool>;
          async fn get_share_accounting(&self, channel_id: &str) -> StorageResult<Option<ShareAccountingData>>;
          // ... other methods
      }
  }

  impl ShareAccounting {
      #[cfg(feature = "storage")]
      pub fn new_with_storage(
          share_batch_size: usize,
          storage: Arc<RwLock<dyn storage::ShareAccountingStorage>>
      ) -> Self { /* ... */ }

      pub fn update_share_accounting(&mut self, /* ... */) {
          // Existing logic...

          #[cfg(feature = "storage")]
          if let Some(storage) = &self.storage {
              // Async storage hook
          }
      }
  }

Option 2: Dedicated Storage Service Role

Description: Create a new role (roles/storage/) that runs as a separate service, receiving storage events via async channels or network protocols.

Pros:

• Complete separation of concerns
• Could serve multiple pool instances
• Independent scaling and deployment
• Storage logic completely outside protocol crates

Cons:

• Significant architectural complexity
• Network/IPC overhead for every share
• Reliability concerns (storage service failure)
• Much more complex implementation and debugging

Implementation approach:

// New roles/storage/ crate

 pub struct StorageService {
     receiver: mpsc::Receiver<StorageEvent>,
     backends: HashMap<String, Box<dyn StorageBackend>>,
 }

 pub enum StorageEvent {
     ShareSubmitted { channel_id: String, share_data: ShareRecord },
     BlockFound { channel_id: String, block_data: BlockRecord },
     BatchAcknowledged { channel_id: String, batch_data: BatchRecord },
 }

 // In channels-sv2, ShareAccounting sends events:
 impl ShareAccounting {
     pub fn update_share_accounting(&mut self, /* ... */) {
         // Existing logic...

         if let Some(sender) = &self.storage_sender {
             sender.try_send(StorageEvent::ShareSubmitted { /* ... */ });
         }
     }
 }

There may even be other places we could hook into for persisting the data but share_accounting.rs already has access to data needed to store everything holistically.

Example Data Structures to Persist

  pub struct ShareRecord {
      pub channel_id: String,
      pub user_identity: String,
      pub sequence_number: u32,
      pub share_work: u64,
      pub share_hash: String,
      pub timestamp: u64,
  }

  pub struct BlockRecord {
      pub channel_id: String,
      pub user_identity: String,
      pub sequence_number: u32,
      pub template_id: Option<u64>,
      pub coinbase: Vec<u8>,
      pub block_hash: String,
      pub difficulty: f64,
      pub timestamp: u64,
  }

---

FWIW, I am partial to Option 1 as it's the shortest path to completion but don't want to build in the wrong direction. :)

[Shourya742]

I had a slightly different approach in mind. For share accounting, we can define a Persistence trait that abstracts over different storage backends. The default implementation would use the in-memory backend from channel_sv2, while downstream applications could provide their own implementations backed by a database or another persistence mechanism. The Persistence trait itself would remain completely storage-agnostic, ensuring that share accounting logic does not depend on whether the backend is in-memory, database-driven, or something else.

[average-gary]

Would the Persistence trait be within the channel_sv2 crate then?

[Shourya742]

Yes, and channel_sv2’s current in-memory logic will implement that trait to provide persistence capabilities. Downstream applications can override it with their own persistence layer.

[average-gary]

Excellent! Option 1 seemed like the correct way to implement but I wanted to garner feedback. Thank you.