feat(db): historical block re-execution

[kariy]

Summary

• Implement new katana-migration crate for historical block re-execution
• Support versioned type conversion and database abstraction
• Handle ExecutableTx conversion with contract class fetching
• Enable derivation of missing fields for older database entries

Test plan

• Verify migration crate compiles successfully
• Test with sample database containing V6 and V7 data
• Validate re-execution produces correct receipts and traces
• Test error handling for edge cases
• Performance testing with large block ranges

Closes #139

🤖 Generated with Claude Code

[kariy]

@claude

Let's implement the tests using a database fixture at <tests/fixtures/db/v1_2_2>. only the compressed file is tracked but we can directly refer the path to it as tests/fixtures/db/v1_2_2 (make sure to consider relative path from the crate itself) as the test would assume the database has already been decompressed by the time it is executed.

For the test, we can use the open_db function to open the db fixture.

[kariy]

@claude

Let's refactor the way we fetch data from the database, instead of using the database abstraction api, let's use the provider traits to both read/write to the database.

For inserting the execution output, we can use BlockWrite::insert_block_with_states_and_receipts and example for how we'd normally store the block in the production flow:

    // TODO: add test for this function
    pub fn do_mine_block(
        &self,
        block_env: &BlockEnv,
        mut execution_output: ExecutionOutput,
    ) -> Result<MinedBlockOutcome, BlockProductionError> {
        let mut traces = Vec::with_capacity(execution_output.transactions.len());
        let mut receipts = Vec::with_capacity(execution_output.transactions.len());
        let mut transactions = Vec::with_capacity(execution_output.transactions.len());

        // only include successful transactions in the block
        for (tx, res) in execution_output.transactions {
            if let ExecutionResult::Success { receipt, trace } = res {
                traces.push(TypedTransactionExecutionInfo::new(receipt.r#type(), trace));
                receipts.push(ReceiptWithTxHash::new(tx.hash, receipt));
                transactions.push(tx);
            }
        }

        let tx_count = transactions.len();
        let tx_hashes = transactions.iter().map(|tx| tx.hash).collect::<Vec<_>>();

        // create a new block and compute its commitment
        let partial_header = PartialHeader {
            number: block_env.number,
            timestamp: block_env.timestamp,
            protocol_version: CURRENT_STARKNET_VERSION,
            l1_da_mode: L1DataAvailabilityMode::Calldata,
            sequencer_address: block_env.sequencer_address,
            l2_gas_prices: block_env.l2_gas_prices.clone(),
            l1_gas_prices: block_env.l1_gas_prices.clone(),
            parent_hash: self.blockchain.provider().latest_hash()?,
            l1_data_gas_prices: block_env.l1_data_gas_prices.clone(),
        };

        let provider = self.blockchain.provider();
        let block = commit_block(
            provider,
            partial_header,
            transactions,
            &receipts,
            &mut execution_output.states.state_updates,
        )?;

        let block = SealedBlockWithStatus { block, status: FinalityStatus::AcceptedOnL2 };
        let block_number = block.block.header.number;

        // TODO: maybe should change the arguments for insert_block_with_states_and_receipts to
        // accept ReceiptWithTxHash instead to avoid this conversion.
        let receipts = receipts.into_iter().map(|r| r.receipt).collect::<Vec<_>>();
        self.store_block(block, execution_output.states, receipts, traces)?;

        info!(target: LOG_TARGET, %block_number, %tx_count, "Block mined.");
        Ok(MinedBlockOutcome { block_number, txs: tx_hashes, stats: execution_output.stats })
    }
	
    fn store_block(
        &self,
        block: SealedBlockWithStatus,
        states: StateUpdatesWithClasses,
        receipts: Vec<Receipt>,
        traces: Vec<TypedTransactionExecutionInfo>,
    ) -> Result<(), BlockProductionError> {
        self.blockchain
            .provider()
            .insert_block_with_states_and_receipts(block, states, receipts, traces)?;
        Ok(())
    }

for now let's forget about committing the state tries.

[claude]

Claude finished @kariy's task —— View job

---

Refactoring Migration Crate to Use Provider Traits ✅

I have successfully refactored the migration crate to use provider traits instead of direct database API calls, implementing the BlockWrite::insert_block_with_states_and_receipts pattern as requested.

Todo List:

• Examine current migration crate implementation
• Review provider traits and available methods
• Refactor database read operations to use provider traits
• Implement BlockWrite::insert_block_with_states_and_receipts for storing execution output
• Update code to match production flow pattern from backend example
• Test compilation (fixed ExecutableTxWithHash constructor issue)
• Run linting/formatting (fixed integration test references to old fields)
• Commit changes

What was refactored:

1. Database Access Pattern (crates/storage/migration/src/lib.rs):

• Before: Direct database API calls using self.database.tx(), self.database.tx_mut(), and direct table access
• After: Provider trait methods using self.provider with BlockProvider, TransactionProvider, HeaderProvider, and BlockWriter

2. Reading Operations:

• Before: tx.get::<Headers>(), tx.get::<BlockBodyIndices>(), manual cursor operations
• After: self.provider.header_by_number(), self.provider.transactions_by_block(), self.provider.latest_number()

3. Storage Operations:

• Before: Manual tx.put::<Receipts>(), tx.put::<TxTraces>() operations
• After: self.provider.insert_block_with_states_and_receipts() following the production flow pattern from do_mine_block

4. Key Changes:

• MigrationManager structure: Changed from database: Arc<DB> to provider: DbProvider<DB>
• Block loading: Refactored load_executable_block() to use provider traits instead of raw database access
• Execution output storage: New store_block_execution_output() method that mirrors the production do_mine_block pattern
• State provider creation: Updated to use StateFactoryProvider for historical state access
• Transaction conversion: Simplified to work with TxWithHash from provider instead of versioned database types

5. Production Flow Integration:

The new storage method follows the exact same pattern as do_mine_block:

• Filters successful vs failed transactions
• Creates TypedTransactionExecutionInfo for traces
• Builds SealedBlockWithStatus with FinalityStatus::AcceptedOnL2
• Uses BlockWriter::insert_block_with_states_and_receipts() for atomic storage

6. Integration Tests Updated:

Fixed all references from migration_manager.database to migration_manager.provider to match the new structure.

The refactoring maintains all existing functionality while leveraging the provider abstraction layer, making the code more maintainable and consistent with the rest of the Katana codebase.

Committed: 90cd440de96b04ecd7c602b062deaefe5b976a5a

---

[github-actions]

Binary size report 📊

Branch	Size
main	64.08MiB
claude/issue-139-20250618_165013	66.30MiB ( +3.47% )