Change structure of Transactions for faster block decoding

[paulhauner]

Issue Addressed

NA

Proposed Changes

Reduces the time to decode a block by ~15% by reducing the effort needed to decode execution_payload.transactions.

Previously we stored transactions as VariableList<VariableList>. Now we store them as (Vec<offsets>, Vec<values>), where each offset points to the start of a new transaction in values. This is how the data is represented as SSZ.

The prior implementation required N+1 allocations, where this requires 2 allocations.

Additional Info

• Blocked on Bump SSZ version for larger bitfield SmallVec #6915

Before

lcli transition-blocks --block-path /tmp/block-10472248.ssz --pre-state-path /tmp/pre-state-10472248.ssh --runs 10
[2024-11-25T05:16:37Z INFO  lcli::transition_blocks] Using mainnet spec
[2024-11-25T05:16:37Z INFO  lcli::transition_blocks] Doing 10 runs
[2024-11-25T05:16:37Z INFO  lcli::transition_blocks] Config { no_signature_verification: false, exclude_cache_builds: false, exclude_post_block_thc: false }
[2024-11-25T05:16:37Z DEBUG lcli::transition_blocks] SSZ decoding /tmp/block-10472248.ssz: 3.534442ms
[2024-11-25T05:16:37Z DEBUG lcli::transition_blocks] SSZ decoding /tmp/block-10472248.ssz: 3.727088ms
[2024-11-25T05:16:37Z DEBUG lcli::transition_blocks] SSZ decoding /tmp/block-10472248.ssz: 3.65126ms
[2024-11-25T05:16:37Z DEBUG lcli::transition_blocks] SSZ decoding /tmp/block-10472248.ssz: 3.520166ms
[2024-11-25T05:16:37Z DEBUG lcli::transition_blocks] SSZ decoding /tmp/block-10472248.ssz: 3.517495ms
[2024-11-25T05:16:37Z DEBUG lcli::transition_blocks] SSZ decoding /tmp/block-10472248.ssz: 3.659826ms
[2024-11-25T05:16:37Z DEBUG lcli::transition_blocks] SSZ decoding /tmp/block-10472248.ssz: 3.532312ms
[2024-11-25T05:16:37Z DEBUG lcli::transition_blocks] SSZ decoding /tmp/block-10472248.ssz: 3.485217ms
[2024-11-25T05:16:37Z DEBUG lcli::transition_blocks] SSZ decoding /tmp/block-10472248.ssz: 3.489904ms
[2024-11-25T05:16:37Z DEBUG lcli::transition_blocks] SSZ decoding /tmp/block-10472248.ssz: 3.495821ms

After

lcli transition-blocks --block-path /tmp/block-10472248.ssz --pre-state-path /tmp/pre-state-10472248.ssh --runs 10
[2024-11-25T05:22:26Z INFO  lcli::transition_blocks] Using mainnet spec
[2024-11-25T05:22:26Z INFO  lcli::transition_blocks] Doing 10 runs
[2024-11-25T05:22:26Z INFO  lcli::transition_blocks] Config { no_signature_verification: false, exclude_cache_builds: false, exclude_post_block_thc: false }
[2024-11-25T05:22:26Z DEBUG lcli::transition_blocks] SSZ decoding /tmp/block-10472248.ssz: 3.156101ms
[2024-11-25T05:22:26Z DEBUG lcli::transition_blocks] SSZ decoding /tmp/block-10472248.ssz: 3.086956ms
[2024-11-25T05:22:26Z DEBUG lcli::transition_blocks] SSZ decoding /tmp/block-10472248.ssz: 3.085763ms
[2024-11-25T05:22:26Z DEBUG lcli::transition_blocks] SSZ decoding /tmp/block-10472248.ssz: 3.197391ms
[2024-11-25T05:22:26Z DEBUG lcli::transition_blocks] SSZ decoding /tmp/block-10472248.ssz: 3.111297ms
[2024-11-25T05:22:26Z DEBUG lcli::transition_blocks] SSZ decoding /tmp/block-10472248.ssz: 3.085961ms
[2024-11-25T05:22:26Z DEBUG lcli::transition_blocks] SSZ decoding /tmp/block-10472248.ssz: 3.089851ms
[2024-11-25T05:22:26Z DEBUG lcli::transition_blocks] SSZ decoding /tmp/block-10472248.ssz: 3.076389ms
[2024-11-25T05:22:26Z DEBUG lcli::transition_blocks] SSZ decoding /tmp/block-10472248.ssz: 3.082601ms
[2024-11-25T05:22:26Z DEBUG lcli::transition_blocks] SSZ decoding /tmp/block-10472248.ssz: 3.07679ms

[michaelsproul]

Paul the Optimiser returns 🤩

[michaelsproul]

omg I did not mean to close that, sorry!

[paulhauner]

This is currently blocked on sigp/tree_hash#18

[michaelsproul]

Updated ssz_types, but Milhouse will still need an update

https://github.com/sigp/ssz_types/releases/tag/v0.10.0

[michaelsproul]

Merged latest unstable into this because I'd like to base some other changes off the latest Milhouse version. Oddly I'm now getting Clippy failures for large_enum_variant that don't exist on unstable. They don't seem related to this PR 🤔

[paulhauner]

I've just created #6915 to isolate the SSZ version bump.

I primarily wanted to get the bitfield SmallVec bump into a discreet PR. Both of these changes have some risk so it's nice to isolate them.

[mergify]

Some required checks have failed. Could you please take a look @paulhauner? 🙏

[mergify]

Hi @paulhauner, this pull request has been closed automatically due to 30 days of inactivity. If you’d like to continue working on it, feel free to reopen at any time.