Skip to content

Comments

test: add encoding spec tests for SSV types#835

Open
shane-moore wants to merge 9 commits intosigp:unstablefrom
shane-moore:feat/encoding-tests
Open

test: add encoding spec tests for SSV types#835
shane-moore wants to merge 9 commits intosigp:unstablefrom
shane-moore:feat/encoding-tests

Conversation

@shane-moore
Copy link
Member

@shane-moore shane-moore commented Feb 17, 2026

Issue Addressed

Continuation of spec test from #833

Proposed Changes

Add encoding spec tests for the remaining SSV types. Each test validates SSZ decode, encode roundtrip, and hash tree root against fixtures from the ssv-spec boole branch.

Encoding tests added:

  • PartialSignatureMessages
  • SignedSSVMessage
  • SSVMessage
  • AggregatorCommitteeConsensusData
  • ProposerConsensusData

Additional Info

The remaining fixtures cover message validation, RSA encryption, duty mapping, and other test categories planned for future PRs.

@claude-code-actions-sigp
Copy link

claude-code-actions-sigp bot commented Feb 17, 2026

Claude finished @shane-moore's task in 1m 41s —— View job


Code Review Complete

I've reviewed the PR and the implementation looks solid. The new spec_tests crate follows good patterns and integrates well with the existing codebase. Below are my findings organized by category.


Summary

This PR adds a new spec_tests crate with encoding tests for SSV types, validating SSZ encode/decode roundtrips and hash tree roots against fixtures from the ssv-spec repository. The implementation is clean, well-structured, and follows Rust best practices.

Verdict: The code quality is high. I've identified a few minor improvements and one commit message typo to fix before merging.


Findings

Code Quality

Strengths:

  • Clean trait-based design with SpecTest trait providing a consistent interface
  • Good separation of concerns: test runner logic in lib.rs, individual test implementations in separate modules
  • Consistent patterns across all encoding test files
  • Appropriate use of custom deserializers for base64 and Hash256 conversions
  • Clear documentation comments explaining each test type's purpose

Minor Issues:

  1. Test code uses .unwrap() and .expect() (anchor/spec_tests/src/lib.rs:40-48)

    • While acceptable in test code, consider whether these could fail in CI and provide clearer error messages
    • Current usage is reasonable for test infrastructure, but the error messages could be more helpful
    • Lines 40-41: These .expect() calls are fine for test setup - they fail fast with clear messages
    • Line 48: .unwrap() on path.file_name() is safe here (we know the path is a file from read_dir())
  2. Submodule is empty in the checkout

    • The anchor/spec_tests/ssv-spec directory exists but is empty (likely not initialized in CI)
    • Tests will fail at runtime with "Fixture directory not found" assertion at lib.rs:31
    • Verify that CI properly initializes submodules with git submodule update --init --recursive

Testing

Strengths:

  • Comprehensive test coverage for all major SSV message types
  • Tests validate three critical properties: SSZ decode, encode roundtrip, and hash tree root
  • Proper error messages that include context about what failed
  • Fixture-based testing approach matches the Go reference implementation

Observations:

  • Share encoding test is deliberately skipped with a clear explanation (architectural difference)
  • Good use of TODO(spec-tests) marker for future test types
  • Test runner properly counts and reports failures

Note: The actual test execution depends on fixtures in the ssv-spec submodule. Ensure CI properly initializes submodules.


Security

No security concerns identified. This is test-only code that:

  • Validates encoding/decoding correctness
  • Uses well-established libraries (ethereum_ssz, tree_hash, serde)
  • Doesn't handle sensitive data or user input
  • Runs in a controlled test environment

Performance

Performance is appropriate for test code:

  • File I/O is acceptable for fixture loading in tests
  • SSZ encoding/decoding operations are necessary for validation
  • Test runner processes files sequentially, which is fine for spec tests
  • No performance-critical paths that need optimization

Documentation ⚠️

Strengths:

  • Each test struct has a doc comment explaining its purpose
  • The SpecTest trait is documented
  • Good inline comments explaining architectural decisions (e.g., Share encoding skip)
  • Functions have clear names that indicate their purpose

Minor Improvements Needed:

  1. Missing crate-level documentation

    • Consider adding a module-level doc comment at the top of lib.rs explaining the purpose and structure of the spec_tests crate
    • Example: What the spec tests validate, how to run them, where fixtures come from
  2. Submodule setup not documented

    • The .gitmodules file is added, but there's no documentation about the ssv-spec submodule
    • Consider adding a comment in lib.rs or a README explaining the submodule dependency and initialization requirements

Commit Messages ⚠️

Found typos in commit messages:

  • bbabc63: "chor:" should be "chore:"
  • 682b544: "creat" should be "create"

Consider squashing commits or rebasing to fix these typos before merging, as they'll be part of the permanent git history.


Specific Code Comments

anchor/spec_tests/src/lib.rs:73-78

The explicit skip for "share.EncodingTest" with explanation is excellent. This kind of architectural documentation is valuable for future maintainers.

anchor/spec_tests/src/utils/deserializers.rs:1-31

The custom deserializers are well-implemented with proper error handling and clear doc comments. The validation in deserialize_bytes_to_hash256 (checking for exactly 32 bytes) is good defensive programming.

All encoding test files

The consistent pattern across all encoding test implementations makes the code easy to understand and maintain. The only variation (SignedSSVMessage not checking hash tree root) is a deliberate difference that matches the spec.


Recommendations

Before Merging:

  1. Fix commit message typos (consider interactive rebase to clean up history)
  2. Verify CI properly initializes the ssv-spec submodule
  3. Consider adding crate-level documentation

Optional Improvements:

  1. Add a README.md in anchor/spec_tests/ explaining the purpose, fixture source, and how to update fixtures
  2. Consider extracting the common encoding test pattern to reduce code duplication (though current duplication is minimal and explicit is fine)

CI Status

The "Pull Request" check has passed ✅. Test suite and local testnet checks are still in progress. The spec tests added in this PR will run as part of the test suite once the submodule is properly initialized.


@shane-moore shane-moore marked this pull request as ready for review February 19, 2026 17:49
@diegomrsantos
Copy link
Member

I know this code already existed, but it's hard to grasp the required context to review the changes for a person unfamiliar with it. lib.rs is required as the crate root in Rust, so the filename itself can’t fully express intent. But the dispatching logic can be moved out to make it explicit.

Suggested structure:

  1. Keep anchor/spec_tests/src/lib.rs as a thin entrypoint only.
  2. Move routing loop into anchor/spec_tests/src/runner/types_dispatcher.rs (or dispatch.rs).
  3. Expose one explicit function like run_all_type_fixtures().
  4. Rename internals for clarity: dispatch_fixture_by_prefix, executed_count, skipped_known_count, skipped_unknown_count.

@diegomrsantos
Copy link
Member

Refactor: Deduplicate Encoding Tests in anchor/spec_tests/src/types

Problem

There is significant duplication across the encoding test files in anchor/spec_tests/src/types.

What is duplicated

1. Same fixture shape in most files (data + expected_root)

Examples:

  • anchor/spec_tests/src/types/partial_sig_message_encoding.rs:16
  • anchor/spec_tests/src/types/ssv_message_encoding.rs:16
  • anchor/spec_tests/src/types/proposer_consensus_data_encoding.rs:16

2. Same run() flow in each file

Every file repeats the same steps:

  • Decode from SSZ bytes
  • Encode back
  • Compare bytes
  • Compare tree hash root

Examples:

  • anchor/spec_tests/src/types/partial_sig_message_encoding.rs:24
  • anchor/spec_tests/src/types/ssv_message_encoding.rs:24
  • anchor/spec_tests/src/types/aggregator_committee_consensus_data_encoding.rs:24

3. The only real variation between files is:

  • The target type being tested
  • Whether the root check is performed (signed_ssv_msg skips it — see anchor/spec_tests/src/types/signed_ssv_msg_encoding.rs:17)

Proposal

Extract shared helpers to eliminate copy-paste and make future encoding tests easier to add and maintain.

Option A — Generic helper functions

pub fn run_roundtrip<T>(...) { ... }
pub fn run_roundtrip_with_root<T>(...) { ... }
  • run_roundtrip::<T> — decode → encode → compare bytes (no root check)
  • run_roundtrip_with_root::<T> — decode → encode → compare bytes → compare tree hash root

Each test file is reduced to calling the appropriate helper with its target type.

Option B — Generic fixture type

Define a single generic fixture struct that captures data + expected_root and drives the run() logic, parameterised over the target type T.

Benefits

  • Removes repeated boilerplate across all encoding test files
  • Makes the presence/absence of the root check explicit and easy to see
  • Adding a new encoding test becomes a one-liner
  • Bug fixes or flow changes propagate automatically to all tests

@shane-moore
Copy link
Member Author

luv the ideas, implemented in last couple of commits:

  • made spec-tests package more modular instead of bloating lib.rs. and also the tests looks a lot cleaner after extracting the encoding logic to helpers, nice!

Comment on lines 38 to 43
// Anchor's `Share` is architecturally different from Go spec's `Share`
// (different fields, decomposed across multiple types). Not applicable.
"share.EncodingTest" => {
*skipped_known_count += 1;
None
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

share.EncodingTest is currently a known inapplicable case, but this branch skips it silently. Could we make this explicit in test output (e.g. eprintln!("SKIP (known-inapplicable): {prefix}")) and include skipped-known/skipped-unknown counts in the final summary?

Reason: share is the one intentional coverage exception in this suite, and surfacing it clearly makes CI output auditable and prevents future confusion about missing fixture coverage.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel you on that, updated in e6b57e4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants