Skip to content

Consolidate the three parallel SSZ stacks #615

Description

@emlautarom1

Summary

SSZ spans ~9,100 LOC across 22 files in three stacks that don't share types, hashing, or error handling:

  1. ethereum_ssz + tree_hash derives for consensus-spec containers (crates/eth2api/src/spec/, 3,498 LOC). Every container repeats a 5-derive line plus per-field #[serde_as] attributes — 93 derive lines, 155 Hex0x occurrences. ssz_types is not a dependency, so crates/ssz/types.rs reimplements SszList/SszVector/BitList/BitVector with 12 manual Encode/Decode/TreeHash impls (844 LOC).
  2. A hand-ported fastssz Hasher (crates/ssz/src/hasher.rs, 640 LOC) driving cluster/src/ssz.rs — 29 versioned free functions selected via function-pointer tables, plus an SSZError<H> that duplicates pluto_ssz::Error — and a second locally-declared SszHashable trait in app/src/obolapi/exit.rs#L30-L39.
  3. Hand-written byte-level codecs for Charon wire formats (core/src/ssz_codec.rs, 1,261 LOC), including deliberate deviations from what a derive would emit (documented in unsigneddata.rs: Charon emits an offset table even for fixed-size containers).

Consequences: two incompatible hash-tree-root paths, invisible semantic differences between types (some deliberately omit TreeHash and nothing flags it), and duplicated list/vector plumbing that upstream ssz_types already provides.

Proposed change

  • Before touching any stack: pin the current hasher against Charon/fastssz with golden-vector tests (known input → known 32-byte root, expected values captured from the Go side). The existing hasher.rs tests are self-consistency checks (manual-tree comparisons, bitlist guards) — nothing cross-implementation. Add binary round-trip tests for the zero-test codec helpers (encode.rs, decode.rs) at the same time; they are part of the surface being consolidated. Same for the crate's other untested seams: helpers.rs put_byte_list/put_bytes_n IncorrectListSize, serde_utils.rs ssz_list_u64_string_serde (string/int parse, MAX rejection, round-trip), the binary Encode/Decode bound checks of SszList/SszVector (only the JSON path is tested today), and the hasher's CountGreaterThanLimit and get_depth/next_power_of_two/calculate_limit boundaries.
  • Adopt ssz_types for List/Vector/Bitfield and delete most of crates/ssz/types.rs.
  • Merge the two hasher-side interfaces (SszHashable in obolapi vs the free-function tables in cluster) into one, e.g. a version-aware trait in pluto_ssz; collapse the duplicated error enum.
  • Keep the Charon wire-format codecs (ssz_codec.rs) as explicit, documented exceptions — they encode incompatibilities that derives cannot express.

Open question

The cluster config/definition/lock hashes must byte-match Charon's fastssz output. Before replacing the custom Hasher with tree_hash, prove equivalence on the existing fixture corpus (the verify_hashes golden files) — if fastssz semantics differ (they do for e.g. bitlist mixins and string hashing), the custom hasher stays and only the interface around it is consolidated.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestrustPull requests that update rust code

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions