Skip to content

Rust consumer: run the spec vectors against pluto - #13

Open
pinebit wants to merge 27 commits into
mainfrom
pinebit/pluto-conformance
Open

Rust consumer: run the spec vectors against pluto#13
pinebit wants to merge 27 commits into
mainfrom
pinebit/pluto-conformance

Conversation

@pinebit

@pinebit pinebit commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds consumers/rust/, a standalone Cargo package that runs all nine published vector suites against an unmodified pluto checkout. It path-depends on a sibling ../pluto and is never copied into pluto, so a spec-repo user can test the current pluto implementation independently. This is the Rust counterpart to consumers/go/, which does the same for charon.

Also adds plans/pluto-conformance.md, the plan this was executed from, which carries the per-suite Results table and the Findings section, and one spec-side change the findings called for: charon_anchor.json gains an "Extended 1.5s proposer round-1 timeout (proposal_timeout)" behaviours-ladder entry (v1.9.0), with the matching README compatibility-table row and a sentence in docs/dv-spec/consensus.md's timer prose stating the proposer round-1 exception the reference implementation and vectors already carried.

Tested against pluto 67088a2. 31 Rust tests, all green. Both FAIL-class findings were re-proven by differential execution (a charon hashProto probe for the map-entry divergence, a layered serde probe for the cluster-parsing one), and every pluto file:line citation in the findings was re-audited against source.

Verdicts

Suite Verdict
secp256k1_signatures PASS
bls_threshold PASS
priority_scoring PASS (18/18, driven end-to-end through the public Prioritiser API)
qbft_hashing FAIL — 23/25 agree; 2 pinned divergences
cluster_hashing FAIL — pluto rejects charon-legitimate JSON shapes
timer_deadlines PASS with ProposalTimeout enabled (the vectors' assumption); the default-config divergence on 9 PROPOSER/round-1 cases is ABSENT-OK against the ladder entry this PR adds; deadline ABSENT-OK; start-delay UNREACHABLE
qbft_msg_limits 5 match, 6 ABSENT-OK; wire-size 4/4 match
qbft_decided_resends ABSENT-OK (pluto has no rebroadcast limiter)
parsigex_sender_binding cases ABSENT-OK; peer_map UNREACHABLE

ABSENT-OK means the divergence is covered by a charon_anchor.json behaviours-ladder entry postdating pluto's charon-v1.7.1 parity anchor; those are pinned in code so they fail when pluto catches up. UNREACHABLE means no public API path exists from an external crate — a coverage gap, not a pass.

Divergences found

  • Protobuf map-entry encoding. prost omits default-valued map-entry key and value fields; charon's Go marshaler emits them. UnsignedDataSet entries with an empty key or empty value therefore encode, and hash, differently. Confirmed by differential execution: a Go probe running charon's hashProto (core/consensus/qbft/msg.go) on both cases reproduces the vector bytes exactly. Reachable only via malformed or adversarial wire input, per code inspection of charon's encode path.
  • Cluster file parsing. pluto rejects three JSON shapes charon legitimately produces: operators: null and validator_addresses: null (Go marshals nil slices as null), an absent partial_deposit_data key (Go omits it via omitempty), and an absent timestamp. A layered probe proved this blocker set complete for the shipped cases (name and deposit_amounts are already handled); charon omitempty fields the cases don't exercise may hide more instances of the same missing-#[serde(default)] root cause.
  • Proposal-timeout default. pluto has Feature::ProposalTimeout off by default, matching charon v1.7.1; charon at the spec anchor has it on, promoted to stable in v1.9.0. Default-configured pluto diverges from the vectors on all 9 PROPOSER/round-1 cases — ABSENT-OK against the ladder entry this PR adds, and pinned by a test that names it.
  • Justification cap. pluto caps justifications at 4 * node_count(), which is neither charon v1.7.1's absence of a cap nor the spec's 2n. Now pinned at its exact boundary.
  • DKG partial-signature verification. pluto's DKG wires parsigex with verification skipped by design (dkg/src/node.rs::setup_p2p), and the aggregation-time check that validates share indices lives in a private module, so the lock-hash exchange verifies nothing per message.

Notes for this repo

  • The behaviours-ladder gap the findings identified (no entry for the proposal-timeout stabilization, though the vectors depend on it) is fixed in this PR, in its own commit, together with the README compatibility-table row tests/test_release.py enforces.
  • The real_keys_3_of_4 lock's operator EIP-712 signatures are placeholders, so verify_signatures short-circuits before the BLS-aggregate and node-signature checks that case exists to exercise. An external consumer cannot reach them; the Go consumer only can because they are in-package in charon. Recorded in the plan, not acted on.

tox.ini gains one skip so codespell does not scan the gitignored Cargo build directory.

pinebit added 21 commits July 31, 2026 20:19
Restructures unsigned_data_set_hashing to skip and separately track the two
cases where prost's map encoding diverges from charon's (empty key/value
map-entry presence), so the FAIL finding stays visible without leaving the
whole suite permanently red.
The skip-set assert_eq! only checks case names, not divergence status; the
comment wrongly claimed it detects a case that stops diverging. Fixed the
comment to attribute that detection to the sibling pinned test instead.
Also drops line numbers from the charon/prost citations in the Findings
entry, per the repo's no-line-anchor convention.
round_timeouts previously only tested with ProposalTimeout enabled,
which overstated parity: pluto's default FeatureSet disables it,
diverging from the vectors on proposer round-1 cases. Split into an
enabled-feature test (matches the vectors) and a default-feature test
that pins the divergence explicitly. Also add a real code-level pin
for slot-invariance (deadline_nanos ABSENT-OK), replacing a doc-only
claim with an assertion that fails if pluto's timers ever grow
slot-dependence.
Asserts the published test_vectors/*.json suites match the COVERED
table's declared suites, and that each declared test file exists.
Mirrors the Go consumer's TestEverySuiteIsCovered.
read_protobuf_with_max_size reports every rejection through the same
io::ErrorKind::InvalidData with no structured reason code, so matching
on err.to_string() is unavoidable here despite test_vectors/README.md's
"the slugs are the contract, wording is not" -- pluto's own p2p.rs
tests use the same idiom.
The feature list was copied wholesale from pluto's workspace Cargo.toml.
Only noise, tokio, and yamux are needed here; the rest (autonat, dns,
identify, mdns, ping, quic, relay, secp256k1, macros, tcp) are unused
by this consumer's code.
The Rust consumer's coverage.rs is not compiled here either, so mirror
the Go check: parse COVERED out of consumers/rust/tests/coverage.rs
and assert it matches the published suites, and that each declared
test file exists.
consumers/README.md: expand the Pluto row and section with suite-by-suite
verdicts, run instructions, and the pluto commit under test (67088a2).
plans/spec-completion.md: mark the pluto consumer bullet done, pointing
at plans/pluto-conformance.md's Results table.
@pinebit pinebit changed the title Add pluto conformance testing plan Rust consumer: run the spec vectors against pluto Aug 1, 2026
pinebit and others added 5 commits August 1, 2026 07:41
The timer_deadlines vectors assume charon's proposal_timeout feature is
enabled (stable since v1.9.0, charon #4296), but the behaviour ladder had
no entry for it, so an implementation pinned to an older charon (pluto at
v1.7.1) had nothing excusing the 9 diverging PROPOSER/round-1 cases.
Record it in charon_anchor.json, the README compatibility table, and the
consensus.md timer prose, which omitted the proposer round-1 exception the
reference implementation and vectors already carried.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Re-validation of every recorded divergence (differential Go probes in the
charon checkout, layered serde probes, and a source audit of pluto 67088a2)
confirmed all findings. Three documentation defects surfaced in review:

- qbft_msg_limits.rs presented a paraphrase of test_vectors/README.md's
  slug rule as a verbatim quote; reworded as a paraphrase.
- qbft_decided_resends.rs claimed pluto's own test reaches quorum COMMIT;
  that test cancels at the node's own COMMIT broadcast, so the external
  COMMIT quorum is this harness's extension. Both comments now say so.
- The plan attributed DKG's no-op parsigex verifier to Exchanger; the
  production wiring is in dkg/src/node.rs::setup_p2p (Exchanger's no-op
  verifiers are cfg(test)-only).

Also sharpen the cluster_hashing finding: a layered probe proved the
all_empty_lists blocker set is exactly operators/validators null-handling
plus the missing timestamp default, and tick the plan's completed step
checkboxes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Go's TestEverySuiteIsCovered also asserts each suite ships cases, read
from its vendored release manifest. The Rust consumer reads test_vectors/
live with no manifest; each suite test pins its exact case count instead.
Say so rather than claiming unqualified equivalence.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The proposal_timeout ladder entry added in ac81aab formally covers the 9
PROPOSER/round-1 cases that diverge under pluto's default FeatureSet, so
the plan's Results row, its finding, the consumer README, and the timer
test's comments now name that entry per the verdict taxonomy instead of
calling it an uncovered known-divergence. Also record the 2026-08-01
re-validation pass in the plan: full suite re-run, both FAIL findings
re-proven by differential execution, all pluto citations re-audited.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The bullet's bold lead still read as an open recommendation ("has no
entry") even though the entry landed in ac81aab; retitle it as an
addressed gap and fix the timer finding's cross-reference to it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant