From b9f5042e741ab08b755045774049f1a10ac1584e Mon Sep 17 00:00:00 2001 From: Andrei Smirnov Date: Fri, 31 Jul 2026 20:09:01 +0300 Subject: [PATCH 01/27] Added pluto-conformance plan --- plans/pluto-conformance.md | 1090 ++++++++++++++++++++++++++++++++++++ 1 file changed, 1090 insertions(+) create mode 100644 plans/pluto-conformance.md diff --git a/plans/pluto-conformance.md b/plans/pluto-conformance.md new file mode 100644 index 0000000..3254de8 --- /dev/null +++ b/plans/pluto-conformance.md @@ -0,0 +1,1090 @@ +# Pluto Conformance Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development +> (recommended) or superpowers:executing-plans to implement this plan task-by-task. +> Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** A Rust test harness in this repo (`consumers/rust/`) that runs every published +vector suite against an unmodified pluto checkout, so a spec-repo user can test the +current pluto implementation independently — the mirror of `consumers/go/` for charon, +but inverted: nothing is copied into pluto, the harness path-depends on pluto's crates. + +**Architecture:** One standalone Cargo package outside pluto's workspace, with +`path = "../../../pluto/crates/…"` dependencies (sibling-checkout convention, like +`~/charon` for the Go consumer). Vectors are read from `../../test_vectors/` at test +time — no vendoring, no build step. Each suite gets one integration-test file; each +case gets a named subcheck. One suite per task, run and triaged before the next starts. + +**Tech Stack:** Rust 1.95.0 (pluto's pin), edition 2024, `serde_json` for loading, +pluto crates as path dependencies, `tokio` with `start_paused` for timer checks. + +## Why this is not a normal feature plan + +A conformance check does not "pass or get fixed" — it produces a **verdict**, and a +red case is a *finding to record*, never a reason to weaken the assertion. Pluto pins +parity to **charon v1.7.1**, while the vectors describe charon main at `6054bcb2`. +`charon_anchor.json` → `behaviours` lists exactly which specified behaviours postdate +v1.7.1. Verdict taxonomy, used in the Results table below: + +| Verdict | Meaning | +| --- | --- | +| `PASS` | Pluto reproduces every case in the group. | +| `FAIL` | Pluto diverges and no ladder entry excuses it — a real finding; record it under Findings and consider reporting upstream to pluto. | +| `ABSENT-OK` | Divergence matches a `behaviours` entry with `first_charon_release` > v1.7.1 (or `null`). The test then **pins pluto's current behaviour** with a comment naming the ladder entry, so it flips loudly when pluto catches up. | +| `UNREACHABLE` | No public API path from an external crate; verified by code inspection instead, with file:line recorded. A coverage gap, not a pass. | + +## Pre-registered expectations (verify, don't assume) + +Found by reading pluto at `67088a2` before writing any test. Each must be confirmed +or refuted by the task that covers it: + +1. Justification cap is `4 * nodes` (`MAX_JUSTIFICATIONS_PER_NODE = 4`, + `crates/consensus/src/qbft/component.rs:42`) — not the spec's `2n`. Ladder entry + "QBFT DECIDED-resend rate limit and message size/count limits" (`null`) covers the + spec's `2n` being absent, but a *third* value (neither charon v1.7.1's no-cap nor + the spec's `2n`) is a finding worth recording regardless. +2. No `values ≤ 2(j+1)` check exists anywhere in pluto. Same ladder entry. +3. `MAX_CONSENSUS_MSG_SIZE = 32 MiB` **is** implemented (`qbft/p2p.rs`) despite being + part of the same `null` ladder entry — the wire-size half may genuinely PASS. +4. No DECIDED-resend limiter (`crates/core/src/qbft/mod.rs:494-503` rebroadcasts + unboundedly). Ladder: same `null` entry → ABSENT-OK expected. +5. No ParSigEx/DKG sender binding (no `share_idx == sender` check found). Ladder: + "Sender-bound share indices in the DKG lock-hash exchange" (`null`). +6. Priority protocol ID is the slash-less `charon/priority/2.0.0` only. Ladder: + "Preferred priority protocol ID" (`null`) — legacy-only is correct for v1.7.1. +7. Round timers are relative (`RoundTimer::timer` measures from now); genesis-derived + deterministic deadlines are the v1.9.0 ladder entry → `deadline_nanos` likely + ABSENT-OK. `linear_subsequent_round_timeout` reproduces charon v1.7.1's + nanosecond bug (200ns, ObolNetwork/charon#4537); the vectors encode the *fixed* + behaviour (ladder entry "Linear round timer subsequent-round timeout fix", `null`). + +## Results + +Progress tracking lives here. Update the row when a task's verdict is in; record the +pluto commit actually tested. Statuses: `todo` / `in progress` / `done`. + +**Pluto commit under test:** _(fill in at Task 1: `git -C ~/pluto rev-parse --short HEAD`)_ + +| # | Check | Suite | Status | Verdict | Notes | +| --- | --- | --- | --- | --- | --- | +| 1 | Harness scaffold + secp256k1 | `secp256k1_signatures` | todo | — | | +| 2 | Proto encoding + SSZ hashing | `qbft_hashing` | todo | — | | +| 3 | BLS threshold aggregation | `bls_threshold` | todo | — | | +| 4 | Cluster hashes + lock verification | `cluster_hashing` | todo | — | | +| 5 | Priority scoring | `priority_scoring` | todo | — | | +| 6 | Round timer deadlines | `timer_deadlines` | todo | — | | +| 7 | QBFT message limits | `qbft_msg_limits` | todo | — | | +| 8 | DECIDED-resend limiting | `qbft_decided_resends` | todo | — | | +| 9 | Sender binding + peer map | `parsigex_sender_binding` | todo | — | | +| 10 | Coverage guard + docs | all | todo | — | | + +## Findings + +_Append findings here as tasks produce them: what diverged, file:line in pluto, +whether a ladder entry covers it, and whether it was reported upstream._ + +## Global Constraints + +- Pluto is **never modified**. The harness lives in `consumers/rust/` and path-depends + on a pluto checkout at `../../../pluto` relative to the package (i.e. `~/pluto` next + to `~/distributed-validator-specs`). If the checkout is elsewhere, the user edits the + paths in `Cargo.toml`; document this in the README, do not build indirection for it. +- Rust toolchain pinned to **1.95.0** (copy pluto's `rust-toolchain.toml`), edition 2024. +- Dependency versions that pluto also uses (`prost`, `serde_json`, `k256`, `tokio`, + `hex`, `libp2p`) must be copied from pluto's `[workspace.dependencies]` so cargo + unifies them — a second `prost` major version makes pluto's generated types unusable. +- The consumer's `Cargo.toml` must replicate pluto's + `[patch.crates-io] multistream-select = { path = "../../../pluto/third_party/multistream-select" }` + (required once `pluto-priority`/`pluto-p2p` are dependencies; harmless before). +- Vectors load from `../../test_vectors/` via `CARGO_MANIFEST_DIR`, overridable with + `SPEC_VECTORS_DIR` (same env var as the Go consumer). +- Hex in suites is lowercase, unprefixed — except inside `cluster_hashing` case + `input`s, which are verbatim charon files (`0x`-prefixed, Gwei as strings, `null` + for empty lists). +- Every case runs as a named subcheck: iterate cases inside one `#[test]`/ + `#[tokio::test]`, collect failures into a `Vec` naming + `"{suite}/{group}/{case}"` and assert the vec is empty at the end — one bad case + must not hide the rest. +- Code blocks below are normative for *what is asserted*. Exact pluto field names, + types and paths were mapped from a read of pluto `67088a2` but not compiled; fix + compile errors against the real crate without changing any assertion. If an + assertion itself proves impossible (API truly unreachable), that is an + `UNREACHABLE` verdict, not a rewrite. +- Never weaken an assertion to make a case green. Red → triage against the ladder → + verdict → record. +- Commit after each task, on a feature branch (never directly to `main`). + +--- + +### Task 1: Harness scaffold + `secp256k1_signatures` + +The smallest suite against the most-public pluto API (`pluto-k1util`: everything +`pub`), so the scaffold is proven by a real check. + +**Files:** +- Create: `consumers/rust/Cargo.toml` +- Create: `consumers/rust/rust-toolchain.toml` +- Create: `consumers/rust/.gitignore` +- Create: `consumers/rust/src/lib.rs` +- Create: `consumers/rust/tests/secp256k1_signatures.rs` +- Modify: `consumers/README.md` (Pluto row: link `rust/`, status "in progress") + +**Interfaces:** +- Produces: `spec_vectors_pluto::load_suite(name: &str) -> serde_json::Value` and + `spec_vectors_pluto::unhex(s: &str) -> Vec` — every later task consumes these. + +- [ ] **Step 1: Write the package scaffold** + +`consumers/rust/Cargo.toml` (copy the exact version requirements for `serde_json`, +`hex`, `k256` from `~/pluto/Cargo.toml` `[workspace.dependencies]`): + +```toml +[package] +name = "spec-vectors-pluto" +version = "0.1.0" +edition = "2024" +publish = false +description = "Runs the spec's test_vectors/ suites against a pluto checkout at ../../../pluto" + +[dependencies] +serde_json = "1" +hex = "0.4.3" + +[dev-dependencies] +pluto-k1util = { path = "../../../pluto/crates/k1util" } +k256 = { version = "0.13", features = ["ecdsa"] } + +[patch.crates-io] +multistream-select = { path = "../../../pluto/third_party/multistream-select" } +``` + +`consumers/rust/rust-toolchain.toml`: + +```toml +[toolchain] +channel = "1.95.0" +components = ["rustfmt", "clippy"] +``` + +`consumers/rust/.gitignore`: + +``` +/target +Cargo.lock +``` + +`consumers/rust/src/lib.rs`: + +```rust +//! Loader for the spec's test_vectors/ suites. +//! +//! Vectors are read from `../../test_vectors/` relative to this package, or from +//! `SPEC_VECTORS_DIR` when set. `load_suite` refuses a file whose `suite` field +//! does not match the requested name, so a stray copy cannot silently substitute. + +use std::path::PathBuf; + +pub fn vectors_dir() -> PathBuf { + match std::env::var_os("SPEC_VECTORS_DIR") { + Some(dir) => PathBuf::from(dir), + None => PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../test_vectors"), + } +} + +pub fn load_suite(name: &str) -> serde_json::Value { + let path = vectors_dir().join(format!("{name}.json")); + let raw = std::fs::read_to_string(&path) + .unwrap_or_else(|e| panic!("read {}: {e}", path.display())); + let suite: serde_json::Value = serde_json::from_str(&raw) + .unwrap_or_else(|e| panic!("parse {}: {e}", path.display())); + assert_eq!(suite["suite"], name, "suite field mismatch in {}", path.display()); + suite +} + +pub fn unhex(s: &str) -> Vec { + hex::decode(s).unwrap_or_else(|e| panic!("bad hex {s:?}: {e}")) +} +``` + +- [ ] **Step 2: Write the failing test** + +`consumers/rust/tests/secp256k1_signatures.rs`. The suite carries one secret key and +cases of `{input.hash_hex, signature_hex, recovered_pubkey_hex}` — signing is RFC 6979 +deterministic, so exact signature bytes are asserted, and recovery is asserted both +ways: + +```rust +use spec_vectors_pluto::{load_suite, unhex}; + +#[test] +fn secp256k1_signatures() { + let suite = load_suite("secp256k1_signatures"); + let secret = k256::SecretKey::from_slice(&unhex(suite["secret_hex"].as_str().unwrap())) + .expect("suite secret key"); + let pubkey = secret.public_key(); + let mut failures = Vec::new(); + + for case in suite["cases"].as_array().unwrap() { + let name = case["name"].as_str().unwrap(); + let hash = unhex(case["input"]["hash_hex"].as_str().unwrap()); + let want_sig = unhex(case["signature_hex"].as_str().unwrap()); + let want_pub = unhex(case["recovered_pubkey_hex"].as_str().unwrap()); + + match pluto_k1util::sign(&secret, &hash) { + Ok(sig) if sig.to_vec() == want_sig => {} + Ok(sig) => failures.push(format!( + "{name}: sign gave {}, want {}", hex::encode(sig), hex::encode(&want_sig))), + Err(e) => failures.push(format!("{name}: sign failed: {e}")), + } + match pluto_k1util::recover(&hash, &want_sig) { + Ok(rec) if rec.to_sec1_bytes().to_vec() == want_pub => {} + Ok(rec) => failures.push(format!( + "{name}: recover gave {}, want {}", + hex::encode(rec.to_sec1_bytes()), hex::encode(&want_pub))), + Err(e) => failures.push(format!("{name}: recover failed: {e}")), + } + match pluto_k1util::verify_65(&pubkey, &hash, &want_sig) { + Ok(true) => {} + other => failures.push(format!("{name}: verify_65 gave {other:?}, want Ok(true)")), + } + } + assert!(failures.is_empty(), "{} failures:\n{}", failures.len(), failures.join("\n")); +} +``` + +Check the vector's pubkey encoding first (`uv run python -c "import json; +print(json.load(open('test_vectors/secp256k1_signatures.json'))['pubkey_hex'])"`): +if it is 33 bytes it is SEC1 compressed and `to_sec1_bytes()` matches; if 65 bytes, +use `k256::EncodedPoint::from(&rec).to_untagged_bytes()` accordingly. + +- [ ] **Step 3: Run it** + +```bash +cd consumers/rust && cargo test --test secp256k1_signatures +``` +Expected: compiles against `~/pluto` and all cases pass (charon and pluto share +RFC 6979 + low-S via k256). Any red case is a FAIL finding — triage per the taxonomy. + +- [ ] **Step 4: Record the verdict** + +Fill the pluto commit under test and row 1 of the Results table in this plan. + +- [ ] **Step 5: Update `consumers/README.md` and commit** + +Add to the consumer table: `| Pluto (Rust) | in progress — see plans/pluto-conformance.md | [rust/](rust) |`, +plus a short "Pluto consumer" section: sibling-checkout requirement, the +`cd consumers/rust && cargo test` invocation, and that pluto is never modified. + +```bash +git checkout -b pluto-conformance +git add consumers/rust consumers/README.md plans/pluto-conformance.md +git commit -m "Add Rust consumer scaffold and secp256k1 suite for pluto" +``` + +--- + +### Task 2: `qbft_hashing` — deterministic encoding + SSZ hash roots + +**Files:** +- Create: `consumers/rust/tests/qbft_hashing.rs` +- Modify: `consumers/rust/Cargo.toml` (add dev-deps) + +**Interfaces:** +- Consumes: `load_suite`, `unhex` from Task 1. +- Pluto APIs: `pluto_consensus::qbft::msg::{hash_proto, hash_proto_bytes}` (both pub), + prost-generated `Duty`, `UnsignedDataSet`, `QbftMsg` from `pluto-core` + (`pluto_core::corepb::v1::…` — confirm the exact module path from + `~/pluto/crates/core/src/lib.rs` before writing; the golden test at + `~/pluto/crates/consensus/src/qbft/msg.rs:420` shows working imports to copy). + +- [ ] **Step 1: Add dependencies** + +```toml +pluto-consensus = { path = "../../../pluto/crates/consensus" } +pluto-core = { path = "../../../pluto/crates/core" } +prost = "" +prost-types = "" +``` + +- [ ] **Step 2: Write the test — four groups, one test fn each** + +```rust +use prost::Message; +use spec_vectors_pluto::{load_suite, unhex}; +// Adjust to pluto's real module path (see msg.rs golden test imports): +use pluto_core::corepb::v1::{Duty, QbftMsg, UnsignedDataSet}; +use pluto_consensus::qbft::msg::{hash_proto, hash_proto_bytes}; + +fn check(failures: &mut Vec, name: &str, encoded: &[u8], hash: [u8; 32], + case: &serde_json::Value) { + let want_enc = unhex(case["encoding_hex"].as_str().unwrap()); + let want_hash = unhex(case["hash_hex"].as_str().unwrap()); + if encoded != want_enc { + failures.push(format!("{name}: encoding {}, want {}", + hex::encode(encoded), hex::encode(&want_enc))); + } + if hash.as_slice() != want_hash { + failures.push(format!("{name}: hash {}, want {}", + hex::encode(hash), hex::encode(&want_hash))); + } +} + +#[test] +fn duty_hashing() { + let suite = load_suite("qbft_hashing"); + let mut failures = Vec::new(); + for case in suite["duty"].as_array().unwrap() { + let name = format!("duty/{}", case["name"].as_str().unwrap()); + let duty = Duty { + slot: case["input"]["slot"].as_i64().unwrap(), + r#type: case["input"]["type"].as_i64().unwrap() as i32, + }; + // Covers the ≤32-byte rule: a short encoding is returned zero-padded, unhashed. + match hash_proto(&duty) { + Ok(h) => check(&mut failures, &name, &duty.encode_to_vec(), h, case), + Err(e) => failures.push(format!("{name}: hash_proto failed: {e}")), + } + } + assert!(failures.is_empty(), "{}", failures.join("\n")); +} + +#[test] +fn unsigned_data_set_hashing() { + let suite = load_suite("qbft_hashing"); + let mut failures = Vec::new(); + for case in suite["unsigned_data_set"].as_array().unwrap() { + let name = format!("unsigned_data_set/{}", case["name"].as_str().unwrap()); + // Map keys/values are hex in the vector; pluto's set is BTreeMap. + // Covers explicit map-entry presence: an empty value must still emit 0x1200. + let set = case["input"]["set"].as_object().unwrap().iter() + .map(|(k, v)| (k.clone(), unhex(v.as_str().unwrap()).into())) + .collect(); + let uds = UnsignedDataSet { set }; + match hash_proto(&uds) { + Ok(h) => check(&mut failures, &name, &uds.encode_to_vec(), h, case), + Err(e) => failures.push(format!("{name}: hash_proto failed: {e}")), + } + } + assert!(failures.is_empty(), "{}", failures.join("\n")); +} + +#[test] +fn qbft_signing_roots() { + let suite = load_suite("qbft_hashing"); + let mut failures = Vec::new(); + for case in suite["qbft_signing_root"].as_array().unwrap() { + let name = format!("qbft_signing_root/{}", case["name"].as_str().unwrap()); + let i = &case["input"]; + // value_hash / prepared_value_hash are always 32 bytes on the wire, + // zeros meaning "no value"; signature is empty when signing. + let msg = QbftMsg { + r#type: i["type"].as_i64().unwrap(), + duty: Some(Duty { + slot: i["slot"].as_i64().unwrap(), + r#type: i["duty_type"].as_i64().unwrap() as i32, + }), + peer_idx: i["peer_idx"].as_i64().unwrap(), + round: i["round"].as_i64().unwrap(), + prepared_round: i["prepared_round"].as_i64().unwrap(), + value_hash: unhex(i["value_hash"].as_str().unwrap()).into(), + prepared_value_hash: unhex(i["prepared_value_hash"].as_str().unwrap()).into(), + signature: Default::default(), + }; + match hash_proto(&msg) { + Ok(h) => check(&mut failures, &name, &msg.encode_to_vec(), h, case), + Err(e) => failures.push(format!("{name}: hash_proto failed: {e}")), + } + } + assert!(failures.is_empty(), "{}", failures.join("\n")); +} + +#[test] +fn any_string_hashing() { + // Charon's *priority* hashProto hashes the Any wrapper itself, type URL included. + // Pluto's equivalent (calculate.rs hash_any) is private, but it is exactly + // hash-of-the-Any-encoding, so hash_proto_bytes over the encoded Any exercises + // pluto's real merkleization path on the same bytes. + let suite = load_suite("qbft_hashing"); + let mut failures = Vec::new(); + for case in suite["any_string"].as_array().unwrap() { + let name = format!("any_string/{}", case["name"].as_str().unwrap()); + let s = case["input"]["string_value"].as_str().unwrap(); + // google.protobuf.StringValue: field 1, length-delimited. + let mut inner = Vec::new(); + prost::encoding::string::encode(1, &s.to_string(), &mut inner); + let any = prost_types::Any { + type_url: "type.googleapis.com/google.protobuf.StringValue".into(), + value: inner, + }; + let encoded = any.encode_to_vec(); + match hash_proto_bytes(&encoded) { + Ok(h) => check(&mut failures, &name, &encoded, h, case), + Err(e) => failures.push(format!("{name}: hash_proto_bytes failed: {e}")), + } + } + assert!(failures.is_empty(), "{}", failures.join("\n")); +} +``` + +- [ ] **Step 3: Run it** + +```bash +cargo test --test qbft_hashing +``` +Expected: all four groups pass — pluto already consumes a charon-v1.7.1 hashproto +golden file and none of these values moved after v1.7.1. If the `any_string` +*encoding* diverges (type URL prefix differs), decode the vector's `encoding_hex` +to see charon's actual type URL and fix the constructed `type_url`, not the assertion. + +- [ ] **Step 4: Record verdict in Results row 2; note in the row that pluto's own + `crates/consensus/testdata/vectors/hashproto.json` is now redundant with this suite + (the stated goal of the qbft_hashing suite) — a candidate upstream cleanup, not + something this repo changes.** + +- [ ] **Step 5: Commit** + +```bash +git add consumers/rust plans/pluto-conformance.md +git commit -m "Run qbft_hashing suite against pluto" +``` + +--- + +### Task 3: `bls_threshold` — Lagrange aggregation and recovery + +**Files:** +- Create: `consumers/rust/tests/bls_threshold.rs` +- Modify: `consumers/rust/Cargo.toml` (add `pluto-crypto = { path = "../../../pluto/crates/crypto" }`) + +**Interfaces:** +- Consumes: `load_suite`, `unhex`. +- Pluto APIs: `pluto_crypto::blst_impl::BlstImpl` implementing `pluto_crypto::tbls::Tbls` + (all trait methods pub); `pluto_crypto::types::{PublicKey, PrivateKey, Signature, Index}` + are plain byte arrays. Share indices are 1-based, matching the suite. + +- [ ] **Step 1: Write the test** + +The suite pins: per-share pubkeys and partial signatures over `message_hex`, four +different quorums whose threshold aggregates must equal the group signature, secret +recovery, and a plain aggregate that must **not** verify under the group key. + +```rust +use std::collections::HashMap; +use pluto_crypto::tbls::Tbls; +use pluto_crypto::blst_impl::BlstImpl; +use spec_vectors_pluto::{load_suite, unhex}; + +fn arr(v: &[u8]) -> [u8; N] { v.try_into().expect("length") } + +#[test] +fn bls_threshold() { + let suite = load_suite("bls_threshold"); + let t = BlstImpl::default(); + let msg = unhex(suite["message_hex"].as_str().unwrap()); + let group_pub: [u8; 48] = arr(&unhex(suite["group_pubkey_hex"].as_str().unwrap())); + let group_sig: [u8; 96] = arr(&unhex(suite["group_signature_hex"].as_str().unwrap())); + let mut failures = Vec::new(); + + // Index every share's secret, expected pubshare and expected partial signature. + let mut secrets: HashMap = HashMap::new(); + let mut partials: HashMap = HashMap::new(); + for p in suite["partials"].as_array().unwrap() { + let idx = p["input"]["share_idx"].as_u64().unwrap(); + let secret: [u8; 32] = arr(&unhex(p["input"]["secret_hex"].as_str().unwrap())); + let want_pub: [u8; 48] = arr(&unhex(p["pubshare_hex"].as_str().unwrap())); + let want_sig: [u8; 96] = arr(&unhex(p["signature_hex"].as_str().unwrap())); + let name = p["name"].as_str().unwrap(); + + match t.secret_to_public_key(&secret) { + Ok(pk) if pk == want_pub => {} + other => failures.push(format!("partials/{name}: pubshare {other:?}")), + } + match t.sign(&secret, &msg) { + Ok(sig) if sig == want_sig => {} + other => failures.push(format!("partials/{name}: partial sig {other:?}")), + } + secrets.insert(idx, secret); + partials.insert(idx, want_sig); + } + + // Four quorums: every threshold aggregate must equal the one group signature. + for c in suite["threshold_aggregates"].as_array().unwrap() { + let name = c["name"].as_str().unwrap(); + let want: [u8; 96] = arr(&unhex(c["signature_hex"].as_str().unwrap())); + let subset: HashMap = c["input"]["share_indices"].as_array().unwrap() + .iter().map(|i| { let i = i.as_u64().unwrap(); (i, partials[&i]) }).collect(); + match t.threshold_aggregate(&subset) { + Ok(sig) if sig == want => {} + other => failures.push(format!("threshold_aggregates/{name}: {other:?}")), + } + } + + for c in suite["recovery"].as_array().unwrap() { + let name = c["name"].as_str().unwrap(); + let want_pub: [u8; 48] = arr(&unhex(c["pubkey_hex"].as_str().unwrap())); + let subset: HashMap = c["input"]["share_indices"].as_array().unwrap() + .iter().map(|i| { let i = i.as_u64().unwrap(); (i, secrets[&i]) }).collect(); + match t.recover_secret(&subset).and_then(|s| t.secret_to_public_key(&s)) { + Ok(pk) if pk == want_pub => {} + other => failures.push(format!("recovery/{name}: {other:?}")), + } + } + + // Plain aggregation is NOT threshold aggregation: exact bytes match the vector, + // and the result must fail verification under the group key. + for c in suite["plain_aggregate"].as_array().unwrap() { + let name = c["name"].as_str().unwrap(); + let want: [u8; 96] = arr(&unhex(c["signature_hex"].as_str().unwrap())); + let sigs: Vec<[u8; 96]> = c["input"]["share_indices"].as_array().unwrap() + .iter().map(|i| partials[&i.as_u64().unwrap()]).collect(); + match t.aggregate(&sigs) { + Ok(sig) if sig == want => { + if t.verify(&group_pub, &msg, &sig).is_ok() { + failures.push(format!( + "plain_aggregate/{name}: verified under group key, must not")); + } + } + other => failures.push(format!("plain_aggregate/{name}: {other:?}")), + } + } + + if t.verify(&group_pub, &msg, &group_sig).is_err() { + failures.push("group signature does not verify under group pubkey".into()); + } + assert!(failures.is_empty(), "{}", failures.join("\n")); +} +``` + +- [ ] **Step 2: Run** `cargo test --test bls_threshold`. Expected: PASS — pluto's + `blst_impl` cites charon v1.7.1 parity and BLS did not change after it. + +- [ ] **Step 3: Record verdict in Results row 3; commit** + (`git commit -m "Run bls_threshold suite against pluto"`). + +--- + +### Task 4: `cluster_hashing` — config/definition/lock hashes and full lock verification + +**Files:** +- Create: `consumers/rust/tests/cluster_hashing.rs` +- Modify: `consumers/rust/Cargo.toml` (add `pluto-cluster = { path = "../../../pluto/crates/cluster" }`, + `tokio = { version = "", features = ["macros", "rt"] }`) + +**Interfaces:** +- Consumes: `load_suite` (case `input`s are verbatim charon cluster files — pass the + raw JSON straight to pluto's serde). +- Pluto APIs: `pluto_cluster::definition::Definition` (serde + `set_definition_hashes`, + `verify_hashes`), `pluto_cluster::lock::Lock` (`set_lock_hash`, `verify_hashes`, + async `verify_signatures(&EthClient)`). Raw hash functions are `pub(crate)`; going + through set-then-compare covers the same code. + +- [ ] **Step 1: Write the test** + +```rust +use spec_vectors_pluto::load_suite; +use pluto_cluster::{definition::Definition, lock::Lock}; + +#[test] +fn definition_hashes() { + let suite = load_suite("cluster_hashing"); + let mut failures = Vec::new(); + for case in suite["definition"].as_array().unwrap() { + let name = format!("definition/{}", case["name"].as_str().unwrap()); + let mut def: Definition = match serde_json::from_value(case["input"].clone()) { + Ok(d) => d, + Err(e) => { failures.push(format!("{name}: parse: {e}")); continue } + }; + if let Err(e) = def.set_definition_hashes() { + failures.push(format!("{name}: set_definition_hashes: {e}")); continue + } + let want_cfg = case["config_hash_hex"].as_str().unwrap(); + let want_def = case["definition_hash_hex"].as_str().unwrap(); + if hex::encode(&def.config_hash) != want_cfg { + failures.push(format!("{name}: config_hash {}, want {want_cfg}", + hex::encode(&def.config_hash))); + } + if hex::encode(&def.definition_hash) != want_def { + failures.push(format!("{name}: definition_hash {}, want {want_def}", + hex::encode(&def.definition_hash))); + } + } + assert!(failures.is_empty(), "{}", failures.join("\n")); +} + +#[tokio::test] +async fn lock_hashes_and_verification() { + let suite = load_suite("cluster_hashing"); + let mut failures = Vec::new(); + for case in suite["lock"].as_array().unwrap() { + let name = format!("lock/{}", case["name"].as_str().unwrap()); + let mut lock: Lock = match serde_json::from_value(case["input"].clone()) { + Ok(l) => l, + Err(e) => { failures.push(format!("{name}: parse: {e}")); continue } + }; + if let Err(e) = lock.set_lock_hash() { + failures.push(format!("{name}: set_lock_hash: {e}")); continue + } + let want = case["lock_hash_hex"].as_str().unwrap(); + if hex::encode(&lock.lock_hash) != want { + failures.push(format!("{name}: lock_hash {}, want {want}", + hex::encode(&lock.lock_hash))); + } + if let Err(e) = lock.verify_hashes() { + failures.push(format!("{name}: verify_hashes: {e}")); + } + } + assert!(failures.is_empty(), "{}", failures.join("\n")); +} +``` + +Before writing the field names for expected hashes, check the suite's actual keys +(`uv run python -c "import json; c=json.load(open('test_vectors/cluster_hashing.json')); +print(list(c['definition'][0]), list(c['lock'][0]))"`) — the plan's +`config_hash_hex`/`definition_hash_hex`/`lock_hash_hex` names must match what is there. + +- [ ] **Step 2: Add the full-verification check for `real_keys_3_of_4`** + +In the same file — this is the case that chains lock hash, public shares, plain BLS +aggregation and node signatures; only it carries real signatures: + +```rust +#[tokio::test] +async fn real_keys_lock_verifies_end_to_end() { + let suite = load_suite("cluster_hashing"); + let case = suite["lock"].as_array().unwrap().iter() + .find(|c| c["name"] == "real_keys_3_of_4").expect("real_keys_3_of_4 present"); + let lock: Lock = serde_json::from_value(case["input"].clone()).expect("parse lock"); + lock.verify_hashes().expect("hashes"); + // EthClient::new("") is pluto's no-op EL client: skips only contract (EIP-1271) + // signatures, still verifies EIP-712 operator sigs, the BLS aggregate over the + // lock hash against all public shares, and every node signature against the + // operator ENR keys. Confirm the exact constructor in crates/eth1wrap. + let eth1 = pluto_eth1wrap::EthClient::new(""); + lock.verify_signatures(ð1).await.expect("signature verification"); +} +``` + +(Add `pluto-eth1wrap = { path = "../../../pluto/crates/eth1wrap" }` — confirm crate +name and constructor; if `verify_signatures` demands a live EL for these operators' +sig type, record that limitation in the Results row instead of mocking one.) + +- [ ] **Step 3: Run** `cargo test --test cluster_hashing`. Expected: v1.10 files parse + (pluto supports V1_0…V1_10) and all hashes match — pluto's own testdata includes + charon's `cluster_lock_v1_10_0.json` golden. Parse *failures* here are findings: + the inputs are verbatim charon files, so a rejection means pluto cannot load a real + charon artifact. + +- [ ] **Step 4: Record verdict in Results row 4; commit** + (`git commit -m "Run cluster_hashing suite against pluto"`). + +--- + +### Task 5: `priority_scoring` — cluster-wide priority results + +The scoring function (`calculate_result`, `crates/priority/src/calculate.rs`) is +`pub(crate)`, so this goes end-to-end through `Prioritiser` with an in-process +libp2p network — pluto's own `crates/priority/tests/prioritiser_test.rs` is an +*external* test doing exactly this; copy its host-setup helpers. + +**Files:** +- Create: `consumers/rust/tests/priority_scoring.rs` +- Modify: `consumers/rust/Cargo.toml` (add `pluto-priority`, `pluto-p2p`, `libp2p`, + `async-trait` — versions from pluto's workspace; the `multistream-select` patch + from Task 1 becomes load-bearing here) + +**Interfaces:** +- Consumes: `load_suite`. +- Pluto APIs: `pluto_priority::{Prioritiser, component::sign_msg, PROTOCOL_ID}`, the + `Consensus` trait (implement a mock capturing the proposed `PriorityResult`), + `MsgVerifier` (`Box::new(|_| Ok(()))`), proto types `PriorityMsg`, `PriorityTopicProposal`. + +- [ ] **Step 1: Write the harness** + +Shape (helpers copied from `prioritiser_test.rs` — that file compiles against public +API only, so everything it does is available here): + +```rust +// Per case: +// 1. Generate input.peers.len() secp256k1 keypairs; derive libp2p PeerIds; SORT the +// PeerIds and assign them to the vector's peers ("0", "1", ...) in order — the +// vector's expected ordering assumes ascending peer order, and scoring tie-breaks +// are first-seen over peer_id-sorted input. +// 2. Start one in-process host per peer (swarm + Prioritiser::new_internal), with: +// min_required = input.min_required, +// msg_validator = Box::new(|_| Ok(())), +// consensus = Arc capturing propose_priority's PriorityResult +// into a oneshot channel (leader only proposes once). +// 3. Build each peer's PriorityMsg: duty = Duty{slot: input.slot, type: PRIORITY}, +// topics = [Any-wrapped input.topic proposal with that peer's input priorities, +// plus an Any-wrapped input.ignored_topic proposal] — mirror how +// prioritiser_test.rs builds topic proposals; sign with +// pluto_priority::component::sign_msg(&msg, &peer_secret). +// 4. Call prioritise() on every host concurrently; await the captured result with +// a timeout (10s). +// 5. Compare: the result's entry for input.topic must list exactly the case's +// `result` array in order — (priority, score) pairs if the vector carries +// scores, else priorities_only(). +``` + +Check the vector's `result` element shape first +(`uv run python -c "import json; d=json.load(open('test_vectors/priority_scoring.json')); +print([c['result'] for c in d['cases'] if c['result']][:2])"`) and assert score values +if present, order-only otherwise. Cases where `result` is `[]` assert the topic is +absent (below `min_required`) — an empty expected list is a *rejection* pin, not a +skip. + +- [ ] **Step 2: Add the protocol-ID pin (ladder item 6)** + +```rust +#[test] +fn priority_protocol_ids() { + // Ladder: "Preferred priority protocol ID /charon/priority/2.0.0" is unreleased + // (first_charon_release: null). At charon-v1.7.1 parity pluto must serve exactly + // the legacy slash-less ID. When pluto adds the preferred ID, this test fails: + // flip it to assert both IDs with the slash form preferred, per priority.md. + assert_eq!(pluto_priority::PROTOCOL_ID, "charon/priority/2.0.0"); + assert_eq!(pluto_priority::protocols(), vec!["charon/priority/2.0.0"]); +} +``` + +- [ ] **Step 3: Run** `cargo test --test priority_scoring`. Expected: scoring cases + PASS (the algorithm predates v1.7.1 and pluto's stable sort matches the spec). + Watch the ties: charon v1.7.1 used an unstable sort — if any tie-case diverges, + that is ladder entry "Stable sort of scored priorities" → triage before verdicting. + +- [ ] **Step 4: Record verdict in Results row 5; commit** + (`git commit -m "Run priority_scoring suite against pluto"`). + +--- + +### Task 6: `timer_deadlines` — round timeouts under a paused clock + +The vectors give `duty_start_delay_nanos`, `round_timeout_nanos` and absolute +`deadline_nanos` (= genesis + slot·duration + delay + timeout). Pluto's timers are +*relative* and the raw duration formulas are private, so this splits three ways: + +- `round_timeout_nanos`: **testable** — await `RoundTimer::timer(round)` under + `#[tokio::test(start_paused = true)]` and measure the deadline delta. +- `deadline_nanos` (genesis-derived determinism): ladder entry "Deterministic + (genesis-derived) eager double linear round deadlines" (v1.9.0 > v1.7.1) → expected + ABSENT-OK unless pluto's `with_duty` timers turn out to consume genesis (recent + pluto commits touch the simnet duty cycle — check before assuming). +- `duty_start_delay_nanos`: `delay_slot_offset` is private in pluto's scheduler → + expected UNREACHABLE; record `crates/core/src/scheduler.rs:853` in the row. + +**Files:** +- Create: `consumers/rust/tests/timer_deadlines.rs` +- Modify: `consumers/rust/Cargo.toml` (`tokio` gains `["test-util", "time"]`, + add `pluto-featureset = { path = "../../../pluto/crates/featureset" }` if + `FeatureSet` lives there) + +- [ ] **Step 1: Write the test** + +```rust +use std::time::Duration; +use spec_vectors_pluto::load_suite; +use pluto_consensus::timer::{EagerDoubleLinearRoundTimer, RoundTimer}; + +#[tokio::test(start_paused = true)] +async fn round_timeouts() { + let suite = load_suite("timer_deadlines"); + let mut failures = Vec::new(); + for case in suite["cases"].as_array().unwrap() { + let name = case["name"].as_str().unwrap(); + let round = case["input"]["round"].as_i64().unwrap(); + let want = Duration::from_nanos(case["round_timeout_nanos"].as_u64().unwrap()); + + // The default timer (no features) for every duty type at charon v1.7.1. + // If the vectors select timers per duty type (proposer → linear), branch on + // input.duty_type_name here, mirroring pluto's get_round_timer_func selection. + let timer = EagerDoubleLinearRoundTimer::new(); + let start = tokio::time::Instant::now(); + let fut = timer.timer(round).expect("timer future"); + let deadline = fut.await; // paused clock: resolves at its Instant immediately + let got = deadline - start; + if got != want { + failures.push(format!("{name}: round timeout {got:?}, want {want:?}")); + } + } + assert!(failures.is_empty(), "{}", failures.join("\n")); +} +``` + +Before running, read `~/pluto/crates/consensus/src/timer.rs` tests for how they await +the timer future under a paused clock and copy that pattern — if the future resolves +to its deadline `Instant` without needing `tokio::time::advance`, the above works; if +it sleeps, wrap with `tokio::time::timeout` + `advance`. + +- [ ] **Step 2: Determine the two remaining columns' verdicts** + +Read `EagerDoubleLinearRoundTimer::with_duty` and its callers: if no genesis/slot +timing enters the deadline, mark `deadline_nanos` ABSENT-OK (ladder v1.9.0 entry) +and add a pinned-divergence comment in the test naming the entry. Confirm +`delay_slot_offset` is still private → mark `duty_start_delay_nanos` UNREACHABLE +with file:line. + +- [ ] **Step 3: Run** `cargo test --test timer_deadlines`. Triage: mismatches on + linear rounds ≥ 2 are the known nanosecond bug (ladder "Linear round timer + subsequent-round timeout fix", `null`) — if the vectors' proposer/linear cases hit + it, split those cases into a pinned-divergence check asserting pluto's 200ns + schedule, named for the ladder entry, and keep the rest strict. + +- [ ] **Step 4: Record verdict in Results row 6 (verdict per column: + timeout / deadline / start-delay); commit** + (`git commit -m "Run timer_deadlines suite against pluto"`). + +--- + +### Task 7: `qbft_msg_limits` — justification/value/wire-size rejection + +**Files:** +- Create: `consumers/rust/tests/qbft_msg_limits.rs` + +**Interfaces:** +- Pluto APIs: `pluto_consensus::qbft::{Consensus, Config, Peer}`, + `pluto_consensus::qbft::p2p::MAX_CONSENSUS_MSG_SIZE`, + `pluto_p2p::proto::{MAX_MESSAGE_SIZE, read_protobuf_with_max_size}`, + `pluto_core::deadline::{DeadlinerHandle, AddOutcome}`, error variant + `Error::TooManyJustifications`. Messages are signed by reproducing pluto's + `sign_msg`: clear signature → `hash_proto` → `pluto_k1util::sign` (Task 2 proved + the root; Task 1 proved the signature). + +- [ ] **Step 1: Write the `counts` half** + +```rust +// Per `counts` case (input: nodes, justification_count, value_count): +// 1. Build a `Consensus` for `nodes` peers: generate k256 keys, Peer{index, name, +// public_key}; local_peer_idx 0; DeadlinerHandle::always(AddOutcome::Scheduled); +// duty_gater = Arc::new(|_| true); broadcaster/sniffer = no-op closures; +// timer_func = get_round_timer_func(FeatureSet::default()); a dummy expired_rx. +// Copy the wiring from crates/consensus/examples/qbft.rs (external-crate-shaped). +// 2. Build a signed QbftConsensusMsg from peer 1 with `justification_count` +// justification msgs (each itself signed) and `value_count` Any values. +// 3. `consensus.handle(msg, &ct).await`: +// accepted == true → expect Ok(()) (well-formed msg required) +// reason "too_many_justifications" → expect Err(TooManyJustifications{..}) +// reason "too_many_values" → see below +// +// Pre-registered divergences (ladder entry "QBFT DECIDED-resend rate limit and +// message size/count limits", first_charon_release: null — absent at v1.7.1): +// - pluto's cap is 4n, not the spec's 2n: cases between 2n+1 and 4n will be +// ACCEPTED. Pin that with an explicit ABSENT-OK branch asserting Ok(()) and +// naming the ladder entry AND pluto's own constant (component.rs +// MAX_JUSTIFICATIONS_PER_NODE = 4 — hardening pluto chose, neither charon +// v1.7.1 nor spec; record as a Finding). +// - `too_many_values` cases: no values cap exists — pin acceptance, ABSENT-OK. +// Cases the spec accepts must be accepted by pluto unconditionally: over-rejection +// breaks liveness and no ladder entry can excuse it. +``` + +- [ ] **Step 2: Write the `wire_size` half** + +```rust +#[test] +fn wire_size_constants() { + // The 32 MiB consensus limit is a spec-pinned narrowing of libp2p's 128 MiB + // default; both constants are public in pluto. + assert_eq!(pluto_consensus::qbft::p2p::MAX_CONSENSUS_MSG_SIZE, 32 * 1024 * 1024); + assert_eq!(pluto_p2p::proto::MAX_MESSAGE_SIZE, 128 << 20); +} + +#[tokio::test] +async fn wire_size_enforcement() { + // Per wire_size case: feed a length-prefixed frame of input.wire_size_bytes + // through read_protobuf_with_max_size(&mut stream, MAX_CONSENSUS_MSG_SIZE) + // over an in-memory duplex (tokio::io::duplex). accepted → decodes (use a + // padded valid message); rejected → Err. Mirror how qbft/p2p.rs invokes the + // reader so the same code path is under test. +} +``` + +- [ ] **Step 3: Run** `cargo test --test qbft_msg_limits`, triage every red case + against the pre-registered expectations — anything *outside* them (e.g. an + at-limit case rejected) is a FAIL finding. + +- [ ] **Step 4: Record verdict in Results row 7 and the 4n-cap finding under + Findings; commit** (`git commit -m "Run qbft_msg_limits suite against pluto"`). + +--- + +### Task 8: `qbft_decided_resends` — post-decision rebroadcast limiting + +Pre-registered: pluto has **no limiter** (`crates/core/src/qbft/mod.rs:494-503` +rebroadcasts on every post-decision ROUND-CHANGE from another source). Ladder entry +`null` → expected ABSENT-OK. The check still runs the real state machine, because +"absent" asserted by inspection alone goes stale. + +**Files:** +- Create: `consumers/rust/tests/qbft_decided_resends.rs` + +**Interfaces:** +- Pluto APIs: `pluto_core::qbft::{run, Definition, Transport, QbftLogger, MessageType, + MSG_ROUND_CHANGE, MSG_DECIDED, SomeMsg}` — all public with public fields; implement + `QbftTypes` and `SomeMsg` for a minimal test message type. + +- [ ] **Step 1: Write the harness** + +```rust +// Per case (input: nodes, decided_round, events[{source, round}], rebroadcast[i]): +// 1. Implement TestTypes: QbftTypes with a trivial Value/Compare, and TestMsg: +// SomeMsg returning the constructed type/round/source/value. +// 2. Definition { nodes, fifo_limit: 100, is_leader: |_,_,_| false (peer under test +// never leads), new_timer: a never-firing timer, decide/compare/logger: no-ops }. +// 3. Transport.broadcast records every outgoing MSG_DECIDED into a Vec; +// Transport.receive is a channel the test feeds. +// 4. Drive the instance to decision at input.decided_round: feed a justified +// PRE-PREPARE for that round plus `quorum(nodes)` COMMITs — crib the exact +// minimal message sequence from pluto's own core qbft tests +// (crates/core/src/qbft/, internal but readable) or charon's TestDecidedRebroadcastLimits. +// 5. Feed each event as a ROUND-CHANGE from event.source at event.round; after each, +// record whether a new DECIDED broadcast appeared. +// 6. Compare the per-event bool sequence with the vector's `rebroadcast` array under +// BOTH readings: +// spec reading — expect mismatches once a source exceeds 16 or repeats a round +// (limiter absent), i.e. the conformance columns FAIL; +// pin reading — pluto rebroadcasts on EVERY event from another source; assert +// that exactly, named for the ladder entry, so the test flips when pluto adds +// the limiter. +// The test passes on the pin reading and prints the spec-reading mismatch count. +``` + +- [ ] **Step 2: Run** `cargo test --test qbft_decided_resends`. + +Fallback, only if driving `run` externally proves impractical after a real attempt +(e.g. `SomeMsg` cannot be implemented outside the crate): verdict the row +UNREACHABLE, cite `crates/core/src/qbft/mod.rs:494-503` as the inspection evidence +for ABSENT-OK, and leave the test file in place with `#[ignore]` and a comment +explaining what blocked it. + +- [ ] **Step 3: Record verdict in Results row 8; commit** + (`git commit -m "Run qbft_decided_resends suite against pluto"`). + +--- + +### Task 9: `parsigex_sender_binding` — share-index binding and peer-map validation + +In charon this rule lives in the **DKG lock-hash exchanger** (`dkg/exchanger.go`). +First establish where pluto's equivalent is: grep `~/pluto/crates/dkg` and +`~/pluto/crates/parsigex` for `share_idx` validation and exchanger construction. +Pre-registered: no sender binding exists in the parsigex crate; ladder entry +"Sender-bound share indices in the DKG lock-hash exchange" (`null`) → ABSENT-OK +expected for the `cases` half. The `peer_map` half (complete-map validation, +positive share indices) predates v1.7.1 in charon, so it must PASS if the +construction is reachable. + +**Files:** +- Create: `consumers/rust/tests/parsigex_sender_binding.rs` +- Modify: `consumers/rust/Cargo.toml` (add `pluto-parsigex`, and `pluto-dkg` if the + exchanger lives there) + +- [ ] **Step 1: Probe and write the test** + +```rust +// cases (input: share_idx_by_peer {self, other}, sender, share_idx; accepted, reason): +// If pluto has a sender-bound verifier (dkg exchanger or parsigex): drive it +// directly per case. If not (pre-registered): pin the absence — build the closest +// verifier pluto has, pluto_parsigex::new_eth2_verifier with a pub-shares map, show +// a wrong-sender share_idx passes index lookup (the binding never fires), assert +// that, name the ladder entry, verdict ABSENT-OK. +// peer_map (input: peers, share_idx_by_peer, peer_idx; accepted, reason): +// Map to pluto's construction-time validation: whichever function builds the +// peer→share-index map for the exchange (Definition::node_idx, dkg setup, or the +// pub_shares_by_key map builder). Assert: complete map accepted; a participant +// with no assigned index rejected (`unknown_peer`); non-positive share index +// rejected (`missing_share_idx`). If the only builder is private +// (crates/app/src/node/mod.rs:800), verdict that half UNREACHABLE with file:line. +``` + +- [ ] **Step 2: Run** `cargo test --test parsigex_sender_binding`, triage: a missing + *peer-map* validation (accepting an incomplete map) is a FAIL finding — charon + v1.7.1 already rejects it, no ladder cover. Sender-binding absence is ABSENT-OK. + +- [ ] **Step 3: Record verdict in Results row 9; commit** + (`git commit -m "Run parsigex_sender_binding suite against pluto"`). + +--- + +### Task 10: Coverage guard, docs, and wrap-up + +**Files:** +- Create: `consumers/rust/tests/coverage.rs` +- Modify: `consumers/README.md` (final Pluto row: suites run, pluto commit, verdicts) +- Modify: `tests/test_consumers.py` (extend to the Rust consumer) +- Modify: `plans/spec-completion.md` (Phase 3 item 2 pluto bullet → point here) + +- [ ] **Step 1: Write the coverage guard** + +The Go consumer's `TestEverySuiteIsCovered` has an equivalent here: an uncovered +suite is indistinguishable from a passing one. + +```rust +use std::collections::BTreeSet; + +/// Suite name -> test file that runs it. Update when a suite or test is added. +const COVERED: &[(&str, &str)] = &[ + ("secp256k1_signatures", "tests/secp256k1_signatures.rs"), + ("qbft_hashing", "tests/qbft_hashing.rs"), + ("bls_threshold", "tests/bls_threshold.rs"), + ("cluster_hashing", "tests/cluster_hashing.rs"), + ("priority_scoring", "tests/priority_scoring.rs"), + ("timer_deadlines", "tests/timer_deadlines.rs"), + ("qbft_msg_limits", "tests/qbft_msg_limits.rs"), + ("qbft_decided_resends", "tests/qbft_decided_resends.rs"), + ("parsigex_sender_binding", "tests/parsigex_sender_binding.rs"), +]; + +#[test] +fn every_suite_is_covered() { + let published: BTreeSet = std::fs::read_dir(spec_vectors_pluto::vectors_dir()) + .expect("vectors dir") + .filter_map(|e| { + let p = e.expect("dir entry").path(); + (p.extension()? == "json") + .then(|| p.file_stem().unwrap().to_string_lossy().into_owned()) + }) + .collect(); + let covered: BTreeSet = COVERED.iter().map(|(s, _)| s.to_string()).collect(); + assert_eq!(published, covered, + "published suites and covered suites disagree — a suite nothing runs is a document, not a test"); + for (suite, file) in COVERED { + assert!(std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join(file).exists(), + "{suite}: declared test file {file} does not exist"); + } +} +``` + +- [ ] **Step 2: Extend `tests/test_consumers.py`** + +Mirror the existing Go mapping check: assert each of the nine suites appears in +`consumers/rust/tests/coverage.rs`'s `COVERED` table and that each declared test +file exists — read the file with a regex over the `("suite", "file")` pairs, the +same technique the file already uses for the Go side (the Rust code is not compiled +here either). Run `uv run pytest tests/test_consumers.py` and confirm green. + +- [ ] **Step 3: Finalize docs** + +- `consumers/README.md`: Pluto row lists suites run / subcheck count / pluto commit / + verdict summary, and a "running it" snippet: + ```bash + # pluto checked out as a sibling of this repo (default: ../pluto) + cd consumers/rust && cargo test + ``` +- `plans/spec-completion.md` Phase 3 item 2: change the pluto bullet from + "not started" to a pointer at this plan and its Results table. + +- [ ] **Step 4: Full run + quality checks** + +```bash +cd consumers/rust && cargo fmt --check && cargo clippy --all-targets && cargo test +cd ../.. && uv run tox -e all-checks && uv run pytest tests/test_consumers.py +``` + +- [ ] **Step 5: Record Results row 10, finalize the Findings section (each finding: + divergence, pluto file:line, ladder cover or not, reported upstream or not), + commit, and open the PR** + +```bash +git add -A && git commit -m "Add coverage guard and docs for the pluto consumer" +gh pr create --title "Rust consumer: run spec vectors against pluto" \ + --body "..." # summarize verdicts per suite; plain factual language +``` From 5e70926ce5020b32785112355858516d4e59ad81 Mon Sep 17 00:00:00 2001 From: Andrei Smirnov Date: Fri, 31 Jul 2026 20:19:36 +0300 Subject: [PATCH 02/27] Add Rust consumer scaffold and secp256k1 suite for pluto --- consumers/README.md | 18 +++++++- consumers/rust/.gitignore | 2 + consumers/rust/Cargo.toml | 19 ++++++++ consumers/rust/rust-toolchain.toml | 3 ++ consumers/rust/src/lib.rs | 33 ++++++++++++++ consumers/rust/tests/secp256k1_signatures.rs | 46 ++++++++++++++++++++ plans/pluto-conformance.md | 4 +- 7 files changed, 122 insertions(+), 3 deletions(-) create mode 100644 consumers/rust/.gitignore create mode 100644 consumers/rust/Cargo.toml create mode 100644 consumers/rust/rust-toolchain.toml create mode 100644 consumers/rust/src/lib.rs create mode 100644 consumers/rust/tests/secp256k1_signatures.rs diff --git a/consumers/README.md b/consumers/README.md index df9f5b0..95cddf9 100644 --- a/consumers/README.md +++ b/consumers/README.md @@ -7,7 +7,7 @@ executes them: a suite nobody runs is a document, not a test. | Consumer | Status | Location | | --------------------------------- | ---------------------------------------------- | ----------- | | Charon (Go) | All 9 suites, 314 subtests, verified at anchor `6054bcb2` | [`go/`](go) | -| Pluto (Rust) | Not written yet | — | +| Pluto (Rust) | in progress — see plans/pluto-conformance.md | [rust/](rust) | These files live here rather than in Charon because this repository cannot merge into Charon. They are laid out to mirror Charon's own tree, so placing them is a @@ -81,3 +81,19 @@ produced by Charon and then published, so a failure means Charon has moved away from the protocol every other implementation was told to implement. The regeneration path is deliberately not automatic — see the spec's `test_vectors/README.md`. + +## Pluto consumer + +`rust/` is a standalone Cargo package that path-depends on a pluto checkout +placed as a sibling of this repository (`../pluto` relative to +`distributed-validator-specs/`, i.e. `../../../pluto` from `consumers/rust/`). +Unlike the Go consumer, nothing is copied into pluto and pluto is never +modified — the harness reads pluto's crates in place. + +```bash +cd consumers/rust && cargo test +``` + +Vectors are read from `../../test_vectors/` at test time (no vendoring), or +from `SPEC_VECTORS_DIR` when set. Progress and per-suite verdicts are tracked +in `plans/pluto-conformance.md`. diff --git a/consumers/rust/.gitignore b/consumers/rust/.gitignore new file mode 100644 index 0000000..96ef6c0 --- /dev/null +++ b/consumers/rust/.gitignore @@ -0,0 +1,2 @@ +/target +Cargo.lock diff --git a/consumers/rust/Cargo.toml b/consumers/rust/Cargo.toml new file mode 100644 index 0000000..443cade --- /dev/null +++ b/consumers/rust/Cargo.toml @@ -0,0 +1,19 @@ +[package] +name = "spec-vectors-pluto" +version = "0.1.0" +edition = "2024" +publish = false +description = "Runs the spec's test_vectors/ suites against a pluto checkout at ../../../pluto" + +[dependencies] +serde_json = "1.0" +hex = "0.4.3" + +[dev-dependencies] +serde_json = "1.0" +hex = "0.4.3" +pluto-k1util = { path = "../../../pluto/crates/k1util" } +k256 = { version = "0.13.4", features = ["ecdsa", "sha256"] } + +[patch.crates-io] +multistream-select = { path = "../../../pluto/third_party/multistream-select" } diff --git a/consumers/rust/rust-toolchain.toml b/consumers/rust/rust-toolchain.toml new file mode 100644 index 0000000..6360c18 --- /dev/null +++ b/consumers/rust/rust-toolchain.toml @@ -0,0 +1,3 @@ +[toolchain] +channel = "1.95.0" +components = ["rustfmt", "clippy"] diff --git a/consumers/rust/src/lib.rs b/consumers/rust/src/lib.rs new file mode 100644 index 0000000..c71f0f1 --- /dev/null +++ b/consumers/rust/src/lib.rs @@ -0,0 +1,33 @@ +//! Loader for the spec's test_vectors/ suites. +//! +//! Vectors are read from `../../test_vectors/` relative to this package, or from +//! `SPEC_VECTORS_DIR` when set. `load_suite` refuses a file whose `suite` field +//! does not match the requested name, so a stray copy cannot silently substitute. + +use std::path::PathBuf; + +pub fn vectors_dir() -> PathBuf { + match std::env::var_os("SPEC_VECTORS_DIR") { + Some(dir) => PathBuf::from(dir), + None => PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../test_vectors"), + } +} + +pub fn load_suite(name: &str) -> serde_json::Value { + let path = vectors_dir().join(format!("{name}.json")); + let raw = + std::fs::read_to_string(&path).unwrap_or_else(|e| panic!("read {}: {e}", path.display())); + let suite: serde_json::Value = + serde_json::from_str(&raw).unwrap_or_else(|e| panic!("parse {}: {e}", path.display())); + assert_eq!( + suite["suite"], + name, + "suite field mismatch in {}", + path.display() + ); + suite +} + +pub fn unhex(s: &str) -> Vec { + hex::decode(s).unwrap_or_else(|e| panic!("bad hex {s:?}: {e}")) +} diff --git a/consumers/rust/tests/secp256k1_signatures.rs b/consumers/rust/tests/secp256k1_signatures.rs new file mode 100644 index 0000000..eb1d441 --- /dev/null +++ b/consumers/rust/tests/secp256k1_signatures.rs @@ -0,0 +1,46 @@ +use spec_vectors_pluto::{load_suite, unhex}; + +#[test] +fn secp256k1_signatures() { + let suite = load_suite("secp256k1_signatures"); + let secret = k256::SecretKey::from_slice(&unhex(suite["secret_hex"].as_str().unwrap())) + .expect("suite secret key"); + let pubkey = secret.public_key(); + let mut failures = Vec::new(); + + for case in suite["cases"].as_array().unwrap() { + let name = case["name"].as_str().unwrap(); + let hash = unhex(case["input"]["hash_hex"].as_str().unwrap()); + let want_sig = unhex(case["signature_hex"].as_str().unwrap()); + let want_pub = unhex(case["recovered_pubkey_hex"].as_str().unwrap()); + + match pluto_k1util::sign(&secret, &hash) { + Ok(sig) if sig.to_vec() == want_sig => {} + Ok(sig) => failures.push(format!( + "{name}: sign gave {}, want {}", + hex::encode(sig), + hex::encode(&want_sig) + )), + Err(e) => failures.push(format!("{name}: sign failed: {e}")), + } + match pluto_k1util::recover(&hash, &want_sig) { + Ok(rec) if rec.to_sec1_bytes().to_vec() == want_pub => {} + Ok(rec) => failures.push(format!( + "{name}: recover gave {}, want {}", + hex::encode(rec.to_sec1_bytes()), + hex::encode(&want_pub) + )), + Err(e) => failures.push(format!("{name}: recover failed: {e}")), + } + match pluto_k1util::verify_65(&pubkey, &hash, &want_sig) { + Ok(true) => {} + other => failures.push(format!("{name}: verify_65 gave {other:?}, want Ok(true)")), + } + } + assert!( + failures.is_empty(), + "{} failures:\n{}", + failures.len(), + failures.join("\n") + ); +} diff --git a/plans/pluto-conformance.md b/plans/pluto-conformance.md index 3254de8..585cb8e 100644 --- a/plans/pluto-conformance.md +++ b/plans/pluto-conformance.md @@ -63,11 +63,11 @@ or refuted by the task that covers it: Progress tracking lives here. Update the row when a task's verdict is in; record the pluto commit actually tested. Statuses: `todo` / `in progress` / `done`. -**Pluto commit under test:** _(fill in at Task 1: `git -C ~/pluto rev-parse --short HEAD`)_ +**Pluto commit under test:** `67088a2` | # | Check | Suite | Status | Verdict | Notes | | --- | --- | --- | --- | --- | --- | -| 1 | Harness scaffold + secp256k1 | `secp256k1_signatures` | todo | — | | +| 1 | Harness scaffold + secp256k1 | `secp256k1_signatures` | done | PASS | Both cases pass: sign, recover, and verify_65 all match the vector via `pluto-k1util` (33-byte SEC1 pubkey). | | 2 | Proto encoding + SSZ hashing | `qbft_hashing` | todo | — | | | 3 | BLS threshold aggregation | `bls_threshold` | todo | — | | | 4 | Cluster hashes + lock verification | `cluster_hashing` | todo | — | | From 73cb64b0b9ef89532748fd350616d9d63f8d57b6 Mon Sep 17 00:00:00 2001 From: Andrei Smirnov Date: Fri, 31 Jul 2026 20:27:27 +0300 Subject: [PATCH 03/27] Prefix suite name in failure labels, drop redundant dev-deps --- consumers/rust/Cargo.toml | 2 -- consumers/rust/tests/secp256k1_signatures.rs | 12 +++++++----- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/consumers/rust/Cargo.toml b/consumers/rust/Cargo.toml index 443cade..6731049 100644 --- a/consumers/rust/Cargo.toml +++ b/consumers/rust/Cargo.toml @@ -10,8 +10,6 @@ serde_json = "1.0" hex = "0.4.3" [dev-dependencies] -serde_json = "1.0" -hex = "0.4.3" pluto-k1util = { path = "../../../pluto/crates/k1util" } k256 = { version = "0.13.4", features = ["ecdsa", "sha256"] } diff --git a/consumers/rust/tests/secp256k1_signatures.rs b/consumers/rust/tests/secp256k1_signatures.rs index eb1d441..8cfdb68 100644 --- a/consumers/rust/tests/secp256k1_signatures.rs +++ b/consumers/rust/tests/secp256k1_signatures.rs @@ -17,24 +17,26 @@ fn secp256k1_signatures() { match pluto_k1util::sign(&secret, &hash) { Ok(sig) if sig.to_vec() == want_sig => {} Ok(sig) => failures.push(format!( - "{name}: sign gave {}, want {}", + "secp256k1_signatures/{name}: sign gave {}, want {}", hex::encode(sig), hex::encode(&want_sig) )), - Err(e) => failures.push(format!("{name}: sign failed: {e}")), + Err(e) => failures.push(format!("secp256k1_signatures/{name}: sign failed: {e}")), } match pluto_k1util::recover(&hash, &want_sig) { Ok(rec) if rec.to_sec1_bytes().to_vec() == want_pub => {} Ok(rec) => failures.push(format!( - "{name}: recover gave {}, want {}", + "secp256k1_signatures/{name}: recover gave {}, want {}", hex::encode(rec.to_sec1_bytes()), hex::encode(&want_pub) )), - Err(e) => failures.push(format!("{name}: recover failed: {e}")), + Err(e) => failures.push(format!("secp256k1_signatures/{name}: recover failed: {e}")), } match pluto_k1util::verify_65(&pubkey, &hash, &want_sig) { Ok(true) => {} - other => failures.push(format!("{name}: verify_65 gave {other:?}, want Ok(true)")), + other => failures.push(format!( + "secp256k1_signatures/{name}: verify_65 gave {other:?}, want Ok(true)" + )), } } assert!( From 7952ce54142ad03655b1666142439d788ca0c2d3 Mon Sep 17 00:00:00 2001 From: Andrei Smirnov Date: Fri, 31 Jul 2026 20:40:40 +0300 Subject: [PATCH 04/27] Run qbft_hashing suite against pluto --- consumers/rust/Cargo.toml | 4 + consumers/rust/tests/qbft_hashing.rs | 172 +++++++++++++++++++++++++++ plans/pluto-conformance.md | 20 +++- 3 files changed, 195 insertions(+), 1 deletion(-) create mode 100644 consumers/rust/tests/qbft_hashing.rs diff --git a/consumers/rust/Cargo.toml b/consumers/rust/Cargo.toml index 6731049..9ae5859 100644 --- a/consumers/rust/Cargo.toml +++ b/consumers/rust/Cargo.toml @@ -11,7 +11,11 @@ hex = "0.4.3" [dev-dependencies] pluto-k1util = { path = "../../../pluto/crates/k1util" } +pluto-core = { path = "../../../pluto/crates/core" } +pluto-consensus = { path = "../../../pluto/crates/consensus" } k256 = { version = "0.13.4", features = ["ecdsa", "sha256"] } +prost = "0.14" +prost-types = "0.14" [patch.crates-io] multistream-select = { path = "../../../pluto/third_party/multistream-select" } diff --git a/consumers/rust/tests/qbft_hashing.rs b/consumers/rust/tests/qbft_hashing.rs new file mode 100644 index 0000000..397ffbf --- /dev/null +++ b/consumers/rust/tests/qbft_hashing.rs @@ -0,0 +1,172 @@ +use prost::Message; +use spec_vectors_pluto::{load_suite, unhex}; + +use pluto_consensus::qbft::msg::{hash_proto, hash_proto_bytes}; +use pluto_core::corepb::v1::{Duty, QbftMsg, UnsignedDataSet}; + +fn check( + failures: &mut Vec, + name: &str, + encoded: &[u8], + hash: [u8; 32], + case: &serde_json::Value, +) { + let want_enc = unhex(case["encoding_hex"].as_str().unwrap()); + let want_hash = unhex(case["hash_hex"].as_str().unwrap()); + if encoded != want_enc { + failures.push(format!( + "{name}: encoding {}, want {}", + hex::encode(encoded), + hex::encode(&want_enc) + )); + } + if hash.as_slice() != want_hash { + failures.push(format!( + "{name}: hash {}, want {}", + hex::encode(hash), + hex::encode(&want_hash) + )); + } +} + +#[test] +fn duty_hashing() { + let suite = load_suite("qbft_hashing"); + let mut failures = Vec::new(); + for case in suite["duty"].as_array().unwrap() { + let name = format!("qbft_hashing/duty/{}", case["name"].as_str().unwrap()); + let duty = Duty { + // Duty.slot is a proto uint64 (generated as Rust u64); the max-slot + // vector exercises the full u64 range, which as_i64 cannot hold. + slot: case["input"]["slot"].as_u64().unwrap(), + r#type: case["input"]["type"].as_i64().unwrap() as i32, + }; + // Covers the <=32-byte rule: a short encoding is returned zero-padded, + // unhashed. hash_proto/hash_proto_bytes already implement this; the + // test only checks the outcome. + match hash_proto(&duty) { + Ok(h) => check(&mut failures, &name, &duty.encode_to_vec(), h, case), + Err(e) => failures.push(format!("{name}: hash_proto failed: {e}")), + } + } + assert_eq!( + suite["duty"].as_array().unwrap().len(), + 3, + "expected 3 duty cases" + ); + assert!(failures.is_empty(), "{}", failures.join("\n")); +} + +#[test] +fn unsigned_data_set_hashing() { + let suite = load_suite("qbft_hashing"); + let mut failures = Vec::new(); + for case in suite["unsigned_data_set"].as_array().unwrap() { + let name = format!( + "qbft_hashing/unsigned_data_set/{}", + case["name"].as_str().unwrap() + ); + // Map keys are the literal strings from the vector (e.g. "0xaabb"), + // not hex-decoded; only values are hex-encoded bytes. pluto's set is + // BTreeMap, whose key ordering gives the deterministic + // encoding regardless of the vector's insertion order. + let set = case["input"]["set"] + .as_object() + .unwrap() + .iter() + .map(|(k, v)| (k.clone(), unhex(v.as_str().unwrap()).into())) + .collect(); + let uds = UnsignedDataSet { set }; + match hash_proto(&uds) { + Ok(h) => check(&mut failures, &name, &uds.encode_to_vec(), h, case), + Err(e) => failures.push(format!("{name}: hash_proto failed: {e}")), + } + } + assert_eq!( + suite["unsigned_data_set"].as_array().unwrap().len(), + 12, + "expected 12 unsigned_data_set cases" + ); + assert!(failures.is_empty(), "{}", failures.join("\n")); +} + +#[test] +fn qbft_signing_roots() { + let suite = load_suite("qbft_hashing"); + let mut failures = Vec::new(); + for case in suite["qbft_signing_root"].as_array().unwrap() { + let name = format!( + "qbft_hashing/qbft_signing_root/{}", + case["name"].as_str().unwrap() + ); + let i = &case["input"]; + // value_hash / prepared_value_hash are always 32 bytes on the wire, + // zeros meaning "no value". The signing root is computed with the + // signature field cleared (mirrors pluto's own sign_msg), which is why + // "attester_pre_prepare" and "attester_pre_prepare_signed" share a + // root despite the vector carrying a non-empty input.signature. + let msg = QbftMsg { + r#type: i["type"].as_i64().unwrap(), + duty: Some(Duty { + slot: i["slot"].as_u64().unwrap(), + r#type: i["duty_type"].as_i64().unwrap() as i32, + }), + peer_idx: i["peer_idx"].as_i64().unwrap(), + round: i["round"].as_i64().unwrap(), + prepared_round: i["prepared_round"].as_i64().unwrap(), + value_hash: unhex(i["value_hash"].as_str().unwrap()).into(), + prepared_value_hash: unhex(i["prepared_value_hash"].as_str().unwrap()).into(), + signature: Default::default(), + }; + match hash_proto(&msg) { + Ok(h) => check(&mut failures, &name, &msg.encode_to_vec(), h, case), + Err(e) => failures.push(format!("{name}: hash_proto failed: {e}")), + } + } + assert_eq!( + suite["qbft_signing_root"].as_array().unwrap().len(), + 6, + "expected 6 qbft_signing_root cases" + ); + assert!(failures.is_empty(), "{}", failures.join("\n")); +} + +#[test] +fn any_string_hashing() { + // Charon's *priority* hashProto hashes the Any wrapper itself, type URL + // included, while pluto's hash_proto rejects Any outright (the consensus + // hasher must bind to inner message bytes). hash_proto_bytes over the + // encoded Any exercises the same merkleization path pluto's priority + // component uses via hash_any (crates/priority/src/calculate.rs). + // + // The vector's type URL decodes to "google.protobuf.Value" with the + // string carried in the oneof's string_value (field 3), not + // "google.protobuf.StringValue" field 1 — confirmed by decoding + // encoding_hex before writing this construction. + let suite = load_suite("qbft_hashing"); + let mut failures = Vec::new(); + for case in suite["any_string"].as_array().unwrap() { + let name = format!("qbft_hashing/any_string/{}", case["name"].as_str().unwrap()); + let s = case["input"]["string_value"].as_str().unwrap(); + let value = prost_types::Value { + kind: Some(prost_types::value::Kind::StringValue(s.to_string())), + }; + // prost_types::Value has no Name impl in this prost-types version, so + // Any::from_msg is unavailable; build the envelope directly. + let any = prost_types::Any { + type_url: "type.googleapis.com/google.protobuf.Value".into(), + value: value.encode_to_vec(), + }; + let encoded = any.encode_to_vec(); + match hash_proto_bytes(&encoded) { + Ok(h) => check(&mut failures, &name, &encoded, h, case), + Err(e) => failures.push(format!("{name}: hash_proto_bytes failed: {e}")), + } + } + assert_eq!( + suite["any_string"].as_array().unwrap().len(), + 4, + "expected 4 any_string cases" + ); + assert!(failures.is_empty(), "{}", failures.join("\n")); +} diff --git a/plans/pluto-conformance.md b/plans/pluto-conformance.md index 585cb8e..3a487dd 100644 --- a/plans/pluto-conformance.md +++ b/plans/pluto-conformance.md @@ -68,7 +68,7 @@ pluto commit actually tested. Statuses: `todo` / `in progress` / `done`. | # | Check | Suite | Status | Verdict | Notes | | --- | --- | --- | --- | --- | --- | | 1 | Harness scaffold + secp256k1 | `secp256k1_signatures` | done | PASS | Both cases pass: sign, recover, and verify_65 all match the vector via `pluto-k1util` (33-byte SEC1 pubkey). | -| 2 | Proto encoding + SSZ hashing | `qbft_hashing` | todo | — | | +| 2 | Proto encoding + SSZ hashing | `qbft_hashing` | done | FAIL (unsigned_data_set only) | `duty` (3/3), `qbft_signing_root` (6/6), `any_string` (4/4) all PASS. `unsigned_data_set` has 2/12 red: `empty_value` and `empty_key`. Real divergence, no ladder entry: prost's map encoding applies proto3 default-value omission *inside* map entries (skips an empty string key or empty bytes value), while charon's Go marshaler always writes both map-entry fields explicitly. See Findings. Pluto's own `crates/consensus/testdata/vectors/hashproto.json` never exercises an empty key/value entry, so this suite gives it strictly broader coverage — a candidate for pluto to adopt these vectors and retire its own file, not something this repo changes. | | 3 | BLS threshold aggregation | `bls_threshold` | todo | — | | | 4 | Cluster hashes + lock verification | `cluster_hashing` | todo | — | | | 5 | Priority scoring | `priority_scoring` | todo | — | | @@ -83,6 +83,24 @@ pluto commit actually tested. Statuses: `todo` / `in progress` / `done`. _Append findings here as tasks produce them: what diverged, file:line in pluto, whether a ladder entry covers it, and whether it was reported upstream._ +- **`UnsignedDataSet` map entries with an empty key or empty value hash differently + than charon (FAIL, task 2, `qbft_hashing/unsigned_data_set`).** prost's generated + `btree_map` encoding (`pluto_core::corepb::v1::UnsignedDataSet.set`, a + `BTreeMap`) skips a map entry's key field when it equals `""` and + skips its value field when it equals `b""` — proto3 default-value omission applied + to the two synthetic fields of each map entry + (`prost-0.14.4/src/encoding.rs:1044-1059`, `encode_with_default`). Charon's + `hashProto`, built on Go's `google.golang.org/protobuf`, always writes both map-entry + fields regardless of default-ness. Confirmed by two vectors: + `empty_value` (`set: {"0xaabb": ""}`) encodes to `0a0a0a063078616162621200` per + charon but `0a080a06307861616262` per pluto (missing the empty-value field 2); + `empty_key` (`set: {"": "01"}`) encodes to `0a050a00120101` per charon but + `0a03120101` per pluto (missing the empty-key field 1). No `charon_anchor.json` + ladder entry covers map-entry presence, so this is not `ABSENT-OK` — it is a real + interop risk: two clusters running pluto vs. charon nodes would compute different + QBFT value hashes for an `UnsignedDataSet` containing an empty pubkey string or an + empty per-DV data payload, breaking quorum. Not reported upstream to pluto yet. + ## Global Constraints - Pluto is **never modified**. The harness lives in `consumers/rust/` and path-depends From dee5f9543cc6f196c20a4c55256999f3ff221a22 Mon Sep 17 00:00:00 2001 From: Andrei Smirnov Date: Fri, 31 Jul 2026 20:51:50 +0300 Subject: [PATCH 05/27] Pin unsigned_data_set map-entry divergence as a named test Restructures unsigned_data_set_hashing to skip and separately track the two cases where prost's map encoding diverges from charon's (empty key/value map-entry presence), so the FAIL finding stays visible without leaving the whole suite permanently red. --- consumers/rust/tests/qbft_hashing.rs | 158 ++++++++++++++++++++++++--- plans/pluto-conformance.md | 23 +++- 2 files changed, 165 insertions(+), 16 deletions(-) diff --git a/consumers/rust/tests/qbft_hashing.rs b/consumers/rust/tests/qbft_hashing.rs index 397ffbf..7a401bf 100644 --- a/consumers/rust/tests/qbft_hashing.rs +++ b/consumers/rust/tests/qbft_hashing.rs @@ -57,26 +57,39 @@ fn duty_hashing() { assert!(failures.is_empty(), "{}", failures.join("\n")); } +/// Map keys are the literal strings from the vector (e.g. "0xaabb"), not +/// hex-decoded; only values are hex-encoded bytes. pluto's set is +/// `BTreeMap`, whose key ordering gives the deterministic +/// encoding regardless of the vector's insertion order. +fn build_unsigned_data_set(case: &serde_json::Value) -> UnsignedDataSet { + let set = case["input"]["set"] + .as_object() + .unwrap() + .iter() + .map(|(k, v)| (k.clone(), unhex(v.as_str().unwrap()).into())) + .collect(); + UnsignedDataSet { set } +} + +/// Cases excluded from the strict check below because pluto's map encoding is +/// known to diverge from charon's on them. See +/// `unsigned_data_set_known_divergence_empty_map_entry_fields` for the +/// recorded FAIL and why only these two are excluded. +const KNOWN_DIVERGENT_UNSIGNED_DATA_SET_CASES: &[&str] = &["empty_value", "empty_key"]; + #[test] fn unsigned_data_set_hashing() { let suite = load_suite("qbft_hashing"); let mut failures = Vec::new(); + let mut skipped = Vec::new(); for case in suite["unsigned_data_set"].as_array().unwrap() { - let name = format!( - "qbft_hashing/unsigned_data_set/{}", - case["name"].as_str().unwrap() - ); - // Map keys are the literal strings from the vector (e.g. "0xaabb"), - // not hex-decoded; only values are hex-encoded bytes. pluto's set is - // BTreeMap, whose key ordering gives the deterministic - // encoding regardless of the vector's insertion order. - let set = case["input"]["set"] - .as_object() - .unwrap() - .iter() - .map(|(k, v)| (k.clone(), unhex(v.as_str().unwrap()).into())) - .collect(); - let uds = UnsignedDataSet { set }; + let case_name = case["name"].as_str().unwrap(); + if KNOWN_DIVERGENT_UNSIGNED_DATA_SET_CASES.contains(&case_name) { + skipped.push(case_name.to_string()); + continue; + } + let name = format!("qbft_hashing/unsigned_data_set/{case_name}"); + let uds = build_unsigned_data_set(case); match hash_proto(&uds) { Ok(h) => check(&mut failures, &name, &uds.encode_to_vec(), h, case), Err(e) => failures.push(format!("{name}: hash_proto failed: {e}")), @@ -87,6 +100,121 @@ fn unsigned_data_set_hashing() { 12, "expected 12 unsigned_data_set cases" ); + // If a third case starts diverging, or one of the known two stops + // diverging, this must fail loudly rather than silently skip a new red + // case or silently leave a fixed one excluded from the strict check. + let mut skipped_sorted = skipped.clone(); + skipped_sorted.sort_unstable(); + let mut want_sorted: Vec = KNOWN_DIVERGENT_UNSIGNED_DATA_SET_CASES + .iter() + .map(|s| s.to_string()) + .collect(); + want_sorted.sort_unstable(); + assert_eq!( + skipped_sorted, want_sorted, + "the set of cases skipped as known-divergent changed; a case may have started or \ + stopped diverging — see unsigned_data_set_known_divergence_empty_map_entry_fields" + ); + assert!(failures.is_empty(), "{}", failures.join("\n")); +} + +/// Known FAIL, recorded in `plans/pluto-conformance.md` under Findings: pluto's +/// map encoding (prost's generated `btree_map` codec) applies proto3 +/// default-value omission *inside* a map entry, skipping the key field when it +/// equals `""` and the value field when it equals `b""` +/// (prost-0.14.4 `src/encoding.rs:1044-1059`, `encode_with_default`). Charon's +/// `hashProto`, built on Go's `google.golang.org/protobuf`, always writes both +/// map-entry fields, default-valued or not. Charon's encoding is correct per +/// spec; pluto's is wrong. This is deliberately not treated as ABSENT-OK: no +/// `charon_anchor.json` ladder entry documents a map-entry-presence gap, so +/// this is pluto being wrong, not pluto legitimately trailing a dated charon +/// release. +/// +/// This test pins pluto's *current* (wrong) output byte-for-byte, and +/// separately asserts that output still differs from the vector's expected +/// bytes. Consequence: if pluto ever fixes its map encoding to match charon's +/// explicit-presence behaviour, the "still differs from the vector" +/// assertion below fails — that failure is the signal to delete this test and +/// move `empty_value`/`empty_key` into `unsigned_data_set_hashing`'s strict +/// checks (and out of `KNOWN_DIVERGENT_UNSIGNED_DATA_SET_CASES` there). If +/// pluto's output instead changes to a *different* wrong value, the "pinned" +/// assertion below fails, surfacing that regression too rather than hiding it +/// behind an already-red case. +#[test] +fn unsigned_data_set_known_divergence_empty_map_entry_fields() { + let suite = load_suite("qbft_hashing"); + let mut failures = Vec::new(); + // (case name, pluto's current encoding, pluto's current hash), captured + // from a real run against the pluto commit under test. + let pinned: &[(&str, &str, &str)] = &[ + ( + "empty_value", + "0a080a06307861616262", + "0a080a0630786161626200000000000000000000000000000000000000000000", + ), + ( + "empty_key", + "0a03120101", + "0a03120101000000000000000000000000000000000000000000000000000000", + ), + ]; + let mut seen = Vec::new(); + for case in suite["unsigned_data_set"].as_array().unwrap() { + let case_name = case["name"].as_str().unwrap(); + let Some(&(_, pinned_enc, pinned_hash)) = + pinned.iter().find(|(name, _, _)| *name == case_name) + else { + continue; + }; + seen.push(case_name.to_string()); + let name = format!("qbft_hashing/unsigned_data_set/{case_name}"); + let uds = build_unsigned_data_set(case); + let encoded = uds.encode_to_vec(); + let want_enc = unhex(case["encoding_hex"].as_str().unwrap()); + let pinned_enc_bytes = unhex(pinned_enc); + if encoded != pinned_enc_bytes { + failures.push(format!( + "{name}: pluto's encoding moved from the pinned divergence {} to {} — \ + investigate before assuming this is the known fix", + hex::encode(&pinned_enc_bytes), + hex::encode(&encoded) + )); + } + if encoded == want_enc { + failures.push(format!( + "{name}: pluto's encoding now matches the vector — promote this case into \ + unsigned_data_set_hashing and remove it from \ + KNOWN_DIVERGENT_UNSIGNED_DATA_SET_CASES" + )); + } + match hash_proto(&uds) { + Ok(h) => { + let pinned_hash_bytes = unhex(pinned_hash); + let want_hash = unhex(case["hash_hex"].as_str().unwrap()); + if h.as_slice() != pinned_hash_bytes { + failures.push(format!( + "{name}: pluto's hash moved from the pinned divergence {} to {} — \ + investigate before assuming this is the known fix", + hex::encode(pinned_hash_bytes), + hex::encode(h) + )); + } + if h.as_slice() == want_hash { + failures.push(format!( + "{name}: pluto's hash now matches the vector — promote this case into \ + unsigned_data_set_hashing and remove it from \ + KNOWN_DIVERGENT_UNSIGNED_DATA_SET_CASES" + )); + } + } + Err(e) => failures.push(format!("{name}: hash_proto failed: {e}")), + } + } + assert_eq!( + seen.len(), + pinned.len(), + "expected to find both known-divergent cases in the vector file" + ); assert!(failures.is_empty(), "{}", failures.join("\n")); } diff --git a/plans/pluto-conformance.md b/plans/pluto-conformance.md index 3a487dd..9533956 100644 --- a/plans/pluto-conformance.md +++ b/plans/pluto-conformance.md @@ -68,7 +68,7 @@ pluto commit actually tested. Statuses: `todo` / `in progress` / `done`. | # | Check | Suite | Status | Verdict | Notes | | --- | --- | --- | --- | --- | --- | | 1 | Harness scaffold + secp256k1 | `secp256k1_signatures` | done | PASS | Both cases pass: sign, recover, and verify_65 all match the vector via `pluto-k1util` (33-byte SEC1 pubkey). | -| 2 | Proto encoding + SSZ hashing | `qbft_hashing` | done | FAIL (unsigned_data_set only) | `duty` (3/3), `qbft_signing_root` (6/6), `any_string` (4/4) all PASS. `unsigned_data_set` has 2/12 red: `empty_value` and `empty_key`. Real divergence, no ladder entry: prost's map encoding applies proto3 default-value omission *inside* map entries (skips an empty string key or empty bytes value), while charon's Go marshaler always writes both map-entry fields explicitly. See Findings. Pluto's own `crates/consensus/testdata/vectors/hashproto.json` never exercises an empty key/value entry, so this suite gives it strictly broader coverage — a candidate for pluto to adopt these vectors and retire its own file, not something this repo changes. | +| 2 | Proto encoding + SSZ hashing | `qbft_hashing` | done | FAIL (unsigned_data_set only) | `duty` (3/3), `qbft_signing_root` (6/6), `any_string` (4/4) all PASS. `unsigned_data_set` has 10/12 strict-PASS; `empty_value` and `empty_key` are a real divergence, no ladder entry: prost's map encoding applies proto3 default-value omission *inside* map entries (skips an empty string key or empty bytes value), while charon's Go marshaler always writes both map-entry fields explicitly. See Findings. Pinned as a named known-divergence test (`unsigned_data_set_known_divergence_empty_map_entry_fields`) rather than left permanently red, so a future regression in the other 10 cases can't hide behind it; that test fails loudly if pluto's output ever changes (fixed or otherwise). Pluto's own `crates/consensus/testdata/vectors/hashproto.json` never exercises an empty key/value entry, so this suite gives it strictly broader coverage — a candidate for pluto to adopt these vectors and retire its own file, not something this repo changes. | | 3 | BLS threshold aggregation | `bls_threshold` | todo | — | | | 4 | Cluster hashes + lock verification | `cluster_hashing` | todo | — | | | 5 | Priority scoring | `priority_scoring` | todo | — | | @@ -100,6 +100,27 @@ whether a ladder entry covers it, and whether it was reported upstream._ interop risk: two clusters running pluto vs. charon nodes would compute different QBFT value hashes for an `UnsignedDataSet` containing an empty pubkey string or an empty per-DV data payload, breaking quorum. Not reported upstream to pluto yet. + Byte-level diff confirmed isolated (no second divergence hiding underneath): for + `empty_value`, pluto's bytes are exactly charon's map-entry content with the + trailing `1200` (empty-value field, tag+len0) truncated, length byte adjusted to + match — nothing else differs. For `empty_key`, pluto's bytes are exactly charon's + map-entry content with the leading `0a00` (empty-key field, tag+len0) truncated — + again nothing else differs. Reachability in a live cluster (code inspection only, + not differential execution, so flagged as such): an empty *value* cannot arise from + honest duty execution on either implementation — `unmarshalUnsignedData` + (`~/charon/core/unsigneddata.go:666-704`) requires bytes decodable as a real + `AttestationData`/`VersionedProposal`/`AggregatedAttestation`/`SyncContribution`, + none of which SSZ/JSON-marshal to zero bytes, and `marshalUnsignedData`'s output + feeding `UnsignedDataSetToProto` (`~/charon/core/proto.go:226-236`) is never empty + for a real duty. An empty *key* similarly cannot arise from honest code — DV pubkeys + are validated to a fixed length by `PubKey.Bytes()`/`PubKeyFromBytes` + (`~/charon/core/types.go:274-298`, `len(k) != pkLen` check) before ever reaching + this map. Both edge cases are therefore reachable only via a malformed or malicious + peer constructing a wire-level `UnsignedDataSet` protobuf directly (bypassing the + normal duty-fetch/marshal path) — not something an honest node in a healthy cluster + ever produces. This lowers the practical severity versus a "happens in normal + operation" reading, but the divergence itself (pluto and charon hashing the same + adversarial/malformed bytes differently) is still real and still a FAIL. ## Global Constraints From ef4c82d63c5f490e8da4065a987180aafdf213cd Mon Sep 17 00:00:00 2001 From: Andrei Smirnov Date: Fri, 31 Jul 2026 21:02:53 +0300 Subject: [PATCH 06/27] Correct skip-set comment and drop line-anchored citations The skip-set assert_eq! only checks case names, not divergence status; the comment wrongly claimed it detects a case that stops diverging. Fixed the comment to attribute that detection to the sibling pinned test instead. Also drops line numbers from the charon/prost citations in the Findings entry, per the repo's no-line-anchor convention. --- consumers/rust/tests/qbft_hashing.rs | 10 +++++++--- plans/pluto-conformance.md | 8 ++++---- 2 files changed, 11 insertions(+), 7 deletions(-) diff --git a/consumers/rust/tests/qbft_hashing.rs b/consumers/rust/tests/qbft_hashing.rs index 7a401bf..e1ef550 100644 --- a/consumers/rust/tests/qbft_hashing.rs +++ b/consumers/rust/tests/qbft_hashing.rs @@ -100,9 +100,13 @@ fn unsigned_data_set_hashing() { 12, "expected 12 unsigned_data_set cases" ); - // If a third case starts diverging, or one of the known two stops - // diverging, this must fail loudly rather than silently skip a new red - // case or silently leave a fixed one excluded from the strict check. + // This checks only that the vector still contains cases named exactly + // these two strings, so a renamed or removed known-divergent case fails + // loudly instead of silently vanishing from the strict check below. It + // cannot detect whether the named cases still actually diverge — that is + // unsigned_data_set_known_divergence_empty_map_entry_fields's job: its + // "still differs from the vector" assertion goes red if pluto's output + // ever starts matching the spec. let mut skipped_sorted = skipped.clone(); skipped_sorted.sort_unstable(); let mut want_sorted: Vec = KNOWN_DIVERGENT_UNSIGNED_DATA_SET_CASES diff --git a/plans/pluto-conformance.md b/plans/pluto-conformance.md index 9533956..d3f0662 100644 --- a/plans/pluto-conformance.md +++ b/plans/pluto-conformance.md @@ -89,7 +89,7 @@ whether a ladder entry covers it, and whether it was reported upstream._ `BTreeMap`) skips a map entry's key field when it equals `""` and skips its value field when it equals `b""` — proto3 default-value omission applied to the two synthetic fields of each map entry - (`prost-0.14.4/src/encoding.rs:1044-1059`, `encode_with_default`). Charon's + (prost 0.14.4, `encoding::encode_with_default`). Charon's `hashProto`, built on Go's `google.golang.org/protobuf`, always writes both map-entry fields regardless of default-ness. Confirmed by two vectors: `empty_value` (`set: {"0xaabb": ""}`) encodes to `0a0a0a063078616162621200` per @@ -108,13 +108,13 @@ whether a ladder entry covers it, and whether it was reported upstream._ again nothing else differs. Reachability in a live cluster (code inspection only, not differential execution, so flagged as such): an empty *value* cannot arise from honest duty execution on either implementation — `unmarshalUnsignedData` - (`~/charon/core/unsigneddata.go:666-704`) requires bytes decodable as a real + (`core/unsigneddata.go`) requires bytes decodable as a real `AttestationData`/`VersionedProposal`/`AggregatedAttestation`/`SyncContribution`, none of which SSZ/JSON-marshal to zero bytes, and `marshalUnsignedData`'s output - feeding `UnsignedDataSetToProto` (`~/charon/core/proto.go:226-236`) is never empty + feeding `UnsignedDataSetToProto` (`core/proto.go`) is never empty for a real duty. An empty *key* similarly cannot arise from honest code — DV pubkeys are validated to a fixed length by `PubKey.Bytes()`/`PubKeyFromBytes` - (`~/charon/core/types.go:274-298`, `len(k) != pkLen` check) before ever reaching + (`core/types.go`, `len(k) != pkLen` check) before ever reaching this map. Both edge cases are therefore reachable only via a malformed or malicious peer constructing a wire-level `UnsignedDataSet` protobuf directly (bypassing the normal duty-fetch/marshal path) — not something an honest node in a healthy cluster From c0426167739611f3dcae7e1a7a20887b0a67c0b4 Mon Sep 17 00:00:00 2001 From: Andrei Smirnov Date: Fri, 31 Jul 2026 21:09:58 +0300 Subject: [PATCH 07/27] Run bls_threshold suite against pluto --- consumers/rust/Cargo.toml | 1 + consumers/rust/tests/bls_threshold.rs | 259 ++++++++++++++++++++++++++ plans/pluto-conformance.md | 2 +- 3 files changed, 261 insertions(+), 1 deletion(-) create mode 100644 consumers/rust/tests/bls_threshold.rs diff --git a/consumers/rust/Cargo.toml b/consumers/rust/Cargo.toml index 9ae5859..1156b70 100644 --- a/consumers/rust/Cargo.toml +++ b/consumers/rust/Cargo.toml @@ -13,6 +13,7 @@ hex = "0.4.3" pluto-k1util = { path = "../../../pluto/crates/k1util" } pluto-core = { path = "../../../pluto/crates/core" } pluto-consensus = { path = "../../../pluto/crates/consensus" } +pluto-crypto = { path = "../../../pluto/crates/crypto" } k256 = { version = "0.13.4", features = ["ecdsa", "sha256"] } prost = "0.14" prost-types = "0.14" diff --git a/consumers/rust/tests/bls_threshold.rs b/consumers/rust/tests/bls_threshold.rs new file mode 100644 index 0000000..f69a3a7 --- /dev/null +++ b/consumers/rust/tests/bls_threshold.rs @@ -0,0 +1,259 @@ +use std::collections::HashMap; + +use pluto_crypto::blst_impl::BlstImpl; +use pluto_crypto::tbls::Tbls; +use spec_vectors_pluto::{load_suite, unhex}; + +fn arr(v: &[u8]) -> [u8; N] { + v.try_into().expect("length") +} + +/// `share_idx` in the vector's `partials` group is 1-based, matching +/// `pluto_crypto`'s `Index` convention; no conversion is needed. +fn share_secrets(suite: &serde_json::Value) -> HashMap { + suite["partials"] + .as_array() + .unwrap() + .iter() + .map(|p| { + let idx = p["input"]["share_idx"].as_u64().unwrap(); + let secret: [u8; 32] = arr(&unhex(p["input"]["secret_hex"].as_str().unwrap())); + (idx, secret) + }) + .collect() +} + +fn share_partial_signatures(suite: &serde_json::Value) -> HashMap { + suite["partials"] + .as_array() + .unwrap() + .iter() + .map(|p| { + let idx = p["input"]["share_idx"].as_u64().unwrap(); + let sig: [u8; 96] = arr(&unhex(p["signature_hex"].as_str().unwrap())); + (idx, sig) + }) + .collect() +} + +#[test] +fn keys() { + let suite = load_suite("bls_threshold"); + let t = BlstImpl; + let mut failures = Vec::new(); + + for case in suite["keys"].as_array().unwrap() { + let name = case["name"].as_str().unwrap(); + let secret: [u8; 32] = arr(&unhex(case["input"]["secret_hex"].as_str().unwrap())); + let want_pub = unhex(case["pubkey_hex"].as_str().unwrap()); + match t.secret_to_public_key(&secret) { + Ok(pk) if pk.to_vec() == want_pub => {} + other => failures.push(format!( + "bls_threshold/keys/{name}: secret_to_public_key gave {other:?}, want {}", + hex::encode(&want_pub) + )), + } + } + assert_eq!( + suite["keys"].as_array().unwrap().len(), + 2, + "expected 2 keys cases" + ); + assert!(failures.is_empty(), "{}", failures.join("\n")); +} + +#[test] +fn partials() { + let suite = load_suite("bls_threshold"); + let t = BlstImpl; + let msg = unhex(suite["message_hex"].as_str().unwrap()); + let mut failures = Vec::new(); + + for case in suite["partials"].as_array().unwrap() { + let name = case["name"].as_str().unwrap(); + let secret: [u8; 32] = arr(&unhex(case["input"]["secret_hex"].as_str().unwrap())); + let want_pub = unhex(case["pubshare_hex"].as_str().unwrap()); + let want_sig = unhex(case["signature_hex"].as_str().unwrap()); + + match t.secret_to_public_key(&secret) { + Ok(pk) if pk.to_vec() == want_pub => {} + other => failures.push(format!( + "bls_threshold/partials/{name}: pubshare gave {other:?}, want {}", + hex::encode(&want_pub) + )), + } + match t.sign(&secret, &msg) { + Ok(sig) if sig.to_vec() == want_sig => {} + other => failures.push(format!( + "bls_threshold/partials/{name}: partial signature gave {other:?}, want {}", + hex::encode(&want_sig) + )), + } + } + assert_eq!( + suite["partials"].as_array().unwrap().len(), + 4, + "expected 4 partials cases" + ); + assert!(failures.is_empty(), "{}", failures.join("\n")); +} + +/// Each of the four distinct 3-of-4 quorums must reproduce the *same* group +/// signature: this is the point of threshold aggregation (any quorum +/// interpolates the same degree-2 polynomial at x=0), and only checking one +/// quorum would not distinguish correct Lagrange coefficients from a +/// coincidentally well-formed but wrong signature. +#[test] +fn threshold_aggregates() { + let suite = load_suite("bls_threshold"); + let t = BlstImpl; + let msg = unhex(suite["message_hex"].as_str().unwrap()); + let group_pub: [u8; 48] = arr(&unhex(suite["group_pubkey_hex"].as_str().unwrap())); + let group_sig: [u8; 96] = arr(&unhex(suite["group_signature_hex"].as_str().unwrap())); + let partials = share_partial_signatures(&suite); + let mut failures = Vec::new(); + + if t.verify(&group_pub, &msg, &group_sig).is_err() { + failures.push( + "bls_threshold/threshold_aggregates/group_signature_hex: does not verify under \ + group_pubkey_hex" + .to_string(), + ); + } + + let mut seen_quorums: Vec> = Vec::new(); + for case in suite["threshold_aggregates"].as_array().unwrap() { + let name = case["name"].as_str().unwrap(); + let want: [u8; 96] = arr(&unhex(case["signature_hex"].as_str().unwrap())); + let mut indices: Vec = case["input"]["share_indices"] + .as_array() + .unwrap() + .iter() + .map(|i| i.as_u64().unwrap()) + .collect(); + let subset: HashMap = indices.iter().map(|i| (*i, partials[i])).collect(); + + match t.threshold_aggregate(&subset) { + Ok(sig) if sig == want && sig == group_sig => {} + Ok(sig) if sig == want => failures.push(format!( + "bls_threshold/threshold_aggregates/{name}: matched the vector but not \ + group_signature_hex, got {}", + hex::encode(sig) + )), + other => failures.push(format!( + "bls_threshold/threshold_aggregates/{name}: gave {other:?}, want {}", + hex::encode(want) + )), + } + + indices.sort_unstable(); + seen_quorums.push(indices); + } + assert_eq!( + suite["threshold_aggregates"].as_array().unwrap().len(), + 4, + "expected 4 threshold_aggregates cases" + ); + let mut unique_quorums = seen_quorums.clone(); + unique_quorums.sort_unstable(); + unique_quorums.dedup(); + assert_eq!( + unique_quorums.len(), + seen_quorums.len(), + "expected 4 distinct quorums, got duplicates: {seen_quorums:?}" + ); + assert!(failures.is_empty(), "{}", failures.join("\n")); +} + +#[test] +fn recovery() { + let suite = load_suite("bls_threshold"); + let t = BlstImpl; + let group_pub: [u8; 48] = arr(&unhex(suite["group_pubkey_hex"].as_str().unwrap())); + let group_secret: [u8; 32] = arr(&unhex(suite["group_secret_hex"].as_str().unwrap())); + let secrets = share_secrets(&suite); + let mut failures = Vec::new(); + + for case in suite["recovery"].as_array().unwrap() { + let name = case["name"].as_str().unwrap(); + let want_pub = unhex(case["pubkey_hex"].as_str().unwrap()); + let subset: HashMap = case["input"]["share_indices"] + .as_array() + .unwrap() + .iter() + .map(|i| { + let i = i.as_u64().unwrap(); + (i, secrets[&i]) + }) + .collect(); + + match t.recover_secret(&subset) { + Ok(secret) if secret != group_secret => failures.push(format!( + "bls_threshold/recovery/{name}: recovered secret {}, want the group secret", + hex::encode(secret) + )), + Ok(secret) => match t.secret_to_public_key(&secret) { + Ok(pk) if pk.to_vec() == want_pub && pk == group_pub => {} + other => failures.push(format!( + "bls_threshold/recovery/{name}: recovered secret's public key gave \ + {other:?}, want {}", + hex::encode(&want_pub) + )), + }, + Err(e) => failures.push(format!( + "bls_threshold/recovery/{name}: recover_secret failed: {e}" + )), + } + } + assert_eq!( + suite["recovery"].as_array().unwrap().len(), + 1, + "expected 1 recovery case" + ); + assert!(failures.is_empty(), "{}", failures.join("\n")); +} + +/// Plain aggregation (used for the cluster lock hash) is not threshold +/// aggregation: it must reproduce the pinned bytes, and — the whole reason +/// threshold aggregation exists — must NOT verify under the group public key. +#[test] +fn plain_aggregate() { + let suite = load_suite("bls_threshold"); + let t = BlstImpl; + let msg = unhex(suite["message_hex"].as_str().unwrap()); + let group_pub: [u8; 48] = arr(&unhex(suite["group_pubkey_hex"].as_str().unwrap())); + let partials = share_partial_signatures(&suite); + let mut failures = Vec::new(); + + for case in suite["plain_aggregate"].as_array().unwrap() { + let name = case["name"].as_str().unwrap(); + let want: [u8; 96] = arr(&unhex(case["signature_hex"].as_str().unwrap())); + let sigs: Vec<[u8; 96]> = case["input"]["share_indices"] + .as_array() + .unwrap() + .iter() + .map(|i| partials[&i.as_u64().unwrap()]) + .collect(); + + match t.aggregate(&sigs) { + Ok(sig) if sig == want => { + if t.verify(&group_pub, &msg, &sig).is_ok() { + failures.push(format!( + "bls_threshold/plain_aggregate/{name}: verified under \ + group_pubkey_hex, must not" + )); + } + } + other => failures.push(format!( + "bls_threshold/plain_aggregate/{name}: gave {other:?}, want {}", + hex::encode(want) + )), + } + } + assert_eq!( + suite["plain_aggregate"].as_array().unwrap().len(), + 1, + "expected 1 plain_aggregate case" + ); + assert!(failures.is_empty(), "{}", failures.join("\n")); +} diff --git a/plans/pluto-conformance.md b/plans/pluto-conformance.md index d3f0662..97edc9c 100644 --- a/plans/pluto-conformance.md +++ b/plans/pluto-conformance.md @@ -69,7 +69,7 @@ pluto commit actually tested. Statuses: `todo` / `in progress` / `done`. | --- | --- | --- | --- | --- | --- | | 1 | Harness scaffold + secp256k1 | `secp256k1_signatures` | done | PASS | Both cases pass: sign, recover, and verify_65 all match the vector via `pluto-k1util` (33-byte SEC1 pubkey). | | 2 | Proto encoding + SSZ hashing | `qbft_hashing` | done | FAIL (unsigned_data_set only) | `duty` (3/3), `qbft_signing_root` (6/6), `any_string` (4/4) all PASS. `unsigned_data_set` has 10/12 strict-PASS; `empty_value` and `empty_key` are a real divergence, no ladder entry: prost's map encoding applies proto3 default-value omission *inside* map entries (skips an empty string key or empty bytes value), while charon's Go marshaler always writes both map-entry fields explicitly. See Findings. Pinned as a named known-divergence test (`unsigned_data_set_known_divergence_empty_map_entry_fields`) rather than left permanently red, so a future regression in the other 10 cases can't hide behind it; that test fails loudly if pluto's output ever changes (fixed or otherwise). Pluto's own `crates/consensus/testdata/vectors/hashproto.json` never exercises an empty key/value entry, so this suite gives it strictly broader coverage — a candidate for pluto to adopt these vectors and retire its own file, not something this repo changes. | -| 3 | BLS threshold aggregation | `bls_threshold` | todo | — | | +| 3 | BLS threshold aggregation | `bls_threshold` | done | PASS | All 5 groups PASS: `keys` (2/2), `partials` (4/4, pubshare + partial signature), `threshold_aggregates` (4/4 distinct 3-of-4 quorums, each reproducing the one `group_signature_hex`), `recovery` (1/1, `recover_secret` yields the exact `group_secret_hex`), `plain_aggregate` (1/1, matches the pinned bytes and does **not** verify under `group_pubkey_hex`, confirming plain aggregation is not a threshold-signature substitute). `group_signature_hex` also verifies under `group_pubkey_hex`. `pluto-crypto`'s `BlstImpl`/`Tbls` API matched the brief's guesses exactly; no divergence found. | | 4 | Cluster hashes + lock verification | `cluster_hashing` | todo | — | | | 5 | Priority scoring | `priority_scoring` | todo | — | | | 6 | Round timer deadlines | `timer_deadlines` | todo | — | | From 00da5cf8fbabff62c4d7b145af8c3e6d7ee22110 Mon Sep 17 00:00:00 2001 From: Andrei Smirnov Date: Fri, 31 Jul 2026 21:25:06 +0300 Subject: [PATCH 08/27] Run cluster_hashing suite against pluto --- consumers/rust/Cargo.toml | 3 + consumers/rust/tests/cluster_hashing.rs | 232 ++++++++++++++++++++++++ plans/pluto-conformance.md | 44 ++++- 3 files changed, 278 insertions(+), 1 deletion(-) create mode 100644 consumers/rust/tests/cluster_hashing.rs diff --git a/consumers/rust/Cargo.toml b/consumers/rust/Cargo.toml index 1156b70..b6b86e2 100644 --- a/consumers/rust/Cargo.toml +++ b/consumers/rust/Cargo.toml @@ -14,9 +14,12 @@ pluto-k1util = { path = "../../../pluto/crates/k1util" } pluto-core = { path = "../../../pluto/crates/core" } pluto-consensus = { path = "../../../pluto/crates/consensus" } pluto-crypto = { path = "../../../pluto/crates/crypto" } +pluto-cluster = { path = "../../../pluto/crates/cluster" } +pluto-eth1wrap = { path = "../../../pluto/crates/eth1wrap" } k256 = { version = "0.13.4", features = ["ecdsa", "sha256"] } prost = "0.14" prost-types = "0.14" +tokio = { version = "1", features = ["macros", "rt"] } [patch.crates-io] multistream-select = { path = "../../../pluto/third_party/multistream-select" } diff --git a/consumers/rust/tests/cluster_hashing.rs b/consumers/rust/tests/cluster_hashing.rs new file mode 100644 index 0000000..3fe3aff --- /dev/null +++ b/consumers/rust/tests/cluster_hashing.rs @@ -0,0 +1,232 @@ +use pluto_cluster::definition::Definition; +use pluto_cluster::lock::Lock; +use spec_vectors_pluto::load_suite; + +/// `all_empty_lists` is a real charon divergence, not a test bug: charon's +/// `Operators`/`ValidatorAddresses` fields lack `omitempty` +/// (`cluster/definition.go`), so Go's `encoding/json` marshals a nil slice as +/// `null`. Pluto's `DefinitionV1x10.operators`/`.validator_addresses` +/// (`crates/cluster/src/definition.rs`) have no `#[serde(default)]` and reject +/// `null` outright. See the cluster_hashing Results row / Findings entry. +const DEFINITION_KNOWN_DIVERGENCE: &str = "all_empty_lists"; + +/// `validator_without_deposit_data` is the lock-side counterpart: charon's +/// `PartialDepositData` field carries `omitempty` +/// (`cluster/distvalidator.go`), so Go omits the key entirely for a validator +/// with no partial deposits. Pluto's +/// `DistValidatorV1x8orLater.partial_deposit_data` +/// (`crates/cluster/src/distvalidator.rs`) has no `#[serde(default)]` and +/// requires the key present. +const LOCK_KNOWN_DIVERGENCE: &str = "validator_without_deposit_data"; + +#[test] +fn definition_hashes() { + let suite = load_suite("cluster_hashing"); + let cases = suite["definition"].as_array().unwrap(); + assert_eq!(cases.len(), 6, "unexpected number of definition cases"); + + let mut failures = Vec::new(); + for case in cases { + let case_name = case["name"].as_str().unwrap(); + if case_name == DEFINITION_KNOWN_DIVERGENCE { + continue; + } + let name = format!("cluster_hashing/definition/{case_name}"); + + let mut def: Definition = match serde_json::from_value(case["input"].clone()) { + Ok(d) => d, + Err(e) => { + failures.push(format!("{name}: parse: {e}")); + continue; + } + }; + + if let Err(e) = def.set_definition_hashes() { + failures.push(format!("{name}: set_definition_hashes: {e}")); + continue; + } + + let want_config = case["config_hash"].as_str().unwrap(); + let want_definition = case["definition_hash"].as_str().unwrap(); + + let got_config = hex::encode(&def.config_hash); + if got_config != want_config { + failures.push(format!( + "{name}: config_hash {got_config}, want {want_config}" + )); + } + + let got_definition = hex::encode(&def.definition_hash); + if got_definition != want_definition { + failures.push(format!( + "{name}: definition_hash {got_definition}, want {want_definition}" + )); + } + } + + assert!(failures.is_empty(), "{}", failures.join("\n")); +} + +/// Pins the `all_empty_lists` divergence: fails loudly (a regression to +/// investigate, not a silent fix) the moment pluto's `Definition` parser +/// starts accepting `operators: null` / `validators: null`, at which point +/// this case should move back into the strict `definition_hashes` loop. +#[test] +fn definition_known_divergence_null_operators_and_validators() { + let suite = load_suite("cluster_hashing"); + let case = suite["definition"] + .as_array() + .unwrap() + .iter() + .find(|c| c["name"] == DEFINITION_KNOWN_DIVERGENCE) + .expect("all_empty_lists present"); + + let err = serde_json::from_value::(case["input"].clone()) + .expect_err("expected pluto to still reject null operators/validators"); + assert!( + err.to_string().contains("null") || err.to_string().contains("sequence"), + "unexpected error shape, pluto's rejection reason may have changed: {err}" + ); +} + +/// The unsigned and signed single-operator cases share a config_hash (signing +/// must not move the config hash) while their definition_hash must differ +/// (signatures are part of the definition hash). +#[test] +fn signing_does_not_move_config_hash() { + let suite = load_suite("cluster_hashing"); + let cases = suite["definition"].as_array().unwrap(); + + let find = |name: &str| -> Definition { + let case = cases + .iter() + .find(|c| c["name"] == name) + .unwrap_or_else(|| panic!("{name} present")); + let mut def: Definition = serde_json::from_value(case["input"].clone()) + .unwrap_or_else(|e| panic!("parse {name}: {e}")); + def.set_definition_hashes() + .unwrap_or_else(|e| panic!("set_definition_hashes {name}: {e}")); + def + }; + + let unsigned = find("unsigned_single_operator"); + let signed = find("signed_single_operator"); + + assert_eq!( + unsigned.config_hash, signed.config_hash, + "config_hash must not move when signatures are added" + ); + assert_ne!( + unsigned.definition_hash, signed.definition_hash, + "definition_hash must move when signatures are added" + ); +} + +#[tokio::test] +async fn lock_hashes_and_verification() { + let suite = load_suite("cluster_hashing"); + let cases = suite["lock"].as_array().unwrap(); + assert_eq!(cases.len(), 4, "unexpected number of lock cases"); + + let mut failures = Vec::new(); + for case in cases { + let case_name = case["name"].as_str().unwrap(); + if case_name == LOCK_KNOWN_DIVERGENCE { + continue; + } + let name = format!("cluster_hashing/lock/{case_name}"); + + let mut lock: Lock = match serde_json::from_value(case["input"].clone()) { + Ok(l) => l, + Err(e) => { + failures.push(format!("{name}: parse: {e}")); + continue; + } + }; + + if let Err(e) = lock.set_lock_hash() { + failures.push(format!("{name}: set_lock_hash: {e}")); + continue; + } + + let want = case["lock_hash"].as_str().unwrap(); + let got = hex::encode(&lock.lock_hash); + if got != want { + failures.push(format!("{name}: lock_hash {got}, want {want}")); + } + + if let Err(e) = lock.verify_hashes() { + failures.push(format!("{name}: verify_hashes: {e}")); + } + } + + assert!(failures.is_empty(), "{}", failures.join("\n")); +} + +/// Pins the `validator_without_deposit_data` divergence: fails loudly the +/// moment pluto's `Lock` parser starts accepting an absent +/// `partial_deposit_data` key, at which point this case should move back +/// into the strict `lock_hashes_and_verification` loop. +#[test] +fn lock_known_divergence_missing_partial_deposit_data() { + let suite = load_suite("cluster_hashing"); + let case = suite["lock"] + .as_array() + .unwrap() + .iter() + .find(|c| c["name"] == LOCK_KNOWN_DIVERGENCE) + .expect("validator_without_deposit_data present"); + + let err = serde_json::from_value::(case["input"].clone()) + .expect_err("expected pluto to still reject an absent partial_deposit_data key"); + assert!( + err.to_string().contains("partial_deposit_data"), + "unexpected error shape, pluto's rejection reason may have changed: {err}" + ); +} + +/// `real_keys_3_of_4` is the only lock case with real signatures: its +/// `signature_aggregate` is a plain BLS aggregate of the lock hash over all +/// four private shares, and its `node_signatures` are secp256k1 signatures of +/// the lock hash by each operator's real ENR key. Its operator and creator +/// EIP-712 signatures are placeholder bytes (charon's signing helpers for +/// those are not exported for vector generation), so `Lock::verify_signatures` +/// — which checks the definition's EIP-712 signatures first and returns on +/// the first error — always rejects this case at that stage. `verify_hashes` +/// covers the hash chain (lock hash, and the definition hash it embeds); the +/// BLS aggregate and node-signature checks are private methods invoked only +/// after the EIP-712 stage, so they cannot be exercised standalone from this +/// external crate. See the Results row / Findings entry for `cluster_hashing` +/// for the reachability discussion. +#[tokio::test] +async fn real_keys_lock_verifies_end_to_end() { + let suite = load_suite("cluster_hashing"); + let case = suite["lock"] + .as_array() + .unwrap() + .iter() + .find(|c| c["name"] == "real_keys_3_of_4") + .expect("real_keys_3_of_4 present"); + + let lock: Lock = serde_json::from_value(case["input"].clone()).expect("parse real_keys_3_of_4"); + + lock.verify_hashes().expect("hashes"); + + // A no-op EL client: it skips only contract-based (EIP-1271) signature + // verification, which this case does not use. + let eth1 = pluto_eth1wrap::EthClient::new("") + .await + .expect("noop eth client"); + + let err = lock + .verify_signatures(ð1) + .await + .expect_err("placeholder operator/creator EIP-712 signatures must not verify"); + assert!( + matches!( + err, + pluto_cluster::lock::LockError::DefinitionSignaturesVerificationFailed(_) + ), + "expected rejection at the definition EIP-712 signature stage, got {err:?}" + ); +} diff --git a/plans/pluto-conformance.md b/plans/pluto-conformance.md index 97edc9c..9a0d471 100644 --- a/plans/pluto-conformance.md +++ b/plans/pluto-conformance.md @@ -70,7 +70,7 @@ pluto commit actually tested. Statuses: `todo` / `in progress` / `done`. | 1 | Harness scaffold + secp256k1 | `secp256k1_signatures` | done | PASS | Both cases pass: sign, recover, and verify_65 all match the vector via `pluto-k1util` (33-byte SEC1 pubkey). | | 2 | Proto encoding + SSZ hashing | `qbft_hashing` | done | FAIL (unsigned_data_set only) | `duty` (3/3), `qbft_signing_root` (6/6), `any_string` (4/4) all PASS. `unsigned_data_set` has 10/12 strict-PASS; `empty_value` and `empty_key` are a real divergence, no ladder entry: prost's map encoding applies proto3 default-value omission *inside* map entries (skips an empty string key or empty bytes value), while charon's Go marshaler always writes both map-entry fields explicitly. See Findings. Pinned as a named known-divergence test (`unsigned_data_set_known_divergence_empty_map_entry_fields`) rather than left permanently red, so a future regression in the other 10 cases can't hide behind it; that test fails loudly if pluto's output ever changes (fixed or otherwise). Pluto's own `crates/consensus/testdata/vectors/hashproto.json` never exercises an empty key/value entry, so this suite gives it strictly broader coverage — a candidate for pluto to adopt these vectors and retire its own file, not something this repo changes. | | 3 | BLS threshold aggregation | `bls_threshold` | done | PASS | All 5 groups PASS: `keys` (2/2), `partials` (4/4, pubshare + partial signature), `threshold_aggregates` (4/4 distinct 3-of-4 quorums, each reproducing the one `group_signature_hex`), `recovery` (1/1, `recover_secret` yields the exact `group_secret_hex`), `plain_aggregate` (1/1, matches the pinned bytes and does **not** verify under `group_pubkey_hex`, confirming plain aggregation is not a threshold-signature substitute). `group_signature_hex` also verifies under `group_pubkey_hex`. `pluto-crypto`'s `BlstImpl`/`Tbls` API matched the brief's guesses exactly; no divergence found. | -| 4 | Cluster hashes + lock verification | `cluster_hashing` | todo | — | | +| 4 | Cluster hashes + lock verification | `cluster_hashing` | done | FAIL (2 known-divergence cases pinned) | `definition` 5/6 strict-PASS (`config_hash` and `definition_hash` both match on 5 cases, including the unsigned/signed pair proving config_hash is signature-independent); `all_empty_lists` is a real divergence, pinned (see Findings). `lock` 3/4 strict-PASS (`lock_hash` matches and `verify_hashes` succeeds on 3); `validator_without_deposit_data` is a second real divergence, pinned (see Findings). `real_keys_3_of_4`: hashes verify; the full `verify_signatures(&EthClient::new(""))` chain is asserted to fail specifically at the definition EIP-712 stage (`LockError::DefinitionSignaturesVerificationFailed`) because its operator/creator signatures are unavoidable placeholders (charon's EIP-712 signing helpers aren't exported for vector generation) — `Lock::verify_signatures` short-circuits there, so its private BLS-aggregate and node-signature checks are never reached from this external crate; that half is a coverage gap (not a failure), noted in the test's doc comment rather than mocked around. | | 5 | Priority scoring | `priority_scoring` | todo | — | | | 6 | Round timer deadlines | `timer_deadlines` | todo | — | | | 7 | QBFT message limits | `qbft_msg_limits` | todo | — | | @@ -122,6 +122,48 @@ whether a ladder entry covers it, and whether it was reported upstream._ operation" reading, but the divergence itself (pluto and charon hashing the same adversarial/malformed bytes differently) is still real and still a FAIL. +- **Pluto's `Definition`/`Lock` parsers reject the exact `null`/absent-key JSON + shapes charon's own Go marshaler produces for empty lists (FAIL, task 4, + `cluster_hashing/definition/all_empty_lists` and + `cluster_hashing/lock/validator_without_deposit_data`).** Two independent + instances of the same root cause: pluto's serde structs are missing + `#[serde(default)]` on `Vec` fields that charon's Go structs mark differently. + (1) `cluster/definition.go`'s `Operators []Operator` and + `ValidatorAddresses []ValidatorAddresses` fields have `json:"operators"` / + `json:"validators"` with **no** `omitempty`, so Go's `encoding/json` marshals a + nil slice as `"operators": null`. Pluto's `DefinitionV1x10.operators: Vec` + and `.validator_addresses: Vec` + (`pluto/crates/cluster/src/definition.rs`) carry no `#[serde(default)]`, so + `serde_json` rejects the key's `null` value with "invalid type: null, expected a + sequence" instead of treating it as empty. (Pluto's own `deposit_amounts` field + does handle `null` correctly, via a custom `DepositAmountsSerde` that explicitly + wraps `DefaultOnNull` — the same treatment was not extended to `operators` or + `validator_addresses`.) (2) `cluster/distvalidator.go`'s + `PartialDepositData []DepositData` field is tagged + `json:"partial_deposit_data,omitempty"`, so Go omits the key entirely when a + validator has no partial deposits. Pluto's + `DistValidatorV1x8orLater.partial_deposit_data: Vec` + (`pluto/crates/cluster/src/distvalidator.rs`) also carries no + `#[serde(default)]`, so `serde_json` rejects the file with "missing field + `partial_deposit_data`" rather than defaulting to an empty vec. No + `charon_anchor.json` ladder entry covers either gap — the ladder tracks + *protocol behaviour* that postdates charon v1.7.1, and both of these field tags + are unchanged, long-standing charon serialization behaviour, not a recent + charon feature. Reachability: a definition with zero operators is degenerate + (no real cluster has none), but a validator with zero partial deposits is a + realistic shape — e.g. a validator whose deposit was already broadcast through + another channel, or an intermediate DKG artifact captured before partial + deposit data is attached. Either way, a real charon-produced JSON file in that + shape is rejected outright by pluto's loader, which is a genuine interop risk + for any tool (this repo's own vector generator included) that hands pluto a + charon artifact verbatim rather than a pluto-shaped round-trip. Pinned as two + named known-divergence tests + (`definition_known_divergence_null_operators_and_validators`, + `lock_known_divergence_missing_partial_deposit_data`) rather than left + permanently red, each asserting the current rejection and its error shape, so + either flips loudly the moment pluto's parser is fixed. Not reported upstream + to pluto yet. + ## Global Constraints - Pluto is **never modified**. The harness lives in `consumers/rust/` and path-depends From f670b8955a695655d4120f6515465aa2e63c957e Mon Sep 17 00:00:00 2001 From: Andrei Smirnov Date: Fri, 31 Jul 2026 21:40:26 +0300 Subject: [PATCH 09/27] Document masked timestamp gap in cluster_hashing findings --- consumers/rust/tests/cluster_hashing.rs | 40 ++++++++++++----- plans/pluto-conformance.md | 58 +++++++++++++++++-------- 2 files changed, 69 insertions(+), 29 deletions(-) diff --git a/consumers/rust/tests/cluster_hashing.rs b/consumers/rust/tests/cluster_hashing.rs index 3fe3aff..7366704 100644 --- a/consumers/rust/tests/cluster_hashing.rs +++ b/consumers/rust/tests/cluster_hashing.rs @@ -2,12 +2,29 @@ use pluto_cluster::definition::Definition; use pluto_cluster::lock::Lock; use spec_vectors_pluto::load_suite; -/// `all_empty_lists` is a real charon divergence, not a test bug: charon's -/// `Operators`/`ValidatorAddresses` fields lack `omitempty` -/// (`cluster/definition.go`), so Go's `encoding/json` marshals a nil slice as -/// `null`. Pluto's `DefinitionV1x10.operators`/`.validator_addresses` -/// (`crates/cluster/src/definition.rs`) have no `#[serde(default)]` and reject -/// `null` outright. See the cluster_hashing Results row / Findings entry. +/// `all_empty_lists` is a real charon divergence, not a test bug — and it has +/// **at least two independent parse blockers stacked**, so fixing only the +/// first will not make this case pass: +/// +/// 1. Charon's `Operators`/`ValidatorAddresses` fields lack `omitempty` +/// (`cluster/definition.go`), so Go's `encoding/json` marshals a nil slice +/// as `null`. Pluto's `DefinitionV1x10.operators`/`.validator_addresses` +/// (`crates/cluster/src/definition.rs`) have no `#[serde(default)]` and +/// reject `null` outright. This is the error this test currently observes. +/// 2. Masked behind (1): charon's `Timestamp` field *does* carry `omitempty` +/// (`definitionJSONv1x10to11`, `cluster/definition.go`), so an empty +/// timestamp is legitimately absent from the JSON. Pluto's +/// `DefinitionV1x10.timestamp` also has no `#[serde(default)]`. Confirmed +/// by mutation: relaxing `operators`/`validators` from `null` to `[]` (as +/// if (1) were fixed) still fails to parse, now on "missing field +/// `timestamp`". +/// +/// Whoever sees this test go red after a pluto change should expect to find +/// the *next* blocker, not assume the case now passes end to end — only after +/// (1) and (2) both resolve does `all_empty_lists` belong back in the strict +/// `definition_hashes` loop. See the cluster_hashing Results row / Findings +/// entry for the full field list (a lower bound, not a total — later fields +/// are never reached while an earlier one blocks parsing). const DEFINITION_KNOWN_DIVERGENCE: &str = "all_empty_lists"; /// `validator_without_deposit_data` is the lock-side counterpart: charon's @@ -67,10 +84,13 @@ fn definition_hashes() { assert!(failures.is_empty(), "{}", failures.join("\n")); } -/// Pins the `all_empty_lists` divergence: fails loudly (a regression to -/// investigate, not a silent fix) the moment pluto's `Definition` parser -/// starts accepting `operators: null` / `validators: null`, at which point -/// this case should move back into the strict `definition_hashes` loop. +/// Pins the *first* of `all_empty_lists`'s at-least-two parse blockers (see +/// the constant's doc comment): fails loudly (a regression to investigate, +/// not a silent fix) the moment pluto's `Definition` parser starts accepting +/// `operators: null` / `validators: null`. That alone does not make the case +/// parse — the masked `timestamp` blocker still applies — so a green here +/// means "go find the next blocker", not "move this case into the strict +/// `definition_hashes` loop". #[test] fn definition_known_divergence_null_operators_and_validators() { let suite = load_suite("cluster_hashing"); diff --git a/plans/pluto-conformance.md b/plans/pluto-conformance.md index 9a0d471..9f8ce41 100644 --- a/plans/pluto-conformance.md +++ b/plans/pluto-conformance.md @@ -70,7 +70,7 @@ pluto commit actually tested. Statuses: `todo` / `in progress` / `done`. | 1 | Harness scaffold + secp256k1 | `secp256k1_signatures` | done | PASS | Both cases pass: sign, recover, and verify_65 all match the vector via `pluto-k1util` (33-byte SEC1 pubkey). | | 2 | Proto encoding + SSZ hashing | `qbft_hashing` | done | FAIL (unsigned_data_set only) | `duty` (3/3), `qbft_signing_root` (6/6), `any_string` (4/4) all PASS. `unsigned_data_set` has 10/12 strict-PASS; `empty_value` and `empty_key` are a real divergence, no ladder entry: prost's map encoding applies proto3 default-value omission *inside* map entries (skips an empty string key or empty bytes value), while charon's Go marshaler always writes both map-entry fields explicitly. See Findings. Pinned as a named known-divergence test (`unsigned_data_set_known_divergence_empty_map_entry_fields`) rather than left permanently red, so a future regression in the other 10 cases can't hide behind it; that test fails loudly if pluto's output ever changes (fixed or otherwise). Pluto's own `crates/consensus/testdata/vectors/hashproto.json` never exercises an empty key/value entry, so this suite gives it strictly broader coverage — a candidate for pluto to adopt these vectors and retire its own file, not something this repo changes. | | 3 | BLS threshold aggregation | `bls_threshold` | done | PASS | All 5 groups PASS: `keys` (2/2), `partials` (4/4, pubshare + partial signature), `threshold_aggregates` (4/4 distinct 3-of-4 quorums, each reproducing the one `group_signature_hex`), `recovery` (1/1, `recover_secret` yields the exact `group_secret_hex`), `plain_aggregate` (1/1, matches the pinned bytes and does **not** verify under `group_pubkey_hex`, confirming plain aggregation is not a threshold-signature substitute). `group_signature_hex` also verifies under `group_pubkey_hex`. `pluto-crypto`'s `BlstImpl`/`Tbls` API matched the brief's guesses exactly; no divergence found. | -| 4 | Cluster hashes + lock verification | `cluster_hashing` | done | FAIL (2 known-divergence cases pinned) | `definition` 5/6 strict-PASS (`config_hash` and `definition_hash` both match on 5 cases, including the unsigned/signed pair proving config_hash is signature-independent); `all_empty_lists` is a real divergence, pinned (see Findings). `lock` 3/4 strict-PASS (`lock_hash` matches and `verify_hashes` succeeds on 3); `validator_without_deposit_data` is a second real divergence, pinned (see Findings). `real_keys_3_of_4`: hashes verify; the full `verify_signatures(&EthClient::new(""))` chain is asserted to fail specifically at the definition EIP-712 stage (`LockError::DefinitionSignaturesVerificationFailed`) because its operator/creator signatures are unavoidable placeholders (charon's EIP-712 signing helpers aren't exported for vector generation) — `Lock::verify_signatures` short-circuits there, so its private BLS-aggregate and node-signature checks are never reached from this external crate; that half is a coverage gap (not a failure), noted in the test's doc comment rather than mocked around. | +| 4 | Cluster hashes + lock verification | `cluster_hashing` | done | FAIL (2 pinned known-divergence cases; at least 3 distinct affected fields — a lower bound, not a total) | `definition` 5/6 strict-PASS (`config_hash` and `definition_hash` both match on 5 cases, including the unsigned/signed pair proving config_hash is signature-independent); `all_empty_lists` is a real divergence, pinned (see Findings) — it carries **at least two independent parse blockers** (`operators`/`validators` null-rejection, then a masked `timestamp` missing-field rejection once the first is hypothetically fixed), so a partial upstream fix will not make it pass. `lock` 3/4 strict-PASS (`lock_hash` matches and `verify_hashes` succeeds on 3); `validator_without_deposit_data` is a second real divergence, pinned (see Findings). `real_keys_3_of_4`: hashes verify; the full `verify_signatures(&EthClient::new(""))` chain is asserted to fail specifically at the definition EIP-712 stage (`LockError::DefinitionSignaturesVerificationFailed`) because its operator/creator signatures are unavoidable placeholders (charon's EIP-712 signing helpers aren't exported for vector generation) — `Lock::verify_signatures` short-circuits there, so its private BLS-aggregate and node-signature checks are never reached from this external crate; that half is a coverage gap (not a failure), noted in the test's doc comment rather than mocked around. | | 5 | Priority scoring | `priority_scoring` | todo | — | | | 6 | Round timer deadlines | `timer_deadlines` | todo | — | | | 7 | QBFT message limits | `qbft_msg_limits` | todo | — | | @@ -122,12 +122,14 @@ whether a ladder entry covers it, and whether it was reported upstream._ operation" reading, but the divergence itself (pluto and charon hashing the same adversarial/malformed bytes differently) is still real and still a FAIL. -- **Pluto's `Definition`/`Lock` parsers reject the exact `null`/absent-key JSON - shapes charon's own Go marshaler produces for empty lists (FAIL, task 4, - `cluster_hashing/definition/all_empty_lists` and - `cluster_hashing/lock/validator_without_deposit_data`).** Two independent - instances of the same root cause: pluto's serde structs are missing - `#[serde(default)]` on `Vec` fields that charon's Go structs mark differently. +- **Pluto's v1.10 `Definition`/`Lock` structs are missing `#[serde(default)]` on + multiple fields charon's Go marshaler may legitimately omit or null out (FAIL, + task 4, `cluster_hashing/definition/all_empty_lists` and + `cluster_hashing/lock/validator_without_deposit_data`).** One root cause — + a missing serde default — recurring across at least three fields; **the true + count of affected fields is not established**, because the first parse error + in each case short-circuits `serde_json` before any later field is checked, so + this list is a lower bound, not a total. Confirmed instances: (1) `cluster/definition.go`'s `Operators []Operator` and `ValidatorAddresses []ValidatorAddresses` fields have `json:"operators"` / `json:"validators"` with **no** `omitempty`, so Go's `encoding/json` marshals a @@ -145,23 +147,41 @@ whether a ladder entry covers it, and whether it was reported upstream._ `DistValidatorV1x8orLater.partial_deposit_data: Vec` (`pluto/crates/cluster/src/distvalidator.rs`) also carries no `#[serde(default)]`, so `serde_json` rejects the file with "missing field - `partial_deposit_data`" rather than defaulting to an empty vec. No - `charon_anchor.json` ladder entry covers either gap — the ladder tracks - *protocol behaviour* that postdates charon v1.7.1, and both of these field tags + `partial_deposit_data`" rather than defaulting to an empty vec. (3) **Masked + behind (1)** in `all_empty_lists`: charon's `definitionJSONv1x10to11` struct + (`cluster/definition.go`) tags `Timestamp` with `omitempty` (as it also does + `Name`), so a definition with an empty timestamp legitimately omits the key. + Pluto's `DefinitionV1x10.name: String` does carry `#[serde(default)]` and + handles this correctly, but `DefinitionV1x10.timestamp: String` + (`pluto/crates/cluster/src/definition.rs`) does not, so once (1) is + hypothetically fixed the same case still fails with "missing field + `timestamp`". Confirmed directly: mutating `all_empty_lists`'s `operators`/ + `validators` from `null` to `[]` (simulating a fix to (1)) still fails to + parse on the missing `timestamp` key; adding an empty `timestamp` string on + top of that mutation parses successfully, with no further blocker found in + that specific probe (not a proof no more exist elsewhere in the struct). No + `charon_anchor.json` ladder entry covers any of this — the ladder tracks + *protocol behaviour* that postdates charon v1.7.1, and all of these field tags are unchanged, long-standing charon serialization behaviour, not a recent charon feature. Reachability: a definition with zero operators is degenerate - (no real cluster has none), but a validator with zero partial deposits is a - realistic shape — e.g. a validator whose deposit was already broadcast through - another channel, or an intermediate DKG artifact captured before partial - deposit data is attached. Either way, a real charon-produced JSON file in that - shape is rejected outright by pluto's loader, which is a genuine interop risk - for any tool (this repo's own vector generator included) that hands pluto a - charon artifact verbatim rather than a pluto-shaped round-trip. Pinned as two - named known-divergence tests + (no real cluster has none), but a validator with zero partial deposits, or a + definition with an empty timestamp, are realistic shapes — e.g. a validator + whose deposit was already broadcast through another channel, an intermediate + DKG artifact captured before partial deposit data is attached, or a definition + built without a timestamp set. Either way, a real charon-produced JSON file in + any of these shapes is rejected outright by pluto's loader, which is a genuine + interop risk for any tool (this repo's own vector generator included) that + hands pluto a charon artifact verbatim rather than a pluto-shaped round-trip. + Pinned as two named known-divergence tests (`definition_known_divergence_null_operators_and_validators`, `lock_known_divergence_missing_partial_deposit_data`) rather than left permanently red, each asserting the current rejection and its error shape, so - either flips loudly the moment pluto's parser is fixed. Not reported upstream + either flips loudly the moment pluto's parser changes — the `definition` test's + doc comment now states explicitly that `all_empty_lists` has at least two + independent parse blockers stacked, so a partial upstream fix (null handling + alone) will not turn it green; the masked `timestamp` gap is not given its own + pinned test, since it is unreachable while the first blocker stands and a test + that cannot run is worse than a documented gap. Not reported upstream to pluto yet. ## Global Constraints From 6e0e6bd725aa8c154d77df38136c29f74283e89e Mon Sep 17 00:00:00 2001 From: Andrei Smirnov Date: Fri, 31 Jul 2026 21:54:38 +0300 Subject: [PATCH 10/27] Run priority_scoring suite against pluto --- consumers/rust/Cargo.toml | 23 +- consumers/rust/tests/priority_scoring.rs | 439 +++++++++++++++++++++++ plans/pluto-conformance.md | 2 +- 3 files changed, 462 insertions(+), 2 deletions(-) create mode 100644 consumers/rust/tests/priority_scoring.rs diff --git a/consumers/rust/Cargo.toml b/consumers/rust/Cargo.toml index b6b86e2..719baf1 100644 --- a/consumers/rust/Cargo.toml +++ b/consumers/rust/Cargo.toml @@ -16,10 +16,31 @@ pluto-consensus = { path = "../../../pluto/crates/consensus" } pluto-crypto = { path = "../../../pluto/crates/crypto" } pluto-cluster = { path = "../../../pluto/crates/cluster" } pluto-eth1wrap = { path = "../../../pluto/crates/eth1wrap" } +pluto-priority = { path = "../../../pluto/crates/priority" } +pluto-p2p = { path = "../../../pluto/crates/p2p" } k256 = { version = "0.13.4", features = ["ecdsa", "sha256"] } prost = "0.14" prost-types = "0.14" -tokio = { version = "1", features = ["macros", "rt"] } +tokio = { version = "1", features = ["macros", "rt", "rt-multi-thread", "time", "sync"] } +tokio-util = "0.7.11" +futures = "0.3" +async-trait = "0.1.89" +chrono = "0.4" +libp2p = { version = "0.56", features = [ + "autonat", + "dns", + "identify", + "macros", + "mdns", + "noise", + "ping", + "quic", + "relay", + "secp256k1", + "tcp", + "tokio", + "yamux", +] } [patch.crates-io] multistream-select = { path = "../../../pluto/third_party/multistream-select" } diff --git a/consumers/rust/tests/priority_scoring.rs b/consumers/rust/tests/priority_scoring.rs new file mode 100644 index 0000000..b836670 --- /dev/null +++ b/consumers/rust/tests/priority_scoring.rs @@ -0,0 +1,439 @@ +//! Conformance suite: `priority_scoring` against pluto's `Prioritiser`. +//! +//! `calculate_result` (`pluto_priority::calculate`) is `pub(crate)`, so this +//! drives it end-to-end through the public `Prioritiser` API over an +//! in-process libp2p network, with a mock `Consensus` that captures the +//! proposed `PriorityResult` (mirrors +//! `pluto/crates/priority/tests/prioritiser_test.rs`, which is itself an +//! external-style test using only public API). + +use std::{ + collections::{HashMap, HashSet}, + sync::{ + Arc, Mutex, + atomic::{AtomicU64, Ordering}, + }, + time::Duration, +}; + +use async_trait::async_trait; +use futures::{FutureExt as _, StreamExt as _, future::select_all}; +use k256::{SecretKey, elliptic_curve::rand_core::OsRng}; +use libp2p::{ + Multiaddr, PeerId, Swarm, + core::{Transport as _, transport::MemoryTransport, upgrade::Version}, + multiaddr::Protocol, + swarm::SwarmEvent, +}; +use pluto_core::{ + corepb::v1::{ + core::Duty as ProtoDuty, + priority::{PriorityMsg, PriorityResult, PriorityTopicProposal}, + }, + deadline::{DeadlineCalculator, DeadlinerHandle, DeadlinerTask}, + types::Duty, +}; +use pluto_p2p::{p2p_context::P2PContext, peer::peer_id_from_key, utils::keypair_from_secret_key}; +use pluto_priority::{ + Consensus, ConsensusError, Prioritiser, PrioritySubscriber, + component::{TopicProposal, TopicResult, sign_msg}, + p2p::Behaviour, +}; +use spec_vectors_pluto::load_suite; +use tokio::{sync::oneshot, time::timeout}; +use tokio_util::sync::CancellationToken; + +/// Calculator that schedules every duty one hour out, so `DeadlinerHandle::add` +/// returns `Scheduled` rather than dropping the instance as expired. +struct FutureCalculator; + +impl DeadlineCalculator for FutureCalculator { + fn deadline( + &self, + _duty: &Duty, + ) -> pluto_core::deadline::Result>> { + Ok(Some( + chrono::Utc::now() + .checked_add_signed(chrono::Duration::hours(1)) + .expect("deadline in range"), + )) + } +} + +/// Mock consensus that decides on the first proposal per duty by invoking its +/// subscribers, and asserts every subsequent proposal for that duty is +/// identical. Verbatim shape from `prioritiser_test.rs`'s `TestConsensus`. +#[derive(Default)] +struct MockConsensus { + subs: Mutex>, + proposed: Mutex>, +} + +#[async_trait] +impl Consensus for MockConsensus { + async fn propose_priority( + &self, + duty: Duty, + result: PriorityResult, + _ct: &CancellationToken, + ) -> Result<(), ConsensusError> { + let slot = duty.slot.inner(); + { + let proposed = self.proposed.lock().expect("proposed mutex"); + if let Some(prev) = proposed.get(&slot) { + assert_eq!( + prev.topics, result.topics, + "all proposals for a duty must be identical" + ); + return Ok(()); + } + } + let subs = self.subs.lock().expect("subs mutex"); + for sub in subs.iter() { + sub(duty.clone(), result.clone())?; + } + drop(subs); + self.proposed + .lock() + .expect("proposed mutex") + .insert(slot, result); + Ok(()) + } + + fn subscribe_priority(&self, callback: PrioritySubscriber) { + self.subs.lock().expect("subs mutex").push(callback); + } +} + +/// A built in-process host: its swarm, prioritiser, and listen address. +struct Host { + swarm: Swarm, + prioritiser: Prioritiser, + addr: Multiaddr, +} + +/// Global counter for in-process memory-transport addresses. +/// +/// Addresses must never repeat across cases within one test run: aborting a +/// prior case's swarm driver doesn't synchronously release its listener from +/// `MemoryTransport`'s global address registry, so reusing an address (e.g. +/// restarting from seed 0 each case) intermittently fails a later case's +/// `listen_on` with `Unreachable`. +static NEXT_MEMORY_ADDR: AtomicU64 = AtomicU64::new(1); + +/// Returns a fresh, never-reused `/memory/` address. +fn memory_addr() -> Multiaddr { + Multiaddr::empty().with(Protocol::Memory( + NEXT_MEMORY_ADDR.fetch_add(1, Ordering::Relaxed), + )) +} + +/// Builds one host wired to the shared `consensus` and `deadliner`, running +/// its priority behaviour over an in-process `MemoryTransport`. +fn build_host( + secret: SecretKey, + peer_id: PeerId, + peers: Vec, + min_required: i64, + consensus: Arc, + deadliner: DeadlinerHandle, + exchange_timeout: Duration, +) -> Host { + let keypair = keypair_from_secret_key(secret).expect("keypair"); + let validator = Box::new(|_: &PriorityMsg| Ok(())); + + let (prioritiser, behaviour) = Prioritiser::new_internal( + peer_id, + peers.clone(), + min_required, + consensus, + validator, + exchange_timeout, + deadliner, + P2PContext::new(peers), + ); + + let swarm = libp2p::SwarmBuilder::with_existing_identity(keypair) + .with_tokio() + .with_other_transport(|key| { + MemoryTransport::default() + .upgrade(Version::V1) + .authenticate(libp2p::noise::Config::new(key).expect("noise config")) + .multiplex(libp2p::yamux::Config::default()) + }) + .expect("transport") + .with_behaviour(|_key| behaviour) + .expect("behaviour") + .build(); + + Host { + swarm, + prioritiser, + addr: memory_addr(), + } +} + +/// Builds a signed `PriorityMsg` proposing `priorities` for `topic`, plus an +/// empty proposal for `ignored_topic` (mirroring `calculate.rs`'s own +/// `build_msgs` test helper, which is the source of this suite's vectors). +fn build_msg( + peer_id: PeerId, + secret: &SecretKey, + slot: u64, + topic: &str, + priorities: &[String], + ignored_topic: &str, +) -> PriorityMsg { + let topics = vec![ + PriorityTopicProposal::from(&TopicProposal { + topic: topic.to_owned(), + priorities: priorities.to_vec(), + }), + PriorityTopicProposal::from(&TopicProposal { + topic: ignored_topic.to_owned(), + priorities: Vec::new(), + }), + ]; + let msg = PriorityMsg { + duty: Some(ProtoDuty { slot, r#type: 0 }), + topics, + peer_id: peer_id.to_string(), + signature: Default::default(), + }; + sign_msg(&msg, secret).expect("sign") +} + +/// Runs one `priority_scoring` case end-to-end and returns the decided +/// `PriorityResult`. +/// +/// Peer ordering is load-bearing: `calculate_result` sorts input messages by +/// peer id string and breaks score ties by first-seen order in that sorted +/// input, so the vector's peer "0", "1", ... assumes ascending peer-id order. +/// Generated keypairs are sorted by their derived `PeerId` string and mapped +/// onto the vector's peers in that order. +async fn run_case(input: &serde_json::Value) -> PriorityResult { + let min_required = input["min_required"].as_i64().expect("min_required"); + let slot = input["slot"].as_u64().expect("slot"); + let topic = input["topic"].as_str().expect("topic"); + let ignored_topic = input["ignored_topic"].as_str().expect("ignored_topic"); + let peers_in = input["peers"].as_array().expect("peers"); + let n = peers_in.len(); + + for (idx, p) in peers_in.iter().enumerate() { + assert_eq!( + p["peer_id"].as_str().expect("peer_id"), + idx.to_string(), + "vector peers must be indexed \"0\", \"1\", ... in order" + ); + } + + let mut keyed: Vec<(SecretKey, PeerId)> = (0..n) + .map(|_| { + let key = SecretKey::random(&mut OsRng); + let peer_id = peer_id_from_key(key.public_key()).expect("peer id"); + (key, peer_id) + }) + .collect(); + keyed.sort_by_key(|(_, peer_id)| peer_id.to_string()); + + let peers: Vec = keyed.iter().map(|(_, p)| *p).collect(); + let consensus = Arc::new(MockConsensus::default()); + let exchange_timeout = Duration::from_millis(500); + + // One deadliner shared across all of this case's hosts; the expired-duty + // receiver is unused since this suite never calls `Prioritiser::start` + // (no cleanup loop needed for a short-lived per-case network). + let ct = CancellationToken::new(); + let (deadliner, _expired) = + DeadlinerTask::start(ct.clone(), "priority_scoring", FutureCalculator); + + let mut hosts: Vec = Vec::with_capacity(n); + for (key, peer_id) in &keyed { + hosts.push(build_host( + key.clone(), + *peer_id, + peers.clone(), + min_required, + consensus.clone(), + deadliner.clone(), + exchange_timeout, + )); + } + + // Capture the decided result once via a subscriber on host 0. Every host's + // `Prioritiser::new_internal` registers a forwarding closure with the + // shared `consensus`, so whichever host's exchange completes first still + // invokes host 0's own subscriber (decide-once semantics). + let (result_tx, result_rx) = oneshot::channel::(); + let result_tx = Arc::new(Mutex::new(Some(result_tx))); + { + let tx = result_tx.clone(); + hosts[0] + .prioritiser + .subscribe(Box::new(move |_duty, result| { + if let Some(sender) = tx.lock().expect("lock").take() { + let _ = sender.send(result); + } + Ok(()) + })); + } + + for host in &mut hosts { + host.swarm.listen_on(host.addr.clone()).expect("listen"); + } + for host in &mut hosts { + loop { + if matches!( + host.swarm.select_next_some().await, + SwarmEvent::NewListenAddr { .. } + ) { + break; + } + } + } + + let addrs: Vec = hosts.iter().map(|h| h.addr.clone()).collect(); + for (i, host) in hosts.iter_mut().enumerate() { + for (j, addr) in addrs.iter().enumerate() { + if i != j { + host.swarm.dial(addr.clone()).expect("dial"); + } + } + } + + if n > 1 { + let mut connected: Vec> = vec![HashSet::new(); n]; + let mesh = async { + while connected.iter().any(|c| c.len() < n - 1) { + let next = hosts + .iter_mut() + .map(|h| h.swarm.select_next_some().boxed()) + .collect::>(); + let (event, idx, _) = select_all(next).await; + if let SwarmEvent::ConnectionEstablished { peer_id, .. } = event { + connected[idx].insert(peer_id); + } + } + }; + timeout(Duration::from_secs(10), mesh) + .await + .expect("full connection mesh within timeout"); + } + + let mut msgs = Vec::with_capacity(n); + for (idx, peer_json) in peers_in.iter().enumerate() { + let priorities: Vec = peer_json["priorities"] + .as_array() + .expect("priorities") + .iter() + .map(|v| v.as_str().expect("priority string").to_owned()) + .collect(); + let (key, peer_id) = &keyed[idx]; + msgs.push(build_msg( + *peer_id, + key, + slot, + topic, + &priorities, + ignored_topic, + )); + } + + let mut drivers = Vec::with_capacity(n); + let mut prioritisers = Vec::with_capacity(n); + for host in hosts.into_iter() { + prioritisers.push(host.prioritiser); + let mut swarm = host.swarm; + drivers.push(tokio::spawn(async move { + loop { + let _ = swarm.select_next_some().await; + } + })); + } + + let mut prioritise_tasks = Vec::with_capacity(n); + for (prio, msg) in prioritisers.iter().zip(msgs) { + let prio = prio.clone(); + let ct = ct.clone(); + prioritise_tasks.push(tokio::spawn(async move { prio.prioritise(msg, ct).await })); + } + + let result = timeout(Duration::from_secs(10), result_rx) + .await + .expect("decided result within timeout") + .expect("result delivered"); + + ct.cancel(); + for d in drivers { + d.abort(); + } + for t in prioritise_tasks { + t.abort(); + } + + result +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn cases() { + let suite = load_suite("priority_scoring"); + let cases = suite["cases"].as_array().expect("cases array"); + let mut failures = Vec::new(); + + for case in cases { + let name = case["name"].as_str().expect("case name"); + let input = &case["input"]; + let topic = input["topic"].as_str().expect("topic"); + + let expected: Vec<(String, i64)> = case["result"] + .as_array() + .expect("result array") + .iter() + .map(|r| { + ( + r["priority"].as_str().expect("priority string").to_owned(), + r["score"].as_i64().expect("score"), + ) + }) + .collect(); + + let result = run_case(input).await; + + let topic_results: Vec = result + .topics + .iter() + .map(|t| TopicResult::try_from(t).expect("decode topic result")) + .collect(); + let Some(versions) = topic_results.iter().find(|t| t.topic == topic) else { + failures.push(format!( + "priority_scoring/{name}: topic {topic:?} absent from decided result entirely" + )); + continue; + }; + let actual: Vec<(String, i64)> = versions + .priorities + .iter() + .map(|p| (p.priority.clone(), p.score)) + .collect(); + + if actual != expected { + failures.push(format!( + "priority_scoring/{name}: got {actual:?}, want {expected:?}" + )); + } + } + + assert_eq!(cases.len(), 18, "expected 18 priority_scoring cases"); + assert!(failures.is_empty(), "{}", failures.join("\n")); +} + +/// Ladder entry "Preferred priority protocol ID `/charon/priority/2.0.0`" is +/// unreleased (`first_charon_release: null`). At the charon-v1.7.1 parity +/// anchor pluto must serve exactly the legacy slash-less ID. When pluto adds +/// the preferred ID, this test fails: flip it to assert both IDs, slash form +/// preferred, per `priority.md`. +#[test] +fn priority_protocol_ids() { + assert_eq!(pluto_priority::PROTOCOL_ID, "charon/priority/2.0.0"); + assert_eq!(pluto_priority::protocols(), vec!["charon/priority/2.0.0"]); +} diff --git a/plans/pluto-conformance.md b/plans/pluto-conformance.md index 9f8ce41..17f8bd6 100644 --- a/plans/pluto-conformance.md +++ b/plans/pluto-conformance.md @@ -71,7 +71,7 @@ pluto commit actually tested. Statuses: `todo` / `in progress` / `done`. | 2 | Proto encoding + SSZ hashing | `qbft_hashing` | done | FAIL (unsigned_data_set only) | `duty` (3/3), `qbft_signing_root` (6/6), `any_string` (4/4) all PASS. `unsigned_data_set` has 10/12 strict-PASS; `empty_value` and `empty_key` are a real divergence, no ladder entry: prost's map encoding applies proto3 default-value omission *inside* map entries (skips an empty string key or empty bytes value), while charon's Go marshaler always writes both map-entry fields explicitly. See Findings. Pinned as a named known-divergence test (`unsigned_data_set_known_divergence_empty_map_entry_fields`) rather than left permanently red, so a future regression in the other 10 cases can't hide behind it; that test fails loudly if pluto's output ever changes (fixed or otherwise). Pluto's own `crates/consensus/testdata/vectors/hashproto.json` never exercises an empty key/value entry, so this suite gives it strictly broader coverage — a candidate for pluto to adopt these vectors and retire its own file, not something this repo changes. | | 3 | BLS threshold aggregation | `bls_threshold` | done | PASS | All 5 groups PASS: `keys` (2/2), `partials` (4/4, pubshare + partial signature), `threshold_aggregates` (4/4 distinct 3-of-4 quorums, each reproducing the one `group_signature_hex`), `recovery` (1/1, `recover_secret` yields the exact `group_secret_hex`), `plain_aggregate` (1/1, matches the pinned bytes and does **not** verify under `group_pubkey_hex`, confirming plain aggregation is not a threshold-signature substitute). `group_signature_hex` also verifies under `group_pubkey_hex`. `pluto-crypto`'s `BlstImpl`/`Tbls` API matched the brief's guesses exactly; no divergence found. | | 4 | Cluster hashes + lock verification | `cluster_hashing` | done | FAIL (2 pinned known-divergence cases; at least 3 distinct affected fields — a lower bound, not a total) | `definition` 5/6 strict-PASS (`config_hash` and `definition_hash` both match on 5 cases, including the unsigned/signed pair proving config_hash is signature-independent); `all_empty_lists` is a real divergence, pinned (see Findings) — it carries **at least two independent parse blockers** (`operators`/`validators` null-rejection, then a masked `timestamp` missing-field rejection once the first is hypothetically fixed), so a partial upstream fix will not make it pass. `lock` 3/4 strict-PASS (`lock_hash` matches and `verify_hashes` succeeds on 3); `validator_without_deposit_data` is a second real divergence, pinned (see Findings). `real_keys_3_of_4`: hashes verify; the full `verify_signatures(&EthClient::new(""))` chain is asserted to fail specifically at the definition EIP-712 stage (`LockError::DefinitionSignaturesVerificationFailed`) because its operator/creator signatures are unavoidable placeholders (charon's EIP-712 signing helpers aren't exported for vector generation) — `Lock::verify_signatures` short-circuits there, so its private BLS-aggregate and node-signature checks are never reached from this external crate; that half is a coverage gap (not a failure), noted in the test's doc comment rather than mocked around. | -| 5 | Priority scoring | `priority_scoring` | todo | — | | +| 5 | Priority scoring | `priority_scoring` | done | PASS | All 18/18 cases PASS end-to-end through the public `Prioritiser` API over an in-process libp2p (`MemoryTransport` + noise + yamux) network, with a mock `Consensus` capturing the proposed `PriorityResult` (`calculate_result` itself is `pub(crate)`, unreachable directly). Covers below-quorum rejection (empty-result pins), single-priority and two-priority scoring, count-then-relative-priority ordering, and the two `test-case`-name-only "deterministic ordering" cases (score ties resolved by first-seen order in peer-id-sorted input — confirmed stable, no divergence from Charon v1.7.1's own stable-sort assumption). Peer `PeerId`s are generated then sorted ascending and mapped onto the vector's peer indices, since scoring tie-breaks are first-seen over peer-id-sorted input. Also pins `PROTOCOL_ID`/`protocols()` to the legacy slash-less `charon/priority/2.0.0` (ladder: preferred slash form is unreleased). Full 18-case suite runs in under 1s (well under the 2-minute budget). | | 6 | Round timer deadlines | `timer_deadlines` | todo | — | | | 7 | QBFT message limits | `qbft_msg_limits` | todo | — | | | 8 | DECIDED-resend limiting | `qbft_decided_resends` | todo | — | | From 27bc6c1ddda4ceb7ad297e716ee0d96113359205 Mon Sep 17 00:00:00 2001 From: Andrei Smirnov Date: Fri, 31 Jul 2026 22:17:02 +0300 Subject: [PATCH 11/27] Run timer_deadlines suite against pluto --- consumers/rust/Cargo.toml | 3 +- consumers/rust/tests/timer_deadlines.rs | 118 ++++++++++++++++++++++++ plans/pluto-conformance.md | 2 +- 3 files changed, 121 insertions(+), 2 deletions(-) create mode 100644 consumers/rust/tests/timer_deadlines.rs diff --git a/consumers/rust/Cargo.toml b/consumers/rust/Cargo.toml index 719baf1..fa7ad37 100644 --- a/consumers/rust/Cargo.toml +++ b/consumers/rust/Cargo.toml @@ -18,10 +18,11 @@ pluto-cluster = { path = "../../../pluto/crates/cluster" } pluto-eth1wrap = { path = "../../../pluto/crates/eth1wrap" } pluto-priority = { path = "../../../pluto/crates/priority" } pluto-p2p = { path = "../../../pluto/crates/p2p" } +pluto-featureset = { path = "../../../pluto/crates/featureset" } k256 = { version = "0.13.4", features = ["ecdsa", "sha256"] } prost = "0.14" prost-types = "0.14" -tokio = { version = "1", features = ["macros", "rt", "rt-multi-thread", "time", "sync"] } +tokio = { version = "1", features = ["macros", "rt", "rt-multi-thread", "time", "sync", "test-util"] } tokio-util = "0.7.11" futures = "0.3" async-trait = "0.1.89" diff --git a/consumers/rust/tests/timer_deadlines.rs b/consumers/rust/tests/timer_deadlines.rs new file mode 100644 index 0000000..1062222 --- /dev/null +++ b/consumers/rust/tests/timer_deadlines.rs @@ -0,0 +1,118 @@ +//! Runs the `timer_deadlines` suite against pluto's consensus round timers. +//! +//! Pluto's timers are purely relative to `round`. The stored `Duty` on each +//! timer is read in exactly one place in `crates/consensus/src/timer.rs`: +//! `is_proposer`, used by `proposal_timeout_duration` to pick the round-1 +//! proposer override. Nothing reads genesis time, slot duration, or slot +//! number. That splits the vectors' three columns: +//! +//! - `round_timeout_nanos`: tested below, against `RoundTimer::timer(round)`. +//! - `deadline_nanos` (absolute, genesis-derived): ABSENT-OK. Pluto's timers +//! never compute an absolute deadline from genesis; the deterministic, +//! genesis-derived eager-double-linear deadline is the +//! `charon_anchor.json` behaviours-ladder entry "Deterministic +//! (genesis-derived) eager double linear round deadlines" +//! (`first_charon_release: v1.9.0`), which is later than pluto's v1.7.1 +//! parity anchor. This pin is intentionally not asserted in code: there is +//! nothing in pluto to assert against, only its absence. +//! - `duty_start_delay_nanos`: UNREACHABLE. It lives in `delay_slot_offset` +//! in `crates/core/src/scheduler.rs`, an `async fn` with no `pub` modifier +//! and no external caller. + +use std::sync::Arc; +use std::time::Duration; + +use pluto_consensus::timer::{RoundTimer, get_round_timer_func}; +use pluto_core::types::{Duty, DutyType, SlotNumber}; +use pluto_featureset::{Config, Feature, FeatureSet, Status}; +use spec_vectors_pluto::load_suite; +use tokio::time::{Instant, advance}; + +/// Larger than the largest vector timeout (round 10 => 10s), so a single +/// jump always carries the paused clock past whichever deadline is pending. +const ADVANCE: Duration = Duration::from_secs(15); + +fn duty_type_from_name(name: &str) -> DutyType { + match name { + "PROPOSER" => DutyType::Proposer, + "ATTESTER" => DutyType::Attester, + "RANDAO" => DutyType::Randao, + "AGGREGATOR" => DutyType::Aggregator, + "SYNC_MESSAGE" => DutyType::SyncMessage, + "SYNC_CONTRIBUTION" => DutyType::SyncContribution, + other => panic!("timer_deadlines: unhandled duty_type_name {other}"), + } +} + +/// Awaits `timer.timer(round)` under a paused clock and returns the elapsed +/// time from just before the call to the future's resolved deadline. +/// +/// `sleep_until` does not resolve on its own under `start_paused = true`: +/// pluto's own timer tests (`assert_fires_after` in +/// `crates/consensus/src/timer.rs`'s test module) spawn the future, call +/// `tokio::time::advance` past the deadline, then `yield_now` before +/// checking it fired. The future's `Output` is the deadline `Instant` +/// itself (captured by the future before any advancing), so the returned +/// duration is exact regardless of how far past the deadline we advance. +async fn measure_round_timeout(timer: &dyn RoundTimer, round: i64) -> Duration { + let start = Instant::now(); + let fut = timer.timer(round).expect("timer future"); + let handle = tokio::spawn(fut); + advance(ADVANCE).await; + tokio::task::yield_now().await; + let deadline = handle.await.expect("timer task panicked"); + deadline - start +} + +#[tokio::test(start_paused = true)] +async fn round_timeouts() { + let suite = load_suite("timer_deadlines"); + let cases = suite["cases"].as_array().unwrap(); + assert_eq!(cases.len(), 216, "expected 216 timer_deadlines cases"); + + // The vectors are the eager double linear timer plus the proposer + // round-1 override. `EagerDoubleLinear` is Stable (enabled by + // `Config::default`'s `min_status: Stable`); `ProposalTimeout` is Alpha + // and must be turned on explicitly to get the proposer round-1 + // override the vectors encode; `Linear` stays off (its default). With + // `Linear` disabled, `get_round_timer_func` selects + // `EagerDoubleLinearRoundTimer` for every duty type -- confirmed + // against `get_round_timer_func`'s own branching and its + // `get_timer_func` unit test in `timer.rs`. + let feature_set = Arc::new( + FeatureSet::from_config(Config { + min_status: Status::Stable, + enabled: vec![Feature::ProposalTimeout], + disabled: vec![], + }) + .expect("valid feature config"), + ); + let timer_func = get_round_timer_func(feature_set); + + let mut failures = Vec::new(); + for case in cases { + let name = case["name"].as_str().unwrap(); + let input = &case["input"]; + let slot = input["slot"].as_u64().unwrap(); + let round = input["round"].as_i64().unwrap(); + let duty_type = duty_type_from_name(input["duty_type_name"].as_str().unwrap()); + let want = Duration::from_nanos(case["round_timeout_nanos"].as_u64().unwrap()); + + // A fresh timer per case: `EagerDoubleLinearRoundTimer` caches a + // round's first deadline and doubles it on a second `timer()` call + // for that same round. Calling `timer_func` once per case (as + // pluto itself does once per consensus instance) means no case + // shares that cache with another. + let duty = Duty::new(SlotNumber::from(slot), duty_type); + let timer = timer_func(duty); + + let got = measure_round_timeout(timer.as_ref(), round).await; + if got != want { + failures.push(format!( + "timer_deadlines/{name}: round timeout {got:?}, want {want:?}" + )); + } + } + + assert!(failures.is_empty(), "{}", failures.join("\n")); +} diff --git a/plans/pluto-conformance.md b/plans/pluto-conformance.md index 17f8bd6..3fbf007 100644 --- a/plans/pluto-conformance.md +++ b/plans/pluto-conformance.md @@ -72,7 +72,7 @@ pluto commit actually tested. Statuses: `todo` / `in progress` / `done`. | 3 | BLS threshold aggregation | `bls_threshold` | done | PASS | All 5 groups PASS: `keys` (2/2), `partials` (4/4, pubshare + partial signature), `threshold_aggregates` (4/4 distinct 3-of-4 quorums, each reproducing the one `group_signature_hex`), `recovery` (1/1, `recover_secret` yields the exact `group_secret_hex`), `plain_aggregate` (1/1, matches the pinned bytes and does **not** verify under `group_pubkey_hex`, confirming plain aggregation is not a threshold-signature substitute). `group_signature_hex` also verifies under `group_pubkey_hex`. `pluto-crypto`'s `BlstImpl`/`Tbls` API matched the brief's guesses exactly; no divergence found. | | 4 | Cluster hashes + lock verification | `cluster_hashing` | done | FAIL (2 pinned known-divergence cases; at least 3 distinct affected fields — a lower bound, not a total) | `definition` 5/6 strict-PASS (`config_hash` and `definition_hash` both match on 5 cases, including the unsigned/signed pair proving config_hash is signature-independent); `all_empty_lists` is a real divergence, pinned (see Findings) — it carries **at least two independent parse blockers** (`operators`/`validators` null-rejection, then a masked `timestamp` missing-field rejection once the first is hypothetically fixed), so a partial upstream fix will not make it pass. `lock` 3/4 strict-PASS (`lock_hash` matches and `verify_hashes` succeeds on 3); `validator_without_deposit_data` is a second real divergence, pinned (see Findings). `real_keys_3_of_4`: hashes verify; the full `verify_signatures(&EthClient::new(""))` chain is asserted to fail specifically at the definition EIP-712 stage (`LockError::DefinitionSignaturesVerificationFailed`) because its operator/creator signatures are unavoidable placeholders (charon's EIP-712 signing helpers aren't exported for vector generation) — `Lock::verify_signatures` short-circuits there, so its private BLS-aggregate and node-signature checks are never reached from this external crate; that half is a coverage gap (not a failure), noted in the test's doc comment rather than mocked around. | | 5 | Priority scoring | `priority_scoring` | done | PASS | All 18/18 cases PASS end-to-end through the public `Prioritiser` API over an in-process libp2p (`MemoryTransport` + noise + yamux) network, with a mock `Consensus` capturing the proposed `PriorityResult` (`calculate_result` itself is `pub(crate)`, unreachable directly). Covers below-quorum rejection (empty-result pins), single-priority and two-priority scoring, count-then-relative-priority ordering, and the two `test-case`-name-only "deterministic ordering" cases (score ties resolved by first-seen order in peer-id-sorted input — confirmed stable, no divergence from Charon v1.7.1's own stable-sort assumption). Peer `PeerId`s are generated then sorted ascending and mapped onto the vector's peer indices, since scoring tie-breaks are first-seen over peer-id-sorted input. Also pins `PROTOCOL_ID`/`protocols()` to the legacy slash-less `charon/priority/2.0.0` (ladder: preferred slash form is unreleased). Full 18-case suite runs in under 1s (well under the 2-minute budget). | -| 6 | Round timer deadlines | `timer_deadlines` | todo | — | | +| 6 | Round timer deadlines | `timer_deadlines` | done | round_timeout: PASS; deadline: ABSENT-OK; duty_start_delay: UNREACHABLE | All 216/216 cases PASS on `round_timeout_nanos`: a fresh `EagerDoubleLinearRoundTimer` per case (via `get_round_timer_func` with `EagerDoubleLinear` Stable-default and `ProposalTimeout` explicitly enabled, `Linear` left off) reproduces every round-1/2/3/10 timeout across all 6 duty types and all 3 slot durations, awaited under `#[tokio::test(start_paused = true)]` with `tokio::time::advance` (the future does not resolve on its own under a paused clock — confirmed against pluto's own `assert_fires_after` test helper before writing assertions). `deadline_nanos` is ABSENT-OK: confirmed by full-file reading of `timer.rs` that no timer consumes genesis time, slot duration, or slot number — the stored `Duty` is read only via `is_proposer` for the round-1 proposer override — matching ladder entry "Deterministic (genesis-derived) eager double linear round deadlines" (`first_charon_release: v1.9.0` > pluto's v1.7.1 anchor); nothing is asserted for this column since pluto computes no absolute deadline to pin. `duty_start_delay_nanos` is UNREACHABLE: `delay_slot_offset` (`crates/core/src/scheduler.rs`) is a non-`pub` `async fn` with no external caller, confirmed still private. No divergence found on the tested column, so no new Findings entry. | | 7 | QBFT message limits | `qbft_msg_limits` | todo | — | | | 8 | DECIDED-resend limiting | `qbft_decided_resends` | todo | — | | | 9 | Sender binding + peer map | `parsigex_sender_binding` | todo | — | | From 48aab75403326404b2b7ef0a72b3d8310d230ec8 Mon Sep 17 00:00:00 2001 From: Andrei Smirnov Date: Fri, 31 Jul 2026 22:31:06 +0300 Subject: [PATCH 12/27] Pin timer_deadlines default-config divergence and slot invariance round_timeouts previously only tested with ProposalTimeout enabled, which overstated parity: pluto's default FeatureSet disables it, diverging from the vectors on proposer round-1 cases. Split into an enabled-feature test (matches the vectors) and a default-feature test that pins the divergence explicitly. Also add a real code-level pin for slot-invariance (deadline_nanos ABSENT-OK), replacing a doc-only claim with an assertion that fails if pluto's timers ever grow slot-dependence. --- consumers/rust/tests/timer_deadlines.rs | 236 ++++++++++++++++++++---- plans/pluto-conformance.md | 34 +++- 2 files changed, 236 insertions(+), 34 deletions(-) diff --git a/consumers/rust/tests/timer_deadlines.rs b/consumers/rust/tests/timer_deadlines.rs index 1062222..8fc7da2 100644 --- a/consumers/rust/tests/timer_deadlines.rs +++ b/consumers/rust/tests/timer_deadlines.rs @@ -4,21 +4,43 @@ //! timer is read in exactly one place in `crates/consensus/src/timer.rs`: //! `is_proposer`, used by `proposal_timeout_duration` to pick the round-1 //! proposer override. Nothing reads genesis time, slot duration, or slot -//! number. That splits the vectors' three columns: +//! number. That splits the vectors' three columns across three tests below: //! -//! - `round_timeout_nanos`: tested below, against `RoundTimer::timer(round)`. -//! - `deadline_nanos` (absolute, genesis-derived): ABSENT-OK. Pluto's timers -//! never compute an absolute deadline from genesis; the deterministic, -//! genesis-derived eager-double-linear deadline is the -//! `charon_anchor.json` behaviours-ladder entry "Deterministic -//! (genesis-derived) eager double linear round deadlines" -//! (`first_charon_release: v1.9.0`), which is later than pluto's v1.7.1 -//! parity anchor. This pin is intentionally not asserted in code: there is -//! nothing in pluto to assert against, only its absence. -//! - `duty_start_delay_nanos`: UNREACHABLE. It lives in `delay_slot_offset` -//! in `crates/core/src/scheduler.rs`, an `async fn` with no `pub` modifier -//! and no external caller. +//! - `round_timeouts_with_proposal_timeout_enabled`: `round_timeout_nanos` +//! under the feature configuration the vectors assume. See Fix round 1 +//! below for why this alone overstates parity. +//! - `round_timeouts_pluto_default_feature_set`: the same column under +//! pluto's actual *default* configuration, which pins a real divergence on +//! `PROPOSER`/round-1 cases (see "Fix round 1"). +//! - `round_timeout_is_duty_slot_invariant`: a real, code-level pin for the +//! `deadline_nanos` ABSENT-OK column -- see that test's doc comment. +//! +//! ## Fix round 1 +//! +//! The first version of this suite enabled `Feature::ProposalTimeout` and +//! reported an unqualified PASS on `round_timeout_nanos`. That is true only +//! under a non-default feature configuration and hid a real divergence: +//! +//! - Pluto's `Config::default()` has `min_status: Status::Stable` and +//! `Feature::ProposalTimeout` is `Status::Alpha` +//! (`crates/featureset/src/lib.rs`), so it is **disabled by default**; +//! pluto's own `default_matches_go_implementation` test asserts this is +//! deliberate charon parity. +//! - Charon at the vectors' anchor (`6054bcb2`) has `ProposalTimeout` at +//! `statusStable` with a `statusStable` minimum +//! (`app/featureset/featureset.go`), i.e. **enabled by default** -- it was +//! promoted from alpha in charon commit `06b6371b` ("promote +//! proposal_timeout feature flag to stable", #4296), first released in +//! **v1.9.0**, after pluto's v1.7.1 parity anchor. +//! +//! So a default-configured pluto disagrees with the vectors on every +//! `PROPOSER`/round-1 case (pluto: 1s; vectors: 1.5s), and that disagreement +//! is the same class of gap as the `deadline_nanos` column below: a v1.9.0 +//! charon behaviour that postdates pluto's pin. `round_timeouts_pluto_default_feature_set` +//! pins it explicitly instead of letting the enabled-feature test's PASS +//! stand in for it. +use std::collections::HashSet; use std::sync::Arc; use std::time::Duration; @@ -44,6 +66,21 @@ fn duty_type_from_name(name: &str) -> DutyType { } } +/// Builds the feature set the vectors assume: `EagerDoubleLinear` at its +/// Stable default, plus `ProposalTimeout` explicitly enabled (it is Alpha, +/// hence off, under `Config::default()`), `Linear` left at its Alpha +/// default (off). +fn feature_set_with_proposal_timeout_enabled() -> Arc { + Arc::new( + FeatureSet::from_config(Config { + min_status: Status::Stable, + enabled: vec![Feature::ProposalTimeout], + disabled: vec![], + }) + .expect("valid feature config"), + ) +} + /// Awaits `timer.timer(round)` under a paused clock and returns the elapsed /// time from just before the call to the future's resolved deadline. /// @@ -65,29 +102,19 @@ async fn measure_round_timeout(timer: &dyn RoundTimer, round: i64) -> Duration { } #[tokio::test(start_paused = true)] -async fn round_timeouts() { +async fn round_timeouts_with_proposal_timeout_enabled() { let suite = load_suite("timer_deadlines"); let cases = suite["cases"].as_array().unwrap(); assert_eq!(cases.len(), 216, "expected 216 timer_deadlines cases"); - // The vectors are the eager double linear timer plus the proposer - // round-1 override. `EagerDoubleLinear` is Stable (enabled by - // `Config::default`'s `min_status: Stable`); `ProposalTimeout` is Alpha - // and must be turned on explicitly to get the proposer round-1 - // override the vectors encode; `Linear` stays off (its default). With - // `Linear` disabled, `get_round_timer_func` selects - // `EagerDoubleLinearRoundTimer` for every duty type -- confirmed - // against `get_round_timer_func`'s own branching and its - // `get_timer_func` unit test in `timer.rs`. - let feature_set = Arc::new( - FeatureSet::from_config(Config { - min_status: Status::Stable, - enabled: vec![Feature::ProposalTimeout], - disabled: vec![], - }) - .expect("valid feature config"), - ); - let timer_func = get_round_timer_func(feature_set); + // With `Linear` off and `EagerDoubleLinear` on (its Stable default), + // `get_round_timer_func` selects `EagerDoubleLinearRoundTimer` for every + // duty type -- confirmed against `get_round_timer_func`'s own branching + // and its `get_timer_func` unit test in `timer.rs`. This PASS holds + // *only* under `ProposalTimeout` enabled; see the module doc comment + // and `round_timeouts_pluto_default_feature_set` below for pluto's + // actual default behaviour. + let timer_func = get_round_timer_func(feature_set_with_proposal_timeout_enabled()); let mut failures = Vec::new(); for case in cases { @@ -109,7 +136,150 @@ async fn round_timeouts() { let got = measure_round_timeout(timer.as_ref(), round).await; if got != want { failures.push(format!( - "timer_deadlines/{name}: round timeout {got:?}, want {want:?}" + "timer_deadlines/{name}: round timeout {got:?}, want {want:?} (ProposalTimeout enabled)" + )); + } + } + + assert!(failures.is_empty(), "{}", failures.join("\n")); +} + +#[tokio::test(start_paused = true)] +async fn round_timeouts_pluto_default_feature_set() { + let suite = load_suite("timer_deadlines"); + let cases = suite["cases"].as_array().unwrap(); + assert_eq!(cases.len(), 216, "expected 216 timer_deadlines cases"); + + // Pluto's actual default: `FeatureSet::default()`, i.e. + // `Config::default()` (`min_status: Stable`, nothing explicitly + // enabled/disabled) -- `ProposalTimeout` stays off. This is the + // configuration pluto ships with at its v1.7.1 parity anchor. + let timer_func = get_round_timer_func(Arc::new(FeatureSet::default())); + + // Pinned known-divergence: proposer round-1 timeout under charon + // v1.7.1 parity (pluto's default) is 1s, not the vectors' 1.5s. See the + // module doc comment ("Fix round 1") for why -- charon promoted + // `ProposalTimeout` to enabled-by-default in v1.9.0 (#4296), after + // pluto's anchor. This must flip loudly (the `assert_eq!` below fails) + // the moment pluto's default configuration changes in either direction. + const PINNED_DEFAULT_PROPOSER_ROUND1_TIMEOUT: Duration = Duration::from_secs(1); + + let mut divergent_cases = 0usize; + let mut failures = Vec::new(); + for case in cases { + let name = case["name"].as_str().unwrap(); + let input = &case["input"]; + let slot = input["slot"].as_u64().unwrap(); + let round = input["round"].as_i64().unwrap(); + let duty_type_name = input["duty_type_name"].as_str().unwrap(); + let duty_type = duty_type_from_name(duty_type_name); + let want = Duration::from_nanos(case["round_timeout_nanos"].as_u64().unwrap()); + + let duty = Duty::new(SlotNumber::from(slot), duty_type); + let timer = timer_func(duty); + let got = measure_round_timeout(timer.as_ref(), round).await; + + let is_pinned_divergence = duty_type_name == "PROPOSER" && round == 1; + if is_pinned_divergence { + divergent_cases += 1; + if got != PINNED_DEFAULT_PROPOSER_ROUND1_TIMEOUT { + failures.push(format!( + "timer_deadlines/{name}: default-config round timeout {got:?}, \ + want pinned {PINNED_DEFAULT_PROPOSER_ROUND1_TIMEOUT:?} (ProposalTimeout default-disabled)" + )); + } + assert_ne!( + got, want, + "timer_deadlines/{name}: expected pluto's default config to diverge from the \ + vector (which assumes ProposalTimeout enabled), but it matched -- pluto's \ + default must have changed; update this pin" + ); + } else if got != want { + failures.push(format!( + "timer_deadlines/{name}: round timeout {got:?}, want {want:?} (pluto default config)" + )); + } + } + + // 6 duty types x 3 slots x 4 rounds x 3 slot durations = 216 cases; + // exactly the PROPOSER/round-1 cases (1 duty type x 3 slots x 3 + // durations = 9) are the pinned divergence. + assert_eq!( + divergent_cases, 9, + "expected exactly 9 PROPOSER/round-1 cases pinning the default-config divergence" + ); + + assert!(failures.is_empty(), "{}", failures.join("\n")); +} + +/// Real, code-level pin for the `deadline_nanos` ABSENT-OK column. +/// +/// The vectors' `deadline_nanos` is genesis- and slot-derived +/// (`deadline_nanos = genesis + slot_duration * slot + duty_start_delay + +/// round_timeout`), a determinism pluto does not implement -- see the +/// module doc comment. That absence is only a real ABSENT-OK pin if it is +/// asserted in code, not just claimed in a comment: this test builds two +/// timers for the same duty type and round, differing only in `slot`, and +/// asserts their measured round timeouts are bit-identical. If +/// `EagerDoubleLinearRoundTimer::with_duty` (or `RoundTimer::timer`) ever +/// grows slot-dependence, this fails loudly -- which is exactly the ladder +/// entry "Deterministic (genesis-derived) eager double linear round +/// deadlines" (`first_charon_release: v1.9.0` > pluto's v1.7.1 anchor) +/// flipping. +/// +/// Genesis-invariance is not separately exercised here: `RoundTimer::timer` +/// takes only `round`, and the `Duty` it is built from +/// (`pluto_core::types::Duty`) carries only `slot` and `duty_type` -- there +/// is no genesis-time parameter anywhere in this API for a case to vary, so +/// genesis-invariance is guaranteed by the type signature rather than +/// something a test can exercise. +#[tokio::test(start_paused = true)] +async fn round_timeout_is_duty_slot_invariant() { + let suite = load_suite("timer_deadlines"); + let cases = suite["cases"].as_array().unwrap(); + + // Every (duty_type, round) combination the vectors actually exercise, + // so this pin never drifts out of sync with the suite it stands in for. + let mut combos: Vec<(String, i64)> = cases + .iter() + .map(|case| { + let input = &case["input"]; + ( + input["duty_type_name"].as_str().unwrap().to_string(), + input["round"].as_i64().unwrap(), + ) + }) + .collect::>() + .into_iter() + .collect(); + combos.sort(); + assert_eq!( + combos.len(), + 24, + "expected 6 duty types x 4 rounds = 24 distinct (duty_type, round) combinations" + ); + + let timer_func = get_round_timer_func(feature_set_with_proposal_timeout_enabled()); + // The vectors' own slot extremes: 0, 1, and a slot far into an epoch + // (7231) -- as wide a spread as the vectors themselves test. + let (slot_a, slot_b) = (0u64, 7231u64); + + let mut failures = Vec::new(); + for (duty_type_name, round) in combos { + let duty_type = duty_type_from_name(&duty_type_name); + + let duty_a = Duty::new(SlotNumber::from(slot_a), duty_type.clone()); + let timer_a = timer_func(duty_a); + let got_a = measure_round_timeout(timer_a.as_ref(), round).await; + + let duty_b = Duty::new(SlotNumber::from(slot_b), duty_type); + let timer_b = timer_func(duty_b); + let got_b = measure_round_timeout(timer_b.as_ref(), round).await; + + if got_a != got_b { + failures.push(format!( + "timer_deadlines/slot_invariance/{duty_type_name}_round{round}: \ + slot {slot_a} gave {got_a:?}, slot {slot_b} gave {got_b:?}" )); } } diff --git a/plans/pluto-conformance.md b/plans/pluto-conformance.md index 3fbf007..06eb5fe 100644 --- a/plans/pluto-conformance.md +++ b/plans/pluto-conformance.md @@ -72,7 +72,7 @@ pluto commit actually tested. Statuses: `todo` / `in progress` / `done`. | 3 | BLS threshold aggregation | `bls_threshold` | done | PASS | All 5 groups PASS: `keys` (2/2), `partials` (4/4, pubshare + partial signature), `threshold_aggregates` (4/4 distinct 3-of-4 quorums, each reproducing the one `group_signature_hex`), `recovery` (1/1, `recover_secret` yields the exact `group_secret_hex`), `plain_aggregate` (1/1, matches the pinned bytes and does **not** verify under `group_pubkey_hex`, confirming plain aggregation is not a threshold-signature substitute). `group_signature_hex` also verifies under `group_pubkey_hex`. `pluto-crypto`'s `BlstImpl`/`Tbls` API matched the brief's guesses exactly; no divergence found. | | 4 | Cluster hashes + lock verification | `cluster_hashing` | done | FAIL (2 pinned known-divergence cases; at least 3 distinct affected fields — a lower bound, not a total) | `definition` 5/6 strict-PASS (`config_hash` and `definition_hash` both match on 5 cases, including the unsigned/signed pair proving config_hash is signature-independent); `all_empty_lists` is a real divergence, pinned (see Findings) — it carries **at least two independent parse blockers** (`operators`/`validators` null-rejection, then a masked `timestamp` missing-field rejection once the first is hypothetically fixed), so a partial upstream fix will not make it pass. `lock` 3/4 strict-PASS (`lock_hash` matches and `verify_hashes` succeeds on 3); `validator_without_deposit_data` is a second real divergence, pinned (see Findings). `real_keys_3_of_4`: hashes verify; the full `verify_signatures(&EthClient::new(""))` chain is asserted to fail specifically at the definition EIP-712 stage (`LockError::DefinitionSignaturesVerificationFailed`) because its operator/creator signatures are unavoidable placeholders (charon's EIP-712 signing helpers aren't exported for vector generation) — `Lock::verify_signatures` short-circuits there, so its private BLS-aggregate and node-signature checks are never reached from this external crate; that half is a coverage gap (not a failure), noted in the test's doc comment rather than mocked around. | | 5 | Priority scoring | `priority_scoring` | done | PASS | All 18/18 cases PASS end-to-end through the public `Prioritiser` API over an in-process libp2p (`MemoryTransport` + noise + yamux) network, with a mock `Consensus` capturing the proposed `PriorityResult` (`calculate_result` itself is `pub(crate)`, unreachable directly). Covers below-quorum rejection (empty-result pins), single-priority and two-priority scoring, count-then-relative-priority ordering, and the two `test-case`-name-only "deterministic ordering" cases (score ties resolved by first-seen order in peer-id-sorted input — confirmed stable, no divergence from Charon v1.7.1's own stable-sort assumption). Peer `PeerId`s are generated then sorted ascending and mapped onto the vector's peer indices, since scoring tie-breaks are first-seen over peer-id-sorted input. Also pins `PROTOCOL_ID`/`protocols()` to the legacy slash-less `charon/priority/2.0.0` (ladder: preferred slash form is unreleased). Full 18-case suite runs in under 1s (well under the 2-minute budget). | -| 6 | Round timer deadlines | `timer_deadlines` | done | round_timeout: PASS; deadline: ABSENT-OK; duty_start_delay: UNREACHABLE | All 216/216 cases PASS on `round_timeout_nanos`: a fresh `EagerDoubleLinearRoundTimer` per case (via `get_round_timer_func` with `EagerDoubleLinear` Stable-default and `ProposalTimeout` explicitly enabled, `Linear` left off) reproduces every round-1/2/3/10 timeout across all 6 duty types and all 3 slot durations, awaited under `#[tokio::test(start_paused = true)]` with `tokio::time::advance` (the future does not resolve on its own under a paused clock — confirmed against pluto's own `assert_fires_after` test helper before writing assertions). `deadline_nanos` is ABSENT-OK: confirmed by full-file reading of `timer.rs` that no timer consumes genesis time, slot duration, or slot number — the stored `Duty` is read only via `is_proposer` for the round-1 proposer override — matching ladder entry "Deterministic (genesis-derived) eager double linear round deadlines" (`first_charon_release: v1.9.0` > pluto's v1.7.1 anchor); nothing is asserted for this column since pluto computes no absolute deadline to pin. `duty_start_delay_nanos` is UNREACHABLE: `delay_slot_offset` (`crates/core/src/scheduler.rs`) is a non-`pub` `async fn` with no external caller, confirmed still private. No divergence found on the tested column, so no new Findings entry. | +| 6 | Round timer deadlines | `timer_deadlines` | done | round_timeout: PASS **only under `ProposalTimeout` enabled**, a documented non-default precondition (pluto's *default* config has a pinned, real divergence on `PROPOSER`/round-1 cases — see Notes and Findings); deadline: ABSENT-OK, with a real code-level slot-invariance pin (not just a comment); duty_start_delay: UNREACHABLE | `round_timeouts_with_proposal_timeout_enabled`: 216/216 PASS, but only with `Feature::ProposalTimeout` explicitly enabled (it is `Status::Alpha`, off under pluto's `Config::default()`). `round_timeouts_pluto_default_feature_set` runs the same 216 cases under pluto's actual default `FeatureSet` and pins the resulting divergence: the 207 non-`PROPOSER`-round-1 cases still match the vectors, but all 9 `PROPOSER`/round-1 cases (3 slots x 3 slot durations) diverge — pluto's default gives 1s, the vectors (which assume `ProposalTimeout` enabled) expect 1.5s. This is a real, pinned known-divergence (see Findings), not swept under the enabled-feature test's PASS. `deadline_nanos`: `round_timeout_is_duty_slot_invariant` builds two timers per (duty_type, round) combination (24 total, covering all cases) differing only in `slot` (0 vs. 7231) and asserts bit-identical measured timeouts — a real code pin for "no timer reads slot", not a doc comment, so it fails loudly if `with_duty`/`RoundTimer::timer` ever grows slot-dependence; genesis-invariance is structurally guaranteed (the API has no genesis parameter at all) and is stated as such rather than tested. Matches ladder entry "Deterministic (genesis-derived) eager double linear round deadlines" (`first_charon_release: v1.9.0` > pluto's v1.7.1 anchor). `duty_start_delay_nanos` is UNREACHABLE: `delay_slot_offset` (`crates/core/src/scheduler.rs`) is a non-`pub` `async fn` with no external caller, confirmed still private. | | 7 | QBFT message limits | `qbft_msg_limits` | todo | — | | | 8 | DECIDED-resend limiting | `qbft_decided_resends` | todo | — | | | 9 | Sender binding + peer map | `parsigex_sender_binding` | todo | — | | @@ -184,6 +184,38 @@ whether a ladder entry covers it, and whether it was reported upstream._ that cannot run is worse than a documented gap. Not reported upstream to pluto yet. +- **Pluto's default feature configuration gives a proposer round-1 timeout of 1s, + not the vectors' 1.5s (pinned known-divergence, task 6, + `timer_deadlines/round_timeouts_pluto_default_feature_set`).** Pluto's + `Config::default()` has `min_status: Status::Stable` and + `Feature::ProposalTimeout` is `Status::Alpha` (`crates/featureset/src/lib.rs`), + so it is disabled under pluto's own default — confirmed deliberate by pluto's + own `default_matches_go_implementation` test. Charon at the vectors' anchor + (`6054bcb2`) has `ProposalTimeout` at `statusStable` with a `statusStable` + minimum (`app/featureset/featureset.go`), i.e. enabled by default: it was + promoted from alpha in charon commit `06b6371b` ("promote proposal_timeout + feature flag to stable", #4296), first released in **v1.9.0**, after pluto's + v1.7.1 parity anchor. So this is the same class of gap as the `deadline_nanos` + ABSENT-OK column (a v1.9.0 charon behaviour pluto's v1.7.1 pin predates), not a + bug — but it means the suite's headline `round_timeout_nanos` PASS holds only + under a non-default `FeatureSet` (`ProposalTimeout` explicitly enabled), which + the task now tests and documents as an explicit precondition rather than + presenting as an unqualified default-config PASS. Not reported upstream to + pluto (this is pluto correctly tracking its own v1.7.1 anchor, not a defect). + +- **Recommendation for this repo (not pluto): `charon_anchor.json`'s `behaviours` + ladder has no entry for the proposal-timeout feature-flag stabilization + (task 6).** The promotion of `ProposalTimeout` from alpha to stable-by-default + in charon v1.9.0 (`06b6371b`, #4296) is exactly the kind of postdating-v1.7.1 + behaviour change the ladder exists to track, and the `timer_deadlines` vectors + already depend on it (they assume `ProposalTimeout` enabled), but no ladder + entry names it — unlike the sibling "Deterministic (genesis-derived) eager + double linear round deadlines" entry, which does. Adding an entry is a + separate, self-contained change to this repo's release surface: `README`'s + compatibility table documents the ladder and is enforced by + `tests/test_release.py`, so the entry and the README update need to land + together, in their own commit — not folded into this conformance task. + ## Global Constraints - Pluto is **never modified**. The harness lives in `consumers/rust/` and path-depends From 858b41dfc00f084f07a308b1f54048d558d7d3c6 Mon Sep 17 00:00:00 2001 From: Andrei Smirnov Date: Fri, 31 Jul 2026 22:46:03 +0300 Subject: [PATCH 13/27] Run qbft_msg_limits suite against pluto --- consumers/rust/tests/qbft_msg_limits.rs | 466 ++++++++++++++++++++++++ plans/pluto-conformance.md | 37 +- 2 files changed, 502 insertions(+), 1 deletion(-) create mode 100644 consumers/rust/tests/qbft_msg_limits.rs diff --git a/consumers/rust/tests/qbft_msg_limits.rs b/consumers/rust/tests/qbft_msg_limits.rs new file mode 100644 index 0000000..ec51534 --- /dev/null +++ b/consumers/rust/tests/qbft_msg_limits.rs @@ -0,0 +1,466 @@ +//! Runs the `qbft_msg_limits` suite against pluto's QBFT message intake. +//! +//! The vectors describe the spec's amplification limits on an inbound +//! `QbftConsensusMsg`: `justifications <= 2*nodes`, `values <= +//! 2*(justifications+1)`, and a wire size of at most 32 MiB. Pluto does not +//! implement either count limit as the spec states it: +//! +//! - Justifications: pluto's own cap is `4 * node_count()` (component.rs +//! `MAX_JUSTIFICATIONS_PER_NODE = 4`, checked with a strict `>`), not the +//! spec's `2n`. This is unilateral defensive hardening pluto chose +//! (documented in its own doc comment as bounding secp256k1-recovery work, +//! independent of charon), not a charon v1.7.1 behaviour and not the +//! spec's formula either. See Findings. +//! - Values: there is no length check on `QbftConsensusMsg.values` anywhere +//! in `crates/consensus/src` or `crates/core/src` — confirmed by +//! exhaustive grep. `values_by_hash` iterates an unbounded `Vec`. +//! +//! Both gaps match the `charon_anchor.json` ladder entry "QBFT +//! DECIDED-resend rate limit and message size/count limits" +//! (`first_charon_release: null` — absent from every released charon, so +//! pluto legitimately trails it at the v1.7.1 anchor). Every `counts` vector +//! case whose `justification_count` sits between the spec's `2n+1` and +//! pluto's `4n` is therefore an ABSENT-OK divergence: the spec says reject, +//! pluto accepts, and no ladder entry excuses over-rejection in the other +//! direction. `counts` below proves structurally (not by case name) that +//! every vector case's `justification_count` stays at or under pluto's `4n`, +//! so pluto is expected to accept every one of the 11 cases regardless of +//! what the spec says; a rejection would be a real FAIL (over-rejection +//! breaks liveness) and is reported as such. +//! +//! Because the vectors never probe pluto's own `4n` boundary (its largest +//! `nodes=4` case only reaches `justification_count=9`, far under `4*4=16`), +//! `pluto_own_justification_cap_accepts_at_boundary` and +//! `pluto_own_justification_cap_rejects_one_over_boundary` pin that contract +//! directly against real messages, independent of the vector file. +//! +//! `wire_size` exercises the actual reader pluto's QBFT p2p layer uses +//! (`pluto_p2p::proto::read_protobuf_with_max_size`) with the 32 MiB +//! consensus-specific limit, mirroring `crates/consensus/src/qbft/p2p.rs`'s +//! own unit tests (`reference_framed_message_decodes`, +//! `inbound_rejects_message_exceeding_max_consensus_size`), including their +//! `futures::io::Cursor` in-memory stream — the brief suggested +//! `tokio::io::duplex`, but pluto's own reference tests use `Cursor`, which +//! needs no additional dependency or feature and exercises the identical +//! `AsyncRead` code path. + +use std::sync::Arc; + +use futures::io::Cursor; +use k256::SecretKey; +use prost::Message; +use prost_types::Any; +use tokio_util::sync::CancellationToken; + +use pluto_consensus::qbft::msg::hash_proto; +use pluto_consensus::qbft::{Config, Consensus, Error, Peer}; +use pluto_consensus::timer::get_round_timer_func; +use pluto_core::corepb::v1::{consensus as pbconsensus, core as pbcore}; +use pluto_core::deadline::{DeadlineCalculator, DeadlinerTask}; +use pluto_core::qbft::{MSG_PRE_PREPARE, MSG_ROUND_CHANGE}; +use pluto_core::types::{Duty, DutyType, SlotNumber}; +use pluto_featureset::FeatureSet; +use spec_vectors_pluto::load_suite; + +/// Pluto's own hardening cap (`crates/consensus/src/qbft/component.rs` +/// `MAX_JUSTIFICATIONS_PER_NODE`), unrelated to the spec's `2n`. See the +/// module doc comment. +const PLUTO_MAX_JUSTIFICATIONS_PER_NODE: usize = 4; + +fn secret_key(seed: u8) -> SecretKey { + SecretKey::from_slice(&[seed; 32]).expect("valid seed bytes") +} + +fn build_peers(nodes: usize) -> Vec { + (0..nodes) + .map(|i| Peer { + index: i64::try_from(i).expect("small node count fits i64"), + name: format!("node-{i}"), + public_key: secret_key(u8::try_from(i + 1).expect("small seed fits u8")).public_key(), + }) + .collect() +} + +/// A `DeadlineCalculator` that always reports a deadline an hour in the +/// future, so `Consensus::handle`'s `add_deadline` call always yields +/// `AddOutcome::Scheduled`. +/// +/// Pluto's public `NeverExpiringCalculator` looks like the obvious fit here, +/// but it returns `Ok(None)` ("no deadline"), which `handle`'s admission +/// check (`add_deadline(...).await != AddOutcome::Scheduled`) treats as a +/// rejection (`Error::DutyExpired`) -- `NoDeadline` is *not* `Scheduled`. +/// This mirrors pluto's own internal test helper `FutureCalculator` +/// (`crates/consensus/src/qbft/component.rs`, used via `config_base(false)` +/// for exactly this reason). +struct FutureDeadlineCalculator; + +impl DeadlineCalculator for FutureDeadlineCalculator { + fn deadline( + &self, + _duty: &Duty, + ) -> pluto_core::deadline::Result>> { + Ok(Some( + chrono::Utc::now() + .checked_add_signed(chrono::Duration::hours(1)) + .expect("one hour ahead fits DateTime"), + )) + } +} + +/// Builds a `Consensus` for `nodes` peers, wired the way +/// `crates/consensus/examples/qbft.rs`'s `build_consensus` does (see the +/// probe doc, section B.1). `DeadlinerHandle::always` turned out to be +/// `#[cfg(test)]`-gated inside pluto_core (not reachable from an external +/// crate, unlike what the probe doc's snippet showed), so this uses the +/// real public `DeadlinerTask::start` with `FutureDeadlineCalculator` +/// (see above) instead. +fn build_consensus(nodes: usize) -> Consensus { + let (deadliner, expired_rx) = DeadlinerTask::start( + CancellationToken::new(), + "qbft-msg-limits-test", + FutureDeadlineCalculator, + ); + Consensus::new(Config { + peers: build_peers(nodes), + local_peer_idx: 0, + privkey: secret_key(1), + deadliner, + expired_rx, + duty_gater: Arc::new(|_: &Duty| true), + broadcaster: Arc::new(|_, _| Box::pin(async { Ok(()) })), + sniffer: Arc::new(|_| {}), + compare_attestations: false, + timer_func: get_round_timer_func(Arc::new(FeatureSet::default())), + feature_set: Arc::new(FeatureSet::default()), + }) + .expect("consensus construction with a valid local_peer_idx must succeed") +} + +/// Reproduces pluto's own (crate-private) `qbft::msg::sign_msg`: clear the +/// signature, `hash_proto` the cleared message, sign the 32-byte root with +/// `pluto_k1util::sign`, reattach. `sign_msg` itself is `pub(crate)`; +/// `hash_proto` and `pluto_k1util::sign` are the public halves it is built +/// from (proven by Task 2's `qbft_hashing` and Task 1's +/// `secp256k1_signatures` suites respectively). +fn sign_qbft_msg(msg: &pbconsensus::QbftMsg, key: &SecretKey) -> pbconsensus::QbftMsg { + let mut clone = msg.clone(); + clone.signature = Default::default(); + let hash = hash_proto(&clone).expect("hash_proto on a QbftMsg must succeed"); + let signature = pluto_k1util::sign(key, &hash).expect("sign must succeed"); + clone.signature = signature.to_vec().into(); + clone +} + +fn duty() -> Duty { + Duty::new(SlotNumber::from(42u64), DutyType::Attester) +} + +/// A single, distinct, validly signed justification message for peer 0, +/// reused `justification_count` times per case -- exactly how pluto's own +/// internal `handle_rejects_too_many_justifications` / +/// `handle_accepts_max_justifications` tests build their fixtures +/// (`crates/consensus/src/qbft/component.rs`): the cap counts entries, not +/// distinct senders, so one repeated valid message is a faithful fixture. +fn signed_justification_msg() -> pbconsensus::QbftMsg { + let unsigned = pbconsensus::QbftMsg { + r#type: i64::from(MSG_ROUND_CHANGE), + duty: Some(pbcore::Duty::try_from(&duty()).expect("valid duty converts")), + peer_idx: 0, + round: 1, + prepared_round: 0, + ..Default::default() + }; + sign_qbft_msg(&unsigned, &secret_key(1)) +} + +fn signed_outer_msg() -> pbconsensus::QbftMsg { + let unsigned = pbconsensus::QbftMsg { + r#type: i64::from(MSG_PRE_PREPARE), + duty: Some(pbcore::Duty::try_from(&duty()).expect("valid duty converts")), + peer_idx: 0, + round: 1, + prepared_round: 0, + ..Default::default() + }; + sign_qbft_msg(&unsigned, &secret_key(1)) +} + +/// A minimal, decodable `values` entry. The outer message's `value_hash` is +/// left at its default (empty) in every case here, which `msg::Msg::new` +/// collapses to the nil hash requiring no lookup (confirmed in +/// `crates/consensus/src/qbft/msg.rs`'s `Msg::new` doc comment), so these +/// entries never need to match anything -- only to decode successfully via +/// `decode_supported_any`, which is all `values_by_hash` ever checks. +fn any_value(tag: u8) -> Any { + Any::from_msg(&pbcore::UnsignedDataSet { + set: [(format!("0x{tag:02x}"), vec![tag].into())].into(), + }) + .expect("pack UnsignedDataSet") +} + +fn build_consensus_msg( + justification_count: usize, + value_count: usize, +) -> pbconsensus::QbftConsensusMsg { + pbconsensus::QbftConsensusMsg { + msg: Some(signed_outer_msg()), + justification: std::iter::repeat_n(signed_justification_msg(), justification_count) + .collect(), + values: (0..value_count) + .map(|i| any_value(u8::try_from(i % 256).expect("masked to u8 range"))) + .collect(), + } +} + +#[tokio::test] +async fn counts() { + let suite = load_suite("qbft_msg_limits"); + let cases = suite["counts"].as_array().unwrap(); + assert_eq!(cases.len(), 11, "expected 11 counts cases"); + + let mut failures = Vec::new(); + let mut matched = 0usize; + let mut absent_ok = 0usize; + + for case in cases { + let name = case["name"].as_str().unwrap(); + let input = &case["input"]; + let nodes = input["nodes"].as_u64().unwrap() as usize; + let justification_count = input["justification_count"].as_u64().unwrap() as usize; + let value_count = input["value_count"].as_u64().unwrap() as usize; + let spec_accepted = case["accepted"].as_bool().unwrap(); + let spec_reason = case["reason"].as_str(); + + // Structural proof (not a name-list) that every counts case stays + // within pluto's own 4n cap, so pluto is expected to accept it + // unconditionally. If this ever breaks, the case needs re-triage + // rather than an automatic ABSENT-OK. + let pluto_max_justifications = PLUTO_MAX_JUSTIFICATIONS_PER_NODE * nodes; + assert!( + justification_count <= pluto_max_justifications, + "qbft_msg_limits/counts/{name}: fixture assumption broken -- \ + justification_count {justification_count} exceeds pluto's own cap \ + {pluto_max_justifications}; this needs re-triage, not an unconditional accept" + ); + + let consensus = build_consensus(nodes); + let msg = build_consensus_msg(justification_count, value_count); + + match consensus.handle(msg, &CancellationToken::new()).await { + Ok(()) => { + if spec_accepted { + matched += 1; + } else { + // ABSENT-OK: the spec would reject (reason spec_reason), + // pluto accepts because it has no equivalent check at + // this input size. Ladder entry: "QBFT DECIDED-resend + // rate limit and message size/count limits" + // (first_charon_release: null). + absent_ok += 1; + } + } + Err(err) => { + failures.push(format!( + "qbft_msg_limits/counts/{name}: pluto rejected ({err:?}) with \ + nodes={nodes} justification_count={justification_count} \ + value_count={value_count}; spec accepted={spec_accepted} \ + reason={spec_reason:?} -- unexpected over-rejection, no ladder \ + entry excuses this" + )); + } + } + } + + assert_eq!( + matched, 5, + "expected 5 cases where pluto's acceptance matches the spec" + ); + assert_eq!( + absent_ok, 6, + "expected 6 ABSENT-OK cases (pluto's 4n justification cap / absent values cap)" + ); + assert!(failures.is_empty(), "{}", failures.join("\n")); +} + +/// Pins pluto's real contract at its own boundary: exactly `4 * node_count()` +/// justifications must be accepted (the check is a strict `>`, per +/// `component.rs`). No vector case reaches this boundary (the largest, +/// `nodes=4`, only goes to `justification_count=9` against `4*4=16`), so +/// without this test pluto's actual cap is asserted only in a doc comment. +#[tokio::test] +async fn pluto_own_justification_cap_accepts_at_boundary() { + let nodes = 4; + let consensus = build_consensus(nodes); + let max = PLUTO_MAX_JUSTIFICATIONS_PER_NODE * nodes; + let msg = build_consensus_msg(max, 0); + + consensus + .handle(msg, &CancellationToken::new()) + .await + .expect("exactly 4*nodes justifications must be accepted (strict `>` comparison)"); +} + +/// Companion to the above: one past pluto's own boundary must be rejected +/// with `Error::TooManyJustifications`, and the reported `count`/`max` must +/// match exactly -- asserting the reason, not just that it failed. +#[tokio::test] +async fn pluto_own_justification_cap_rejects_one_over_boundary() { + let nodes = 4; + let consensus = build_consensus(nodes); + let max = PLUTO_MAX_JUSTIFICATIONS_PER_NODE * nodes; + let msg = build_consensus_msg(max + 1, 0); + + let err = consensus + .handle(msg, &CancellationToken::new()) + .await + .expect_err("4*nodes + 1 justifications must be rejected"); + + match err { + Error::TooManyJustifications { + count, + max: reported_max, + } => { + assert_eq!(count, max + 1); + assert_eq!(reported_max, max); + } + other => panic!("expected Error::TooManyJustifications, got {other:?}"), + } +} + +#[test] +fn wire_size_constants() { + // The 32 MiB consensus limit is a narrowing of libp2p's 128 MiB default; + // both constants are public in pluto. + assert_eq!( + pluto_consensus::qbft::p2p::MAX_CONSENSUS_MSG_SIZE, + 32 * 1024 * 1024 + ); + assert_eq!(pluto_p2p::proto::MAX_MESSAGE_SIZE, 128 << 20); +} + +/// LEB128 varint length-prefix, replicating pluto's own +/// `inbound_rejects_message_exceeding_max_consensus_size` test +/// (`crates/consensus/src/qbft/p2p.rs`). `read_length_delimited` rejects an +/// oversized declared length as soon as it reads this prefix, before ever +/// reading a body -- so oversized rejected cases below need only these few +/// bytes, never a multi-hundred-megabyte buffer. +fn varint_len_prefix(mut remaining: usize) -> Vec { + let mut varint = Vec::new(); + loop { + let mut byte = u8::try_from(remaining & 0x7f).expect("7-bit masked value fits in u8"); + remaining >>= 7; + if remaining != 0 { + byte |= 0x80; + } + varint.push(byte); + if remaining == 0 { + break; + } + } + varint +} + +/// Builds a `QbftConsensusMsg` whose encoded length is exactly `target_len` +/// bytes, by padding a single `values` entry's `Any.value`. Converges in a +/// handful of iterations: the only overhead is protobuf tag/length-prefix +/// bytes, which only change size at varint-width boundaries. +fn qbft_consensus_msg_of_encoded_len(target_len: usize) -> pbconsensus::QbftConsensusMsg { + let mut padding_len = target_len; + for _ in 0..16 { + let candidate = pbconsensus::QbftConsensusMsg { + msg: None, + justification: vec![], + values: vec![Any { + type_url: String::new(), + value: vec![0u8; padding_len], + }], + }; + let encoded_len = candidate.encoded_len(); + if encoded_len == target_len { + return candidate; + } + let diff = i64::try_from(encoded_len).expect("size fits i64") + - i64::try_from(target_len).expect("size fits i64"); + padding_len = usize::try_from(i64::try_from(padding_len).expect("size fits i64") - diff) + .expect("padding length stays non-negative while sizing the test message"); + } + panic!("qbft_consensus_msg_of_encoded_len({target_len}) did not converge"); +} + +/// Mirrors how `crates/consensus/src/qbft/p2p.rs`'s own tests +/// (`reference_framed_message_decodes`, +/// `inbound_rejects_message_exceeding_max_consensus_size`) invoke +/// `read_protobuf_with_max_size` with `MAX_CONSENSUS_MSG_SIZE`, over the same +/// `futures::io::Cursor` in-memory stream type. +#[tokio::test] +async fn wire_size_enforcement() { + let suite = load_suite("qbft_msg_limits"); + let cases = suite["wire_size"].as_array().unwrap(); + assert_eq!(cases.len(), 4, "expected 4 wire_size cases"); + + let mut failures = Vec::new(); + for case in cases { + let name = case["name"].as_str().unwrap(); + let wire_size_bytes = case["input"]["wire_size_bytes"].as_u64().unwrap() as usize; + let accepted = case["accepted"].as_bool().unwrap(); + + if accepted { + let msg = qbft_consensus_msg_of_encoded_len(wire_size_bytes); + assert_eq!( + msg.encoded_len(), + wire_size_bytes, + "qbft_msg_limits/wire_size/{name}: test fixture sizing bug" + ); + + let mut cursor = Cursor::new(Vec::new()); + pluto_p2p::proto::write_protobuf(&mut cursor, &msg) + .await + .expect("frame write should succeed"); + cursor.set_position(0); + + match pluto_p2p::proto::read_protobuf_with_max_size::( + &mut cursor, + pluto_consensus::qbft::p2p::MAX_CONSENSUS_MSG_SIZE, + ) + .await + { + Ok(decoded) if decoded == msg => {} + Ok(_) => failures.push(format!( + "qbft_msg_limits/wire_size/{name}: decoded message did not round-trip" + )), + Err(err) => failures.push(format!( + "qbft_msg_limits/wire_size/{name}: expected accept at {wire_size_bytes} \ + bytes, got {err}" + )), + } + } else { + let mut cursor = Cursor::new(varint_len_prefix(wire_size_bytes)); + + match pluto_p2p::proto::read_protobuf_with_max_size::( + &mut cursor, + pluto_consensus::qbft::p2p::MAX_CONSENSUS_MSG_SIZE, + ) + .await + { + Ok(_) => failures.push(format!( + "qbft_msg_limits/wire_size/{name}: expected reject at {wire_size_bytes} \ + bytes, got Ok" + )), + Err(err) => { + // Assert the reason ("msg_too_large"), not just that it + // failed: `read_length_delimited` reports oversized + // frames with this exact wording. + let text = err.to_string(); + if !text.contains("too large") { + failures.push(format!( + "qbft_msg_limits/wire_size/{name}: rejected for the wrong reason: \ + {text} (want a \"too large\" message-size error)" + )); + } + } + } + } + } + + assert!(failures.is_empty(), "{}", failures.join("\n")); +} diff --git a/plans/pluto-conformance.md b/plans/pluto-conformance.md index 06eb5fe..7d9a790 100644 --- a/plans/pluto-conformance.md +++ b/plans/pluto-conformance.md @@ -73,7 +73,7 @@ pluto commit actually tested. Statuses: `todo` / `in progress` / `done`. | 4 | Cluster hashes + lock verification | `cluster_hashing` | done | FAIL (2 pinned known-divergence cases; at least 3 distinct affected fields — a lower bound, not a total) | `definition` 5/6 strict-PASS (`config_hash` and `definition_hash` both match on 5 cases, including the unsigned/signed pair proving config_hash is signature-independent); `all_empty_lists` is a real divergence, pinned (see Findings) — it carries **at least two independent parse blockers** (`operators`/`validators` null-rejection, then a masked `timestamp` missing-field rejection once the first is hypothetically fixed), so a partial upstream fix will not make it pass. `lock` 3/4 strict-PASS (`lock_hash` matches and `verify_hashes` succeeds on 3); `validator_without_deposit_data` is a second real divergence, pinned (see Findings). `real_keys_3_of_4`: hashes verify; the full `verify_signatures(&EthClient::new(""))` chain is asserted to fail specifically at the definition EIP-712 stage (`LockError::DefinitionSignaturesVerificationFailed`) because its operator/creator signatures are unavoidable placeholders (charon's EIP-712 signing helpers aren't exported for vector generation) — `Lock::verify_signatures` short-circuits there, so its private BLS-aggregate and node-signature checks are never reached from this external crate; that half is a coverage gap (not a failure), noted in the test's doc comment rather than mocked around. | | 5 | Priority scoring | `priority_scoring` | done | PASS | All 18/18 cases PASS end-to-end through the public `Prioritiser` API over an in-process libp2p (`MemoryTransport` + noise + yamux) network, with a mock `Consensus` capturing the proposed `PriorityResult` (`calculate_result` itself is `pub(crate)`, unreachable directly). Covers below-quorum rejection (empty-result pins), single-priority and two-priority scoring, count-then-relative-priority ordering, and the two `test-case`-name-only "deterministic ordering" cases (score ties resolved by first-seen order in peer-id-sorted input — confirmed stable, no divergence from Charon v1.7.1's own stable-sort assumption). Peer `PeerId`s are generated then sorted ascending and mapped onto the vector's peer indices, since scoring tie-breaks are first-seen over peer-id-sorted input. Also pins `PROTOCOL_ID`/`protocols()` to the legacy slash-less `charon/priority/2.0.0` (ladder: preferred slash form is unreleased). Full 18-case suite runs in under 1s (well under the 2-minute budget). | | 6 | Round timer deadlines | `timer_deadlines` | done | round_timeout: PASS **only under `ProposalTimeout` enabled**, a documented non-default precondition (pluto's *default* config has a pinned, real divergence on `PROPOSER`/round-1 cases — see Notes and Findings); deadline: ABSENT-OK, with a real code-level slot-invariance pin (not just a comment); duty_start_delay: UNREACHABLE | `round_timeouts_with_proposal_timeout_enabled`: 216/216 PASS, but only with `Feature::ProposalTimeout` explicitly enabled (it is `Status::Alpha`, off under pluto's `Config::default()`). `round_timeouts_pluto_default_feature_set` runs the same 216 cases under pluto's actual default `FeatureSet` and pins the resulting divergence: the 207 non-`PROPOSER`-round-1 cases still match the vectors, but all 9 `PROPOSER`/round-1 cases (3 slots x 3 slot durations) diverge — pluto's default gives 1s, the vectors (which assume `ProposalTimeout` enabled) expect 1.5s. This is a real, pinned known-divergence (see Findings), not swept under the enabled-feature test's PASS. `deadline_nanos`: `round_timeout_is_duty_slot_invariant` builds two timers per (duty_type, round) combination (24 total, covering all cases) differing only in `slot` (0 vs. 7231) and asserts bit-identical measured timeouts — a real code pin for "no timer reads slot", not a doc comment, so it fails loudly if `with_duty`/`RoundTimer::timer` ever grows slot-dependence; genesis-invariance is structurally guaranteed (the API has no genesis parameter at all) and is stated as such rather than tested. Matches ladder entry "Deterministic (genesis-derived) eager double linear round deadlines" (`first_charon_release: v1.9.0` > pluto's v1.7.1 anchor). `duty_start_delay_nanos` is UNREACHABLE: `delay_slot_offset` (`crates/core/src/scheduler.rs`) is a non-`pub` `async fn` with no external caller, confirmed still private. | -| 7 | QBFT message limits | `qbft_msg_limits` | todo | — | | +| 7 | QBFT message limits | `qbft_msg_limits` | done | ABSENT-OK (counts, both directions); PASS (wire_size) | `counts` (11/11 cases run): pluto has neither the spec's `2n` justification cap nor any `2*(j+1)` values cap. Its own cap is `4 * node_count()` (component.rs `MAX_JUSTIFICATIONS_PER_NODE = 4`, strict `>`), and there is no values-length check anywhere in `crates/consensus/src`/`crates/core/src`. Every counts case's `justification_count` stays at/under pluto's `4n` (proven structurally in the test, not by case name), so pluto accepts all 11 cases: 5 match the spec's `accepted: true` cases outright, 6 are ABSENT-OK divergences (spec says reject — `too_many_justifications` or `too_many_values` — pluto accepts) against ladder entry "QBFT DECIDED-resend rate limit and message size/count limits" (`first_charon_release: null`). Two extra tests pin pluto's own real boundary directly, since no vector case reaches it: `4 * nodes` justifications accepted, `4 * nodes + 1` rejected with `Error::TooManyJustifications { count, max }` asserted exactly (not just "any error"). `wire_size` (4/4 cases): `MAX_CONSENSUS_MSG_SIZE` (32 MiB) and `MAX_MESSAGE_SIZE` (128 MiB) constants pinned; enforcement runs the real `pluto_p2p::proto::read_protobuf_with_max_size` reader (mirroring `qbft/p2p.rs`'s own unit tests) over `futures::io::Cursor` — 1024 bytes and exactly 32 MiB accepted (round-trip decode verified byte-for-byte), 32 MiB+1 and 128 MiB rejected with a "too large" reason, matching `msg_too_large` exactly. `DeadlinerHandle::always` from the probe doc turned out to be `#[cfg(test)]`-gated inside pluto_core (not reachable externally); worked around with the real public `DeadlinerTask::start` and a custom future-deadline `DeadlineCalculator` (pluto's own public `NeverExpiringCalculator` returns `NoDeadline`, which `handle`'s strict `!= Scheduled` check treats as `Error::DutyExpired` — a real gotcha, not a doc typo). See Findings for the `4n` cap as its own hardening decision. | | 8 | DECIDED-resend limiting | `qbft_decided_resends` | todo | — | | | 9 | Sender binding + peer map | `parsigex_sender_binding` | todo | — | | | 10 | Coverage guard + docs | all | todo | — | | @@ -216,6 +216,41 @@ whether a ladder entry covers it, and whether it was reported upstream._ `tests/test_release.py`, so the entry and the README update need to land together, in their own commit — not folded into this conformance task. +- **Pluto's justification-count cap is `4n`, not the spec's `2n` (task 7, + `qbft_msg_limits/counts`).** `Consensus::handle` + (`crates/consensus/src/qbft/component.rs`) enforces + `MAX_JUSTIFICATIONS_PER_NODE.saturating_mul(node_count())` with + `MAX_JUSTIFICATIONS_PER_NODE = 4`, checked with a strict `>` (a count exactly + equal to `4n` is accepted). This is pluto's own unilateral defensive + hardening — its doc comment states it bounds secp256k1-recovery work against + a maliciously large message, independent of charon, which the same doc + comment notes has no explicit cap at all. It is neither charon v1.7.1's + absence (which would be no cap) nor the spec's `2n` formula; it happens to + sit strictly looser than the spec everywhere the vectors probe it (every + `justification_count` between the spec's `2n+1` and pluto's `4n` is accepted + by pluto, rejected by the spec). Pinned directly in code (not just this + note): `pluto_own_justification_cap_accepts_at_boundary` / + `_rejects_one_over_boundary` assert `4 * nodes` accepted and `4 * nodes + 1` + rejected with `Error::TooManyJustifications { count, max }` matching + exactly. Not reported upstream — this is pluto's own choice, not a defect, + though it is worth pluto documenting the `4n` constant against the spec's + `2n` explicitly rather than only against "no cap" charon. + +- **Pluto has no cap at all on `QbftConsensusMsg.values` length (task 7, + `qbft_msg_limits/counts`).** Confirmed by exhaustive grep of + `crates/consensus/src` and `crates/core/src`: `values_by_hash` + (`component.rs`) iterates the incoming `Vec` and inserts each into a + `HashMap` with no length bound anywhere, unlike the explicit + `MAX_JUSTIFICATIONS_PER_NODE` cap on justifications. The spec's `2*(j+1)` + values cap (`charon_anchor.json` ladder entry "QBFT DECIDED-resend rate + limit and message size/count limits", `first_charon_release: null` — absent + from every released charon) is therefore an ABSENT-OK divergence at every + vector case that exercises it (`no_justifications_three_values`, + `values_one_over_limit_for_full_justification_set`, + `both_limits_exceeded_reports_justifications`): the spec rejects, pluto + accepts. Not reported upstream — matches charon's own absence at the pinned + anchor. + ## Global Constraints - Pluto is **never modified**. The harness lives in `consumers/rust/` and path-depends From 07bd200c1383bd64da42de4465d0de23e33934d9 Mon Sep 17 00:00:00 2001 From: Andrei Smirnov Date: Fri, 31 Jul 2026 23:06:09 +0300 Subject: [PATCH 14/27] Run qbft_decided_resends suite against pluto --- consumers/rust/Cargo.toml | 2 + consumers/rust/tests/qbft_decided_resends.rs | 380 +++++++++++++++++++ plans/pluto-conformance.md | 25 +- 3 files changed, 406 insertions(+), 1 deletion(-) create mode 100644 consumers/rust/tests/qbft_decided_resends.rs diff --git a/consumers/rust/Cargo.toml b/consumers/rust/Cargo.toml index fa7ad37..1d8cd2d 100644 --- a/consumers/rust/Cargo.toml +++ b/consumers/rust/Cargo.toml @@ -10,6 +10,8 @@ serde_json = "1.0" hex = "0.4.3" [dev-dependencies] +crossbeam = "0.8.4" +cancellation = "0.1.0" pluto-k1util = { path = "../../../pluto/crates/k1util" } pluto-core = { path = "../../../pluto/crates/core" } pluto-consensus = { path = "../../../pluto/crates/consensus" } diff --git a/consumers/rust/tests/qbft_decided_resends.rs b/consumers/rust/tests/qbft_decided_resends.rs new file mode 100644 index 0000000..3ceabc2 --- /dev/null +++ b/consumers/rust/tests/qbft_decided_resends.rs @@ -0,0 +1,380 @@ +//! Runs the `qbft_decided_resends` suite directly against pluto's core QBFT +//! state machine (`pluto_core::qbft::run`), one level below the `Consensus` +//! wrapper Task 7 (`qbft_msg_limits`) exercises. +//! +//! The vectors describe the spec's post-decision rebroadcast limiter: once an +//! instance has decided, a ROUND-CHANGE should trigger at most one DECIDED +//! rebroadcast per source per strictly-increasing round, capped at 16 per +//! source. Pluto implements no such limiter -- confirmed here by driving a +//! real instance to DECIDED and observing its behaviour, not by code +//! inspection alone (`crates/core/src/qbft/mod.rs`'s main `recv(t.receive)` +//! arm, inside the `if let Some(v) = q_commit.as_ref() ...` branch): every +//! inbound `MSG_ROUND_CHANGE` from a source other than the deciding process +//! rebroadcasts DECIDED, unconditionally, regardless of round or repetition. +//! Ladder entry "QBFT DECIDED-resend rate limit and message size/count +//! limits" has `first_charon_release: null`, so this gap is ABSENT-OK: no +//! released charon carries the limiter either, at pluto's charon-v1.7.1 +//! anchor. +//! +//! Two readings are checked per event, per the task brief: +//! - **pin reading** (asserted, must pass): pluto rebroadcasts DECIDED on +//! every post-decision ROUND-CHANGE from another source. This is what +//! actually happens today; if pluto ever adds the limiter, some of these +//! assertions will start failing, which is the point -- the test is +//! supposed to flip loudly rather than silently keep passing. +//! - **spec reading** (reported, not asserted): the vector's `rebroadcast` +//! array, which encodes the limiter pluto lacks. Mismatches are counted and +//! printed per case; they are expected once a source repeats/lowers a +//! round or exceeds 16 rebroadcasts. + +use std::sync::Arc; +use std::thread; +use std::time::Duration; + +use cancellation::CancellationTokenSource; +use crossbeam::channel as mpmc; + +use pluto_core::qbft::{ + self, BroadcastRequest, CompareRequest, DecideRequest, Definition, LeaderRequest, MSG_COMMIT, + MSG_DECIDED, MSG_PRE_PREPARE, MSG_PREPARE, MSG_ROUND_CHANGE, MessageType, Msg, QbftError, + QbftLogger, QbftTypes, RoundChangeLog, SomeMsg, Timer, Transport, UnjustLog, UponRuleLog, +}; +use spec_vectors_pluto::load_suite; + +/// Local process under test. Fixed across all cases; never the leader and +/// never one of the vector's event sources, so it can only ever be on the +/// receiving end of a rebroadcast decision. +const PROCESS: i64 = 0; +const VALUE: i64 = 7; +/// Generous but bounded: pluto's rebroadcast happens synchronously in the +/// same message-processing loop, so a real rebroadcast arrives near- +/// instantly. This only gets exhausted if pluto stops rebroadcasting, which +/// is itself a (loud, intended) test failure. +const RECV_TIMEOUT: Duration = Duration::from_secs(2); + +struct TestTypes; + +impl QbftTypes for TestTypes { + type Instance = i64; + type Value = i64; + type Compare = i64; +} + +/// Minimal external `SomeMsg` implementation, mirroring pluto's own +/// `internal_test.rs::TestMsg` (`crates/core/src/qbft/internal_test.rs`). +/// None of the post-decision ROUND-CHANGE events need real justification -- +/// confirmed by reading `mod.rs`: once `q_commit` is set, the early-return +/// branch never calls `is_justified`, so `prepared_round`/`prepared_value`/ +/// `justification` go unused for those messages. +#[derive(Debug, Clone)] +struct TestMsg { + kind: MessageType, + source: i64, + round: i64, + value: i64, +} + +impl TestMsg { + fn pre_prepare(source: i64, round: i64, value: i64) -> Msg { + Arc::new(TestMsg { + kind: MSG_PRE_PREPARE, + source, + round, + value, + }) + } + + fn prepare(source: i64, round: i64, value: i64) -> Msg { + Arc::new(TestMsg { + kind: MSG_PREPARE, + source, + round, + value, + }) + } + + fn commit(source: i64, round: i64, value: i64) -> Msg { + Arc::new(TestMsg { + kind: MSG_COMMIT, + source, + round, + value, + }) + } + + fn round_change(source: i64, round: i64) -> Msg { + Arc::new(TestMsg { + kind: MSG_ROUND_CHANGE, + source, + round, + value: 0, + }) + } +} + +impl SomeMsg for TestMsg { + fn type_(&self) -> MessageType { + self.kind + } + + fn instance(&self) -> i64 { + 0 + } + + fn source(&self) -> i64 { + self.source + } + + fn round(&self) -> i64 { + self.round + } + + fn value(&self) -> i64 { + self.value + } + + fn value_source(&self) -> std::result::Result { + Ok(0) + } + + fn prepared_round(&self) -> i64 { + 0 + } + + fn prepared_value(&self) -> i64 { + 0 + } + + fn justification(&self) -> Vec> { + vec![] + } + + fn as_any(&self) -> &dyn std::any::Any { + self + } +} + +/// Runs a `qbft::run` instance on a background thread, feeds it the minimal +/// justified message sequence pluto's own `broadcast_request_maps_protocol_fields` +/// test uses (`crates/core/src/qbft/internal_test.rs`) to reach quorum +/// PREPARE and quorum COMMIT at round 1, then confirms decision was actually +/// reached before handing control back -- so a harness bug (failure to +/// decide) can never be mistaken for a conformance result. +struct Harness { + receive_tx: mpmc::Sender>, + decided_rx: mpmc::Receiver<()>, + cts: CancellationTokenSource, + join: thread::JoinHandle>, + // Kept alive only so `run`'s `input_value_ch`/`input_value_source_ch` + // channels stay open (a disconnected channel would make `run` error out + // via its `mpmc::select!` arm). Never written to. + _input_tx: mpmc::Sender, + _source_tx: mpmc::Sender, +} + +impl Harness { + fn shutdown(self) { + self.cts.cancel(); + let outcome = self.join.join().expect("qbft::run thread must not panic"); + assert!( + matches!(outcome, Err(QbftError::ContextCanceled)), + "expected ContextCanceled on shutdown, got {outcome:?}" + ); + } +} + +fn spawn_decided_instance(nodes: i64, decided_round: i64) -> Harness { + assert_eq!( + decided_round, 1, + "harness only drives a round-1 decision (the minimal justified sequence from \ + pluto's own internal_test.rs); case has decided_round={decided_round}, which needs \ + a round-change dance this harness does not implement" + ); + assert!( + nodes >= 2, + "need at least one external process to act as leader; got nodes={nodes}" + ); + + let others: Vec = (0..nodes).filter(|&p| p != PROCESS).collect(); + let need = usize::try_from(qbft::quorum(nodes)).expect("quorum fits usize"); + assert!( + others.len() >= need, + "not enough external processes ({}) to reach quorum ({need}) for nodes={nodes}", + others.len() + ); + let leader = others[0]; + + let (receive_tx, receive_rx) = mpmc::unbounded::>(); + let (decided_tx, decided_rx) = mpmc::unbounded::<()>(); + let (_input_tx, input_rx) = mpmc::bounded::(1); + let (_source_tx, source_rx) = mpmc::bounded::(1); + + let cts = CancellationTokenSource::new(); + let token = cts.token().clone(); + + let def = Definition:: { + is_leader: Box::new(move |req: LeaderRequest<'_, TestTypes>| req.process == leader), + new_timer: Box::new(|_round: i64| Timer { + // Never fires: round timeouts must not inject unscripted events. + receive: mpmc::never(), + stop: Box::new(|| {}), + }), + compare: Arc::new(|req: CompareRequest<'_, TestTypes>| { + req.return_err + .send(Ok(())) + .expect("compare return channel open"); + }), + decide: Box::new(|_req: DecideRequest<'_, TestTypes>| {}), + logger: QbftLogger { + upon_rule: Box::new(|_: UponRuleLog<'_, TestTypes>| {}), + round_change: Box::new(|_: RoundChangeLog<'_, TestTypes>| {}), + unjust: Box::new(|_: UnjustLog<'_, TestTypes>| {}), + }, + nodes, + fifo_limit: 100, + }; + + let transport = Transport:: { + broadcast: Box::new(move |req: BroadcastRequest<'_, TestTypes>| { + if req.type_ == MSG_DECIDED { + decided_tx.send(()).expect("decided channel open"); + } + Ok(()) + }), + receive: receive_rx, + }; + + let join = thread::spawn(move || { + qbft::run( + &token, &def, &transport, &0i64, PROCESS, input_rx, source_rx, + ) + }); + + // Minimal justified sequence to reach DECIDED at round 1: 1 PRE-PREPARE + // from the leader, a quorum of PREPAREs, then a quorum of COMMITs + // (mirrors `broadcast_request_maps_protocol_fields`, + // `crates/core/src/qbft/internal_test.rs`). + receive_tx + .send(TestMsg::pre_prepare(leader, 1, VALUE)) + .expect("receive channel open"); + for &source in others.iter().take(need) { + receive_tx + .send(TestMsg::prepare(source, 1, VALUE)) + .expect("receive channel open"); + } + for &source in others.iter().take(need) { + receive_tx + .send(TestMsg::commit(source, 1, VALUE)) + .expect("receive channel open"); + } + + // Warm-up: confirm the instance actually decided before measuring + // anything. A round-change from the leader at an arbitrary round is not + // part of any vector's event sequence, so consuming its rebroadcast here + // cannot skew the measured events below. + receive_tx + .send(TestMsg::round_change(leader, 999)) + .expect("receive channel open"); + decided_rx.recv_timeout(RECV_TIMEOUT).unwrap_or_else(|_| { + panic!( + "harness failed to reach DECIDED: no DECIDED rebroadcast observed for the warm-up \ + ROUND-CHANGE within {RECV_TIMEOUT:?}" + ) + }); + + Harness { + receive_tx, + decided_rx, + cts, + join, + _input_tx, + _source_tx, + } +} + +#[test] +fn qbft_decided_resends() { + let suite = load_suite("qbft_decided_resends"); + let cases = suite["cases"].as_array().unwrap(); + assert_eq!(cases.len(), 2, "expected 2 qbft_decided_resends cases"); + + let mut pin_failures = Vec::new(); + let mut report = String::new(); + + for case in cases { + let name = case["name"].as_str().unwrap(); + let input = &case["input"]; + let nodes = input["nodes"].as_i64().unwrap(); + let decided_round = input["decided_round"].as_i64().unwrap(); + let events: Vec<(i64, i64)> = input["events"] + .as_array() + .unwrap() + .iter() + .map(|e| (e["source"].as_i64().unwrap(), e["round"].as_i64().unwrap())) + .collect(); + let spec_rebroadcast: Vec = case["rebroadcast"] + .as_array() + .unwrap() + .iter() + .map(|v| v.as_bool().unwrap()) + .collect(); + let spec_total = case["total_rebroadcasts"].as_u64().unwrap() as usize; + + assert_eq!( + events.len(), + spec_rebroadcast.len(), + "qbft_decided_resends/{name}: events/rebroadcast length mismatch" + ); + assert_eq!( + spec_rebroadcast.iter().filter(|&&b| b).count(), + spec_total, + "qbft_decided_resends/{name}: vector's total_rebroadcasts disagrees with its own \ + rebroadcast array" + ); + + let harness = spawn_decided_instance(nodes, decided_round); + + let mut actual = Vec::with_capacity(events.len()); + for &(source, round) in &events { + harness + .receive_tx + .send(TestMsg::round_change(source, round)) + .expect("receive channel open"); + actual.push(harness.decided_rx.recv_timeout(RECV_TIMEOUT).is_ok()); + } + + for (i, (&got, &(source, round))) in actual.iter().zip(events.iter()).enumerate() { + // Pin reading: pluto rebroadcasts DECIDED on every post-decision + // ROUND-CHANGE from a source other than the deciding process -- + // no per-source cap, no round-monotonicity check. Every event + // here has source != PROCESS, so every one must rebroadcast. If + // pluto ever adds the ladder's limiter, some entries will flip + // to `false` and this assertion fails loudly, as intended. + if !got { + pin_failures.push(format!( + "qbft_decided_resends/{name}: event {i} (source={source}, round={round}) \ + did not trigger a DECIDED rebroadcast; pluto's pinned no-limiter rule \ + expects every post-decision ROUND-CHANGE from another source to rebroadcast" + )); + } + } + + let spec_mismatches = actual + .iter() + .zip(spec_rebroadcast.iter()) + .filter(|(a, s)| a != s) + .count(); + report.push_str(&format!( + "qbft_decided_resends/{name}: {spec_mismatches}/{} events diverge from the spec \ + reading (ABSENT-OK -- pluto has no resend rate/round limiter; ladder entry \ + \"QBFT DECIDED-resend rate limit and message size/count limits\", \ + first_charon_release: null)\n", + events.len() + )); + + harness.shutdown(); + } + + println!("{report}"); + assert!(pin_failures.is_empty(), "{}", pin_failures.join("\n")); +} diff --git a/plans/pluto-conformance.md b/plans/pluto-conformance.md index 7d9a790..bfe8407 100644 --- a/plans/pluto-conformance.md +++ b/plans/pluto-conformance.md @@ -74,7 +74,7 @@ pluto commit actually tested. Statuses: `todo` / `in progress` / `done`. | 5 | Priority scoring | `priority_scoring` | done | PASS | All 18/18 cases PASS end-to-end through the public `Prioritiser` API over an in-process libp2p (`MemoryTransport` + noise + yamux) network, with a mock `Consensus` capturing the proposed `PriorityResult` (`calculate_result` itself is `pub(crate)`, unreachable directly). Covers below-quorum rejection (empty-result pins), single-priority and two-priority scoring, count-then-relative-priority ordering, and the two `test-case`-name-only "deterministic ordering" cases (score ties resolved by first-seen order in peer-id-sorted input — confirmed stable, no divergence from Charon v1.7.1's own stable-sort assumption). Peer `PeerId`s are generated then sorted ascending and mapped onto the vector's peer indices, since scoring tie-breaks are first-seen over peer-id-sorted input. Also pins `PROTOCOL_ID`/`protocols()` to the legacy slash-less `charon/priority/2.0.0` (ladder: preferred slash form is unreleased). Full 18-case suite runs in under 1s (well under the 2-minute budget). | | 6 | Round timer deadlines | `timer_deadlines` | done | round_timeout: PASS **only under `ProposalTimeout` enabled**, a documented non-default precondition (pluto's *default* config has a pinned, real divergence on `PROPOSER`/round-1 cases — see Notes and Findings); deadline: ABSENT-OK, with a real code-level slot-invariance pin (not just a comment); duty_start_delay: UNREACHABLE | `round_timeouts_with_proposal_timeout_enabled`: 216/216 PASS, but only with `Feature::ProposalTimeout` explicitly enabled (it is `Status::Alpha`, off under pluto's `Config::default()`). `round_timeouts_pluto_default_feature_set` runs the same 216 cases under pluto's actual default `FeatureSet` and pins the resulting divergence: the 207 non-`PROPOSER`-round-1 cases still match the vectors, but all 9 `PROPOSER`/round-1 cases (3 slots x 3 slot durations) diverge — pluto's default gives 1s, the vectors (which assume `ProposalTimeout` enabled) expect 1.5s. This is a real, pinned known-divergence (see Findings), not swept under the enabled-feature test's PASS. `deadline_nanos`: `round_timeout_is_duty_slot_invariant` builds two timers per (duty_type, round) combination (24 total, covering all cases) differing only in `slot` (0 vs. 7231) and asserts bit-identical measured timeouts — a real code pin for "no timer reads slot", not a doc comment, so it fails loudly if `with_duty`/`RoundTimer::timer` ever grows slot-dependence; genesis-invariance is structurally guaranteed (the API has no genesis parameter at all) and is stated as such rather than tested. Matches ladder entry "Deterministic (genesis-derived) eager double linear round deadlines" (`first_charon_release: v1.9.0` > pluto's v1.7.1 anchor). `duty_start_delay_nanos` is UNREACHABLE: `delay_slot_offset` (`crates/core/src/scheduler.rs`) is a non-`pub` `async fn` with no external caller, confirmed still private. | | 7 | QBFT message limits | `qbft_msg_limits` | done | ABSENT-OK (counts, both directions); PASS (wire_size) | `counts` (11/11 cases run): pluto has neither the spec's `2n` justification cap nor any `2*(j+1)` values cap. Its own cap is `4 * node_count()` (component.rs `MAX_JUSTIFICATIONS_PER_NODE = 4`, strict `>`), and there is no values-length check anywhere in `crates/consensus/src`/`crates/core/src`. Every counts case's `justification_count` stays at/under pluto's `4n` (proven structurally in the test, not by case name), so pluto accepts all 11 cases: 5 match the spec's `accepted: true` cases outright, 6 are ABSENT-OK divergences (spec says reject — `too_many_justifications` or `too_many_values` — pluto accepts) against ladder entry "QBFT DECIDED-resend rate limit and message size/count limits" (`first_charon_release: null`). Two extra tests pin pluto's own real boundary directly, since no vector case reaches it: `4 * nodes` justifications accepted, `4 * nodes + 1` rejected with `Error::TooManyJustifications { count, max }` asserted exactly (not just "any error"). `wire_size` (4/4 cases): `MAX_CONSENSUS_MSG_SIZE` (32 MiB) and `MAX_MESSAGE_SIZE` (128 MiB) constants pinned; enforcement runs the real `pluto_p2p::proto::read_protobuf_with_max_size` reader (mirroring `qbft/p2p.rs`'s own unit tests) over `futures::io::Cursor` — 1024 bytes and exactly 32 MiB accepted (round-trip decode verified byte-for-byte), 32 MiB+1 and 128 MiB rejected with a "too large" reason, matching `msg_too_large` exactly. `DeadlinerHandle::always` from the probe doc turned out to be `#[cfg(test)]`-gated inside pluto_core (not reachable externally); worked around with the real public `DeadlinerTask::start` and a custom future-deadline `DeadlineCalculator` (pluto's own public `NeverExpiringCalculator` returns `NoDeadline`, which `handle`'s strict `!= Scheduled` check treats as `Error::DutyExpired` — a real gotcha, not a doc typo). See Findings for the `4n` cap as its own hardening decision. | -| 8 | DECIDED-resend limiting | `qbft_decided_resends` | todo | — | | +| 8 | DECIDED-resend limiting | `qbft_decided_resends` | done | ABSENT-OK (pin reading PASS; spec-reading mismatches reported per case) | Both cases run against the real `pluto_core::qbft::run` state machine (not code inspection alone), driven to DECIDED at round 1 via the minimal justified PRE-PREPARE + quorum PREPARE + quorum COMMIT sequence, then fed each case's ROUND-CHANGE event sequence one at a time. Pin reading (pluto's actual, unconditional rule — rebroadcast DECIDED on every post-decision ROUND-CHANGE from a source other than the deciding process, no cap, no round check) holds for all 28 events across both cases: PASS. Spec reading (the vector's per-event `rebroadcast` expectations, which encode a 16-per-source cap plus a strictly-increasing-round-per-source dedup) diverges on 4/7 events in `dedup_duplicates_and_stale_rounds` (the duplicate-round and stale-round entries) and 5/21 in `resend_cap_per_source` (the 17th-through-21st events, past pluto's absent 16-cap) — reported, not asserted, matching ladder entry "QBFT DECIDED-resend rate limit and message size/count limits" (`first_charon_release: null`). `SomeMsg`/`Definition`/`Transport`/`QbftLogger` were externally implementable exactly as the probe doc claimed, confirming `run` is fully reachable from an external crate. | | 9 | Sender binding + peer map | `parsigex_sender_binding` | todo | — | | | 10 | Coverage guard + docs | all | todo | — | | @@ -251,6 +251,29 @@ whether a ladder entry covers it, and whether it was reported upstream._ accepts. Not reported upstream — matches charon's own absence at the pinned anchor. +- **Pluto has no post-decision DECIDED-rebroadcast limiter at all (task 8, + `qbft_decided_resends`).** Confirmed by running the real state machine, not + just reading it: `crates/core/src/qbft/mod.rs`'s main `recv(t.receive)` arm, + inside the `if let Some(v) = q_commit.as_ref() && !v.is_empty()` branch — + once an instance has decided, *any* inbound `MSG_ROUND_CHANGE` from a source + other than the deciding process triggers an immediate `MSG_DECIDED` + rebroadcast, unconditionally. There is no per-source cap (the spec expects + 16), no strictly-increasing-round requirement (the spec expects stale or + repeated rounds from the same source to be ignored after the first), and + `is_justified` is not even called on these messages (the branch `continue`s + before reaching it). Driving a real instance to DECIDED (round-1 quorum + PRE-PREPARE/PREPARE/COMMIT dance) and feeding both vectors' event sequences + confirmed this exactly: `dedup_duplicates_and_stale_rounds` mismatches the + spec's expected `rebroadcast` array on 4/7 events (the repeated- and + stale-round entries, which the spec says should not re-trigger but pluto + rebroadcasts anyway); `resend_cap_per_source` mismatches on 5/21 (events 17 + through 21 for the same source, past the spec's 16-cap, which pluto has no + concept of). Ladder entry "QBFT DECIDED-resend rate limit and message + size/count limits" has `first_charon_release: null` — absent from every + released charon — so this is ABSENT-OK at pluto's charon-v1.7.1 anchor, the + same ladder entry task 7 matches for the justification/values caps. Not + reported upstream — matches charon's own absence at the pinned anchor. + ## Global Constraints - Pluto is **never modified**. The harness lives in `consumers/rust/` and path-depends From 5313af752676d02ab025a9ff8443136d230fd219 Mon Sep 17 00:00:00 2001 From: Andrei Smirnov Date: Fri, 31 Jul 2026 23:33:08 +0300 Subject: [PATCH 15/27] Run parsigex_sender_binding suite against pluto --- consumers/rust/Cargo.toml | 5 + .../rust/tests/parsigex_sender_binding.rs | 352 ++++++++++++++++++ plans/pluto-conformance.md | 45 ++- 3 files changed, 401 insertions(+), 1 deletion(-) create mode 100644 consumers/rust/tests/parsigex_sender_binding.rs diff --git a/consumers/rust/Cargo.toml b/consumers/rust/Cargo.toml index 1d8cd2d..4c91c68 100644 --- a/consumers/rust/Cargo.toml +++ b/consumers/rust/Cargo.toml @@ -21,6 +21,11 @@ pluto-eth1wrap = { path = "../../../pluto/crates/eth1wrap" } pluto-priority = { path = "../../../pluto/crates/priority" } pluto-p2p = { path = "../../../pluto/crates/p2p" } pluto-featureset = { path = "../../../pluto/crates/featureset" } +pluto-parsigex = { path = "../../../pluto/crates/parsigex" } +pluto-eth2api = { path = "../../../pluto/crates/eth2api" } +pluto-eth2util = { path = "../../../pluto/crates/eth2util" } +pluto-testutil = { path = "../../../pluto/crates/testutil" } +rand = { version = "0.8", features = ["std_rng"] } k256 = { version = "0.13.4", features = ["ecdsa", "sha256"] } prost = "0.14" prost-types = "0.14" diff --git a/consumers/rust/tests/parsigex_sender_binding.rs b/consumers/rust/tests/parsigex_sender_binding.rs new file mode 100644 index 0000000..a041436 --- /dev/null +++ b/consumers/rust/tests/parsigex_sender_binding.rs @@ -0,0 +1,352 @@ +//! Runs the `parsigex_sender_binding` suite against pluto's `dkg`/`parsigex` +//! crates. +//! +//! The vectors are transcribed from charon's `dkg/exchanger_internal_test.go` +//! (`TestVerifyPeerShareIdx` for the `cases` group, and +//! `TestNewExchangerRejectsIncompletePeerMap` for `peer_map`): a peer may only +//! contribute a partial signature under the share index the cluster assigned +//! it, resolved through a peer map rather than the peer's position. +//! +//! **Probe summary (see the report for the full write-up):** +//! - `crates/dkg/src/exchanger.rs`'s `Exchanger` is the structural +//! counterpart of charon's lock-hash exchanger, but it never validates a +//! share index at all: `Exchanger::new` takes `peers: Vec` (a bare +//! list, no index map) and only uses `peers.len()` as an exchange +//! threshold. Both share-index checking and DKG's peer-map construction +//! live elsewhere -- and every place they do is unreachable from an +//! external crate (see the `peer_map` test below for citations). +//! - `crates/dkg/src/aggregate.rs::agg_lock_hash_sig`/`verify_threshold_partials` +//! *do* look up a partial signature's `share_idx` in a `Share`'s +//! `public_shares` map and BLS-verify it -- structurally the closest +//! "verify a claimed share index" code in the whole crate -- but +//! `crates/dkg/src/lib.rs` declares `mod aggregate;` (no `pub`), so this +//! function is not reachable either. +//! - The one reachable share-index verifier is +//! `pluto_parsigex::new_eth2_verifier` (`crates/parsigex/src/lib.rs` +//! re-exports it from `behaviour.rs`), used in production for ongoing +//! validator-duty signing (attestations, etc.), not DKG's lock-hash round +//! (DKG wires parsigex with a no-op verifier and checks signatures itself +//! in the now-unreachable `aggregate.rs`, per `exchanger.rs`'s "verify +//! before we aggregate" comment). It is nonetheless pluto's closest +//! reachable equivalent of the rule under test: it also looks up +//! `pub_shares_by_key[pubkey][share_idx]` and BLS-verifies. Its `Verifier` +//! closure type is `Fn(Duty, PubKey, ParSignedData) -> VerifyFuture` -- +//! **it has no sender/peer-identity parameter at all**, confirmed by +//! reading `new_eth2_verifier` (`crates/parsigex/src/behaviour.rs`) and its +//! only caller (`crates/parsigex/src/handler.rs`) in full. +//! +//! **`cases` group -- ABSENT-OK.** Ladder entry "Sender-bound share indices +//! in the DKG lock-hash exchange" (`first_charon_release: null`, so no +//! released charon carries it either at pluto's v1.7.1 anchor) covers this +//! gap. Because `new_eth2_verifier`'s closure never receives a sender +//! identity, its accept/reject decision reduces entirely to: is `share_idx` a +//! key in `pub_shares_by_key[pubkey]`, and does the presented signature +//! verify under that key's pubshare. There is no way to even pose "the +//! sender is not the assigned owner of this index" as a question to this +//! API. Each case below is driven for real (`BeaconMock` for eth2-domain +//! resolution, a genuine BLS signature over the resolved signing root) rather +//! than asserting a re-implementation of the rule. +//! +//! **`peer_map` group -- UNREACHABLE.** This predates charon v1.7.1 (no +//! ladder entry excuses it), but every builder of a peer/share-index map is +//! either purely position-derived (so it cannot even represent the vectors' +//! non-contiguous assignment) or a private/`pub(crate)` item with no external +//! entry point. See the dedicated test below for the four citations. + +use std::collections::HashMap; + +use pluto_core::eth2signeddata::Eth2SignedData; +use pluto_core::signeddata::SignedVoluntaryExit as CoreSignedVoluntaryExit; +use pluto_core::types::{Duty, ParSignedData, PubKey, SignedData, SlotNumber}; +use pluto_crypto::blst_impl::BlstImpl; +use pluto_crypto::tbls::Tbls; +use pluto_crypto::types::{PrivateKey, PublicKey}; +use pluto_eth2api::spec::phase0; +use pluto_eth2util::signing; +use pluto_parsigex::{VerifyError, new_eth2_verifier}; +use pluto_testutil::BeaconMock; +use spec_vectors_pluto::load_suite; + +/// Real BLS keypair for one participant's share, used both to populate +/// `pub_shares_by_key` and to produce a genuinely valid signature for that +/// share index when a case calls for one. +struct ShareKey { + secret: PrivateKey, + public: PublicKey, +} + +impl ShareKey { + fn generate() -> Self { + let tbls = BlstImpl; + let secret = tbls + .generate_secret_key(rand::thread_rng()) + .expect("secret generation should succeed"); + let public = tbls + .secret_to_public_key(&secret) + .expect("public key derivation should succeed"); + Self { secret, public } + } +} + +/// Builds a `SignedVoluntaryExit` genuinely signed with `secret`, resolving +/// the signing domain/root against the real (mocked) beacon-node client -- +/// mirrors `pluto_core::eth2signeddata`'s own `assert_verifies` test helper +/// (`crates/core/src/eth2signeddata.rs`), which is the only place in pluto +/// that exercises this signing path end-to-end. +async fn sign_exit( + client: &pluto_eth2api::EthBeaconNodeApiClient, + secret: &PrivateKey, +) -> CoreSignedVoluntaryExit { + let unsigned = CoreSignedVoluntaryExit::new(phase0::SignedVoluntaryExit { + message: phase0::VoluntaryExit { + epoch: 5, + validator_index: 7, + }, + signature: [0u8; 96], + }); + + let epoch = unsigned.epoch(client).await.expect("epoch resolution"); + let root = unsigned.message_root().expect("message root"); + let sig_root = signing::get_data_root(client, unsigned.domain_name(), epoch, root) + .await + .expect("signing-domain data root"); + let sig = BlstImpl.sign(secret, &sig_root).expect("BLS sign"); + + unsigned.set_signature(sig).expect("set_signature") +} + +/// Drives one `cases` vector entry through the real `new_eth2_verifier`. +/// +/// The vector's `sender`/`share_idx_by_peer` fields describe a rule pluto's +/// API cannot even represent (no sender parameter exists), so this always +/// signs the exit with whichever real secret key corresponds to the *claimed* +/// `share_idx` -- i.e. it assumes the message genuinely was produced with +/// that share's private key, isolating "does the binding fire" from "is this +/// forged crypto" (a forged signature would fail for an unrelated reason: +/// `VerifyError::InvalidSignature`, not the absent-binding gap this suite +/// pins). When `share_idx` is not one of the two registered indices (1, 4), +/// there is no real key for it, so an arbitrary throwaway key is used -- +/// irrelevant, since the map lookup rejects before any signature is checked. +async fn run_case( + client: &pluto_eth2api::EthBeaconNodeApiClient, + pub_shares_by_key: &HashMap>, + self_key: &ShareKey, + other_key: &ShareKey, + dv_pubkey: PubKey, + share_idx: u64, +) -> Result<(), VerifyError> { + let secret = match share_idx { + 1 => &self_key.secret, + 4 => &other_key.secret, + _ => &ShareKey::generate().secret, + }; + let signed = sign_exit(client, secret).await; + let duty = Duty::new_signature_duty(SlotNumber::new(101)); + let par_signed = ParSignedData::new(signed, share_idx); + let verifier = new_eth2_verifier(client.clone(), pub_shares_by_key.clone()); + verifier(duty, dv_pubkey, par_signed).await +} + +#[tokio::test] +async fn cases() { + let suite = load_suite("parsigex_sender_binding"); + let cases = suite["cases"].as_array().unwrap(); + assert_eq!( + cases.len(), + 6, + "expected 6 parsigex_sender_binding/cases cases" + ); + + let mock = BeaconMock::builder() + .build() + .await + .expect("beacon mock should start"); + let client = mock.client(); + + let self_key = ShareKey::generate(); + let other_key = ShareKey::generate(); + let dv_pubkey = PubKey::new([0x42; 48]); + let pub_shares_by_key: HashMap> = HashMap::from([( + dv_pubkey, + HashMap::from([(1u64, self_key.public), (4u64, other_key.public)]), + )]); + + let mut pin_failures = Vec::new(); + let mut report = String::new(); + + for case in cases { + let name = case["name"].as_str().unwrap(); + let input = &case["input"]; + let share_idx = input["share_idx"].as_u64().unwrap(); + let spec_accepted = case["accepted"].as_bool().unwrap(); + let spec_reason = case["reason"].as_str(); + + // Sanity-check the vector's own shape: it always describes the same + // two-peer map with self=1, other=4 -- if a future vector regen + // changes this, the fixture above (built once, outside the loop) + // would silently stop matching the case, so fail loudly instead. + let share_idx_by_peer = &input["share_idx_by_peer"]; + assert_eq!(share_idx_by_peer["self"].as_u64(), Some(1)); + assert_eq!(share_idx_by_peer["other"].as_u64(), Some(4)); + + let result = run_case( + client, + &pub_shares_by_key, + &self_key, + &other_key, + dv_pubkey, + share_idx, + ) + .await; + + // Pin reading: pluto's real, current behaviour. The verifier has no + // sender parameter, so acceptance depends solely on whether + // `share_idx` is a registered key (1 or 4) -- exactly what this + // fixture signs a genuinely valid signature for. This must hold + // regardless of what the vector's `sender`/`accepted` fields say; + // if it ever stops holding, either pluto grew a sender check (in + // which case this whole pin should be revisited and likely deleted) + // or the fixture/vector shape changed underneath this test. + let registered = matches!(share_idx, 1 | 4); + let pluto_accepted = result.is_ok(); + if pluto_accepted != registered { + pin_failures.push(format!( + "parsigex_sender_binding/cases/{name}: expected pluto to \ + {}(share_idx={share_idx} is{} registered), got {result:?}", + if registered { "accept " } else { "reject " }, + if registered { "" } else { " not" } + )); + } + // Where pluto rejects, the only reason its API can produce here is + // an unregistered share index -- assert the exact variant so a + // change in error shape (e.g. pluto starting to distinguish a + // signature failure) is visible. + if !registered { + match &result { + Err(VerifyError::InvalidShareIndex) => {} + other => pin_failures.push(format!( + "parsigex_sender_binding/cases/{name}: expected \ + VerifyError::InvalidShareIndex for an unregistered share_idx, got {other:?}" + )), + } + } + + // Spec reading: reported, not asserted. Pluto's `pub_shares_by_key` + // lookup has no concept of "sender", so it cannot reproduce charon's + // sender-bound rejections -- diverges exactly on the two cases where + // the claimed index is registered to *some* peer but the vector's + // `sender` is not that peer (`another_peers_share_index_rejected`) + // or is not a peer at all (`unknown_sender_rejected`). + if pluto_accepted == spec_accepted { + report.push_str(&format!( + "parsigex_sender_binding/cases/{name}: matches spec (accepted={spec_accepted})\n" + )); + } else { + report.push_str(&format!( + "parsigex_sender_binding/cases/{name}: diverges from spec (spec accepted=\ + {spec_accepted}, reason={spec_reason:?}; pluto accepted={pluto_accepted}) -- \ + ABSENT-OK, ladder entry \"Sender-bound share indices in the DKG lock-hash \ + exchange\" (first_charon_release: null): new_eth2_verifier has no sender \ + parameter, so it cannot enforce which peer may claim which share index\n" + )); + } + } + + println!("{report}"); + assert!(pin_failures.is_empty(), "{}", pin_failures.join("\n")); +} + +/// `peer_map` group -- construction-time validation of a peer/share-index +/// map. UNREACHABLE: every function in pluto that builds or validates such a +/// map is either arithmetically incapable of representing the vectors' +/// non-contiguous assignment, or private/`pub(crate)`. +/// +/// Citations (file + function, no line numbers per the report contract): +/// - `crates/p2p/src/peer.rs::Peer::share_idx` / `crates/cluster/src/ +/// definition.rs::Definition::node_idx` (both `pub`, reachable) -- but +/// `share_idx` is defined as `self.index.wrapping_add(1)`, i.e. purely the +/// operator's position in the definition's peer list. There is no `pub` +/// path that can produce share index 4 for the second of two peers (this +/// suite's `share_idx_by_peer = {self: 1, other: 4}`): the API has no +/// parameter through which to supply an assigned index at all, so it +/// cannot even be driven with these inputs, let alone tested for +/// accepting/rejecting them. +/// - `crates/dkg/src/node.rs::setup_p2p` -- builds a real `share_idx_by_peer: +/// HashMap` (again from `peer.share_idx()`, so +/// position-derived) before constructing the FROST P2P transport, but is +/// `pub(crate) async fn`; `crates/dkg/src/lib.rs` declares `mod node;` +/// (private), so this function is unreachable from an external crate. +/// - `crates/dkg/src/frostp2p/transport.rs::validate_peer_share_indices` -- +/// the one function that actually validates a map (rejects `share_idx == +/// 0`, duplicate indices, and a map missing the local peer's index) has no +/// visibility modifier at all (private to its module); its only caller, +/// `new_frost_p2p`, is `pub(crate)`; and `crates/dkg/src/lib.rs` declares +/// `mod frostp2p;` (private) -- three layers of unreachability. +/// - `crates/app/src/node/mod.rs::build_pub_shares_by_key` -- builds +/// `pub_shares_by_key` for `new_eth2_verifier` from a `Lock`, but is a +/// private `fn` (no `pub`), and is itself position-derived +/// (`(pos as u64).saturating_add(1)`), so it has the same representational +/// gap as `Definition::node_idx` even where it is compiled from a +/// `pub`-reachable crate. +/// +/// Each case is still enumerated and its shape asserted below, per the task +/// brief's "cases must still be enumerated and accounted for" requirement -- +/// there is simply no reachable pluto API left to call with its inputs. +#[test] +fn peer_map() { + let suite = load_suite("parsigex_sender_binding"); + let cases = suite["peer_map"].as_array().unwrap(); + assert_eq!( + cases.len(), + 3, + "expected 3 parsigex_sender_binding/peer_map cases" + ); + + let mut accounted = Vec::new(); + for case in cases { + let name = case["name"].as_str().unwrap(); + let input = &case["input"]; + let peers: Vec<&str> = input["peers"] + .as_array() + .unwrap() + .iter() + .map(|p| p.as_str().unwrap()) + .collect(); + let share_idx_by_peer: HashMap<&str, u64> = input["share_idx_by_peer"] + .as_object() + .unwrap() + .iter() + .map(|(k, v)| (k.as_str(), v.as_u64().unwrap())) + .collect(); + let peer_idx = input["peer_idx"].as_u64().unwrap(); + let accepted = case["accepted"].as_bool().unwrap(); + let reason = case["reason"].as_str(); + + // Confirm the very thing that makes this UNREACHABLE rather than a + // reachable PASS/FAIL: at least one peer in every case's roster has + // no map entry, or the map assigns index 0, or the map is + // non-contiguous (index 4 for a 2-peer map) -- none of which + // `Peer::share_idx`'s position-derived formula (`position + 1`) can + // even receive as input, since it has no map parameter at all. + let all_positional = peers + .iter() + .enumerate() + .all(|(i, peer)| share_idx_by_peer.get(peer) == Some(&(i as u64 + 1))); + assert!( + !all_positional || share_idx_by_peer.len() != peers.len(), + "parsigex_sender_binding/peer_map/{name}: this case's map is fully \ + position-derived, so it no longer demonstrates non-reachability -- \ + re-check whether a public position-derived builder now applies" + ); + + accounted.push(format!( + "parsigex_sender_binding/peer_map/{name}: UNREACHABLE (spec expects \ + accepted={accepted}, reason={reason:?}, for peers={peers:?} peer_idx={peer_idx}) -- \ + no reachable pluto construction validates a peer/share-index map; see the module \ + doc comment for the four citations" + )); + } + + assert_eq!(accounted.len(), 3); + println!("{}", accounted.join("\n")); +} diff --git a/plans/pluto-conformance.md b/plans/pluto-conformance.md index bfe8407..53be50b 100644 --- a/plans/pluto-conformance.md +++ b/plans/pluto-conformance.md @@ -75,7 +75,7 @@ pluto commit actually tested. Statuses: `todo` / `in progress` / `done`. | 6 | Round timer deadlines | `timer_deadlines` | done | round_timeout: PASS **only under `ProposalTimeout` enabled**, a documented non-default precondition (pluto's *default* config has a pinned, real divergence on `PROPOSER`/round-1 cases — see Notes and Findings); deadline: ABSENT-OK, with a real code-level slot-invariance pin (not just a comment); duty_start_delay: UNREACHABLE | `round_timeouts_with_proposal_timeout_enabled`: 216/216 PASS, but only with `Feature::ProposalTimeout` explicitly enabled (it is `Status::Alpha`, off under pluto's `Config::default()`). `round_timeouts_pluto_default_feature_set` runs the same 216 cases under pluto's actual default `FeatureSet` and pins the resulting divergence: the 207 non-`PROPOSER`-round-1 cases still match the vectors, but all 9 `PROPOSER`/round-1 cases (3 slots x 3 slot durations) diverge — pluto's default gives 1s, the vectors (which assume `ProposalTimeout` enabled) expect 1.5s. This is a real, pinned known-divergence (see Findings), not swept under the enabled-feature test's PASS. `deadline_nanos`: `round_timeout_is_duty_slot_invariant` builds two timers per (duty_type, round) combination (24 total, covering all cases) differing only in `slot` (0 vs. 7231) and asserts bit-identical measured timeouts — a real code pin for "no timer reads slot", not a doc comment, so it fails loudly if `with_duty`/`RoundTimer::timer` ever grows slot-dependence; genesis-invariance is structurally guaranteed (the API has no genesis parameter at all) and is stated as such rather than tested. Matches ladder entry "Deterministic (genesis-derived) eager double linear round deadlines" (`first_charon_release: v1.9.0` > pluto's v1.7.1 anchor). `duty_start_delay_nanos` is UNREACHABLE: `delay_slot_offset` (`crates/core/src/scheduler.rs`) is a non-`pub` `async fn` with no external caller, confirmed still private. | | 7 | QBFT message limits | `qbft_msg_limits` | done | ABSENT-OK (counts, both directions); PASS (wire_size) | `counts` (11/11 cases run): pluto has neither the spec's `2n` justification cap nor any `2*(j+1)` values cap. Its own cap is `4 * node_count()` (component.rs `MAX_JUSTIFICATIONS_PER_NODE = 4`, strict `>`), and there is no values-length check anywhere in `crates/consensus/src`/`crates/core/src`. Every counts case's `justification_count` stays at/under pluto's `4n` (proven structurally in the test, not by case name), so pluto accepts all 11 cases: 5 match the spec's `accepted: true` cases outright, 6 are ABSENT-OK divergences (spec says reject — `too_many_justifications` or `too_many_values` — pluto accepts) against ladder entry "QBFT DECIDED-resend rate limit and message size/count limits" (`first_charon_release: null`). Two extra tests pin pluto's own real boundary directly, since no vector case reaches it: `4 * nodes` justifications accepted, `4 * nodes + 1` rejected with `Error::TooManyJustifications { count, max }` asserted exactly (not just "any error"). `wire_size` (4/4 cases): `MAX_CONSENSUS_MSG_SIZE` (32 MiB) and `MAX_MESSAGE_SIZE` (128 MiB) constants pinned; enforcement runs the real `pluto_p2p::proto::read_protobuf_with_max_size` reader (mirroring `qbft/p2p.rs`'s own unit tests) over `futures::io::Cursor` — 1024 bytes and exactly 32 MiB accepted (round-trip decode verified byte-for-byte), 32 MiB+1 and 128 MiB rejected with a "too large" reason, matching `msg_too_large` exactly. `DeadlinerHandle::always` from the probe doc turned out to be `#[cfg(test)]`-gated inside pluto_core (not reachable externally); worked around with the real public `DeadlinerTask::start` and a custom future-deadline `DeadlineCalculator` (pluto's own public `NeverExpiringCalculator` returns `NoDeadline`, which `handle`'s strict `!= Scheduled` check treats as `Error::DutyExpired` — a real gotcha, not a doc typo). See Findings for the `4n` cap as its own hardening decision. | | 8 | DECIDED-resend limiting | `qbft_decided_resends` | done | ABSENT-OK (pin reading PASS; spec-reading mismatches reported per case) | Both cases run against the real `pluto_core::qbft::run` state machine (not code inspection alone), driven to DECIDED at round 1 via the minimal justified PRE-PREPARE + quorum PREPARE + quorum COMMIT sequence, then fed each case's ROUND-CHANGE event sequence one at a time. Pin reading (pluto's actual, unconditional rule — rebroadcast DECIDED on every post-decision ROUND-CHANGE from a source other than the deciding process, no cap, no round check) holds for all 28 events across both cases: PASS. Spec reading (the vector's per-event `rebroadcast` expectations, which encode a 16-per-source cap plus a strictly-increasing-round-per-source dedup) diverges on 4/7 events in `dedup_duplicates_and_stale_rounds` (the duplicate-round and stale-round entries) and 5/21 in `resend_cap_per_source` (the 17th-through-21st events, past pluto's absent 16-cap) — reported, not asserted, matching ladder entry "QBFT DECIDED-resend rate limit and message size/count limits" (`first_charon_release: null`). `SomeMsg`/`Definition`/`Transport`/`QbftLogger` were externally implementable exactly as the probe doc claimed, confirming `run` is fully reachable from an external crate. | -| 9 | Sender binding + peer map | `parsigex_sender_binding` | todo | — | | +| 9 | Sender binding + peer map | `parsigex_sender_binding` | done | `cases`: ABSENT-OK; `peer_map`: UNREACHABLE | `cases` (6/6 run) driven for real through `pluto_parsigex::new_eth2_verifier` (a genuine BLS-signed `SignedVoluntaryExit`, resolved against a `BeaconMock`): its `Verifier` closure has no sender parameter at all, so acceptance depends solely on whether the claimed `share_idx` is a registered key with a genuinely valid signature. 4/6 cases coincide with the vector (`own_share_index_accepted`, `assigned_non_contiguous_share_index_accepted`, `mismatched_share_index_rejected`, `non_positive_share_index_rejected`); 2/6 diverge (`another_peers_share_index_rejected`, `unknown_sender_rejected`) because pluto accepts a genuinely-signed but wrongly-attributed share index — pinned against ladder entry "Sender-bound share indices in the DKG lock-hash exchange" (`first_charon_release: null`). `peer_map` (3/3 enumerated, not runnable): every pluto function that builds or validates a peer/share-index map is either arithmetically position-derived (`Peer::share_idx` = `position + 1`, `crates/p2p/src/peer.rs`, used by the reachable `Definition::node_idx`, `crates/cluster/src/definition.rs`) and so cannot even represent the vectors' non-contiguous assignment (share index 4 for the second of two peers), or private/`pub(crate)` with no external entry point (`crates/dkg/src/node.rs::setup_p2p`, `crates/dkg/src/frostp2p/transport.rs::validate_peer_share_indices`, `crates/app/src/node/mod.rs::build_pub_shares_by_key`). This predates charon v1.7.1 with no ladder cover, so it is a genuine coverage gap, not excused as ABSENT-OK. | | 10 | Coverage guard + docs | all | todo | — | | ## Findings @@ -274,6 +274,49 @@ whether a ladder entry covers it, and whether it was reported upstream._ same ladder entry task 7 matches for the justification/values caps. Not reported upstream — matches charon's own absence at the pinned anchor. +- **No sender-bound share-index check anywhere reachable in `parsigex`/`dkg` + (ABSENT-OK, task 9, `parsigex_sender_binding/cases`).** `pluto_parsigex::new_eth2_verifier` + (`crates/parsigex/src/behaviour.rs`, re-exported at `crates/parsigex/src/lib.rs`) is the + one reachable share-index verifier in pluto, and its `Verifier` closure type, + `Fn(Duty, PubKey, ParSignedData) -> VerifyFuture`, has no sender/peer-identity + parameter — confirmed by reading `new_eth2_verifier` and its only caller + (`crates/parsigex/src/handler.rs`) in full. Driven for real (a genuinely BLS-signed + `SignedVoluntaryExit`, signing domain resolved against a `pluto_testutil::BeaconMock`), + its accept/reject decision reduces entirely to "is `share_idx` a registered key with a + valid signature": 2 of 6 vector cases diverge because pluto accepts a share index that + is genuinely, validly signed but belongs to a different peer than the one claiming it + (`another_peers_share_index_rejected`) or to no registered peer at all + (`unknown_sender_rejected`). Ladder entry "Sender-bound share indices in the DKG + lock-hash exchange" (`first_charon_release: null`) covers this exactly — no released + charon carries the check either, so this is ABSENT-OK, not reported upstream. + Note that DKG's own lock-hash exchange does not even go through this verifier: `Exchanger` + (`crates/dkg/src/exchanger.rs`) wires parsigex with a no-op verifier and instead checks + signatures post-hoc in `crates/dkg/src/aggregate.rs::agg_lock_hash_sig`/ + `verify_threshold_partials` (same no-sender-parameter shape) — but `crates/dkg/src/lib.rs` + declares `mod aggregate;` without `pub`, so that code is not reachable from an external + crate at all; `new_eth2_verifier` is the closest reachable equivalent, not the exact + code path DKG itself uses. + +- **No reachable construction validates a peer/share-index map (FAIL by omission — + coverage gap, task 9, `parsigex_sender_binding/peer_map`).** This rule predates charon + v1.7.1, so no ladder entry excuses it, but no reachable pluto API can even be exercised + to check it. Every share_idx map-builder found is one of: (a) arithmetically + position-derived and thus structurally incapable of producing the vectors' non-contiguous + assignment (share index 4 for the second of two peers) — `crates/p2p/src/peer.rs::Peer::share_idx` + (`self.index.wrapping_add(1)`), used by the reachable `crates/cluster/src/definition.rs::Definition::node_idx`, + and by `crates/app/src/node/mod.rs::build_pub_shares_by_key` (`(pos as u64).saturating_add(1)`, + itself private); or (b) a genuine peer-indexed map with real validation + (`crates/dkg/src/frostp2p/transport.rs::validate_peer_share_indices`, which does reject + `share_idx == 0`, duplicate indices, and a map missing the local peer's index) that is + private to its module, whose only caller `new_frost_p2p` is `pub(crate)`, inside a `mod frostp2p;` + (private) declared in `crates/dkg/src/lib.rs`; or (c) `crates/dkg/src/node.rs::setup_p2p`, + which builds the map passed to that validator but is itself `pub(crate)` inside a private + `mod node;`. Verdict: UNREACHABLE, not FAIL — there is no reachable path to demonstrate + actual acceptance of an invalid map, only its structural absence. Worth flagging to pluto + as a coverage gap (no unit test anywhere in the `dkg`/`parsigex` crates exercises + `validate_peer_share_indices` or an equivalent map-completeness check), not reported + upstream as a defect since we can't observe the behavior from outside the crate. + ## Global Constraints - Pluto is **never modified**. The harness lives in `consumers/rust/` and path-depends From 13e62104e9150a458e627808dcf99278efdceb03 Mon Sep 17 00:00:00 2001 From: Andrei Smirnov Date: Sat, 1 Aug 2026 07:08:34 +0300 Subject: [PATCH 16/27] Add coverage guard for the Rust pluto consumer Asserts the published test_vectors/*.json suites match the COVERED table's declared suites, and that each declared test file exists. Mirrors the Go consumer's TestEverySuiteIsCovered. --- consumers/rust/tests/coverage.rs | 48 ++++++++++++++++++++++++++++++++ 1 file changed, 48 insertions(+) create mode 100644 consumers/rust/tests/coverage.rs diff --git a/consumers/rust/tests/coverage.rs b/consumers/rust/tests/coverage.rs new file mode 100644 index 0000000..1683f7e --- /dev/null +++ b/consumers/rust/tests/coverage.rs @@ -0,0 +1,48 @@ +//! Guards against a published vector suite that nothing runs. +//! +//! An uncovered suite is indistinguishable from a passing one: the Go consumer's +//! `TestEverySuiteIsCovered` (`consumers/go/testutil/specvectors/specvectors.go`) +//! catches this for Charon; this is the equivalent for the Rust side. + +use std::collections::BTreeSet; + +/// Suite name -> test file that runs it. Update when a suite or test is added. +const COVERED: &[(&str, &str)] = &[ + ("secp256k1_signatures", "tests/secp256k1_signatures.rs"), + ("qbft_hashing", "tests/qbft_hashing.rs"), + ("bls_threshold", "tests/bls_threshold.rs"), + ("cluster_hashing", "tests/cluster_hashing.rs"), + ("priority_scoring", "tests/priority_scoring.rs"), + ("timer_deadlines", "tests/timer_deadlines.rs"), + ("qbft_msg_limits", "tests/qbft_msg_limits.rs"), + ("qbft_decided_resends", "tests/qbft_decided_resends.rs"), + ( + "parsigex_sender_binding", + "tests/parsigex_sender_binding.rs", + ), +]; + +#[test] +fn every_suite_is_covered() { + let published: BTreeSet = std::fs::read_dir(spec_vectors_pluto::vectors_dir()) + .expect("vectors dir") + .filter_map(|e| { + let p = e.expect("dir entry").path(); + (p.extension()? == "json") + .then(|| p.file_stem().unwrap().to_string_lossy().into_owned()) + }) + .collect(); + let covered: BTreeSet = COVERED.iter().map(|(s, _)| s.to_string()).collect(); + assert_eq!( + published, covered, + "published suites and covered suites disagree -- a suite nothing runs is a document, not a test" + ); + for (suite, file) in COVERED { + assert!( + std::path::Path::new(env!("CARGO_MANIFEST_DIR")) + .join(file) + .exists(), + "{suite}: declared test file {file} does not exist" + ); + } +} From 596161ae633a9125f40fb013412d9c0206bfb34d Mon Sep 17 00:00:00 2001 From: Andrei Smirnov Date: Sat, 1 Aug 2026 07:08:48 +0300 Subject: [PATCH 17/27] Note the wire_size_enforcement string-matching tradeoff read_protobuf_with_max_size reports every rejection through the same io::ErrorKind::InvalidData with no structured reason code, so matching on err.to_string() is unavoidable here despite test_vectors/README.md's "the slugs are the contract, wording is not" -- pluto's own p2p.rs tests use the same idiom. --- consumers/rust/tests/qbft_msg_limits.rs | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/consumers/rust/tests/qbft_msg_limits.rs b/consumers/rust/tests/qbft_msg_limits.rs index ec51534..4b7b50d 100644 --- a/consumers/rust/tests/qbft_msg_limits.rs +++ b/consumers/rust/tests/qbft_msg_limits.rs @@ -392,6 +392,13 @@ fn qbft_consensus_msg_of_encoded_len(target_len: usize) -> pbconsensus::QbftCons /// `inbound_rejects_message_exceeding_max_consensus_size`) invoke /// `read_protobuf_with_max_size` with `MAX_CONSENSUS_MSG_SIZE`, over the same /// `futures::io::Cursor` in-memory stream type. +/// +/// Matching on `err.to_string()` below is in tension with +/// `test_vectors/README.md`'s "the slugs are the contract, wording is not": +/// it is unavoidable here because every rejection path in +/// `read_protobuf_with_max_size` reports through the same `io::ErrorKind::InvalidData` +/// with no structured reason code, and pluto's own `p2p.rs` tests assert the +/// same substring for the same reason. #[tokio::test] async fn wire_size_enforcement() { let suite = load_suite("qbft_msg_limits"); From 69f54882c271c2a50191ea891e4db617ccca7977 Mon Sep 17 00:00:00 2001 From: Andrei Smirnov Date: Sat, 1 Aug 2026 07:09:44 +0300 Subject: [PATCH 18/27] Trim libp2p features to those the Rust consumer uses The feature list was copied wholesale from pluto's workspace Cargo.toml. Only noise, tokio, and yamux are needed here; the rest (autonat, dns, identify, mdns, ping, quic, relay, secp256k1, macros, tcp) are unused by this consumer's code. --- consumers/rust/Cargo.toml | 16 +--------------- 1 file changed, 1 insertion(+), 15 deletions(-) diff --git a/consumers/rust/Cargo.toml b/consumers/rust/Cargo.toml index 4c91c68..3cbe50d 100644 --- a/consumers/rust/Cargo.toml +++ b/consumers/rust/Cargo.toml @@ -34,21 +34,7 @@ tokio-util = "0.7.11" futures = "0.3" async-trait = "0.1.89" chrono = "0.4" -libp2p = { version = "0.56", features = [ - "autonat", - "dns", - "identify", - "macros", - "mdns", - "noise", - "ping", - "quic", - "relay", - "secp256k1", - "tcp", - "tokio", - "yamux", -] } +libp2p = { version = "0.56", features = ["noise", "tokio", "yamux"] } [patch.crates-io] multistream-select = { path = "../../../pluto/third_party/multistream-select" } From c3e55943a9ed9b0a10fb03f9c4106ed5f3ac7d3c Mon Sep 17 00:00:00 2001 From: Andrei Smirnov Date: Sat, 1 Aug 2026 07:10:45 +0300 Subject: [PATCH 19/27] Extend the consumer coverage guard to the Rust side The Rust consumer's coverage.rs is not compiled here either, so mirror the Go check: parse COVERED out of consumers/rust/tests/coverage.rs and assert it matches the published suites, and that each declared test file exists. --- tests/test_consumers.py | 46 ++++++++++++++++++++++++++++++++++------- 1 file changed, 38 insertions(+), 8 deletions(-) diff --git a/tests/test_consumers.py b/tests/test_consumers.py index fa56058..2d7cc7c 100644 --- a/tests/test_consumers.py +++ b/tests/test_consumers.py @@ -1,13 +1,14 @@ """Guard the consumer suites in `consumers/` against drift from this repo. -The Go code is not compiled here — it compiles inside a Charon checkout — so -nothing else in this repository notices when it goes stale. The failure that -matters is silent: adding a vector suite leaves the Go side unaware of it, and -Charon keeps passing while covering less than it claims. - -Charon's own `TestEverySuiteIsCovered` catches the same drift, but only for -whoever runs it in a Charon checkout. These tests catch it here, where the suite -is added. +The Go and Rust code are not compiled here — the Go side compiles inside a +Charon checkout, and the Rust side path-depends on a pluto checkout — so +nothing else in this repository notices when either goes stale. The failure +that matters is silent: adding a vector suite leaves a consumer unaware of it, +and it keeps passing while covering less than it claims. + +Charon's own `TestEverySuiteIsCovered` and pluto consumer's `every_suite_is_covered` +catch the same drift, but only for whoever runs them in the respective checkout. +These tests catch it here, where the suite is added. """ from __future__ import annotations @@ -21,6 +22,7 @@ REPO_ROOT = Path(__file__).resolve().parent.parent CONSUMERS = REPO_ROOT / "consumers" LOADER = CONSUMERS / "go" / "testutil" / "specvectors" / "specvectors.go" +RUST_COVERAGE_GUARD = CONSUMERS / "rust" / "tests" / "coverage.rs" VECTOR_ROOT = REPO_ROOT / "test_vectors" @@ -29,6 +31,11 @@ def loader_source() -> str: return LOADER.read_text() +@pytest.fixture(scope="module") +def rust_coverage_source() -> str: + return RUST_COVERAGE_GUARD.read_text() + + def covered_suites(source: str) -> set[str]: """Parse the suite names out of the Go loader's CoveredSuites map.""" block = re.search(r"var CoveredSuites = map\[string\]string\{(.*?)\n\}", source, re.DOTALL) @@ -74,3 +81,26 @@ def test_every_go_file_is_placed_under_a_charon_package_path() -> None: assert relative.parts[0] in {"core", "dkg", "testutil"}, ( f"{relative} is not under a known Charon top-level directory" ) + + +def rust_covered_suites(source: str) -> dict[str, str]: + """Parse the suite -> test file pairs out of the Rust guard's COVERED table.""" + block = re.search(r"const COVERED: &\[\(&str, &str\)\] = &\[(.*?)\n\];", source, re.DOTALL) + assert block, "COVERED table not found in consumers/rust/tests/coverage.rs" + + pairs = re.findall(r'\(\s*"([a-z0-9_]+)"\s*,\s*"([^"]+)"\s*,?\s*\)', block.group(1)) + return dict(pairs) + + +def test_rust_consumer_covers_every_suite(rust_coverage_source: str) -> None: + published = {path.stem for path in VECTOR_ROOT.glob("*.json")} + + assert set(rust_covered_suites(rust_coverage_source)) == published + + +def test_rust_consumer_declared_test_files_exist(rust_coverage_source: str) -> None: + # The Rust code is not compiled here, so a stale entry in COVERED — pointing + # at a renamed or deleted test file — would otherwise go unnoticed. + for suite, relative_file in rust_covered_suites(rust_coverage_source).items(): + path = CONSUMERS / "rust" / relative_file + assert path.exists(), f"{suite}: declared test file {relative_file} does not exist" From b2a7b8c5fa455240808571831c57f9d01d66dfa8 Mon Sep 17 00:00:00 2001 From: Andrei Smirnov Date: Sat, 1 Aug 2026 07:11:53 +0300 Subject: [PATCH 20/27] Document the pluto consumer's final verdicts consumers/README.md: expand the Pluto row and section with suite-by-suite verdicts, run instructions, and the pluto commit under test (67088a2). plans/spec-completion.md: mark the pluto consumer bullet done, pointing at plans/pluto-conformance.md's Results table. --- consumers/README.md | 32 +++++++++++++++++++++++++++++--- plans/spec-completion.md | 7 +++---- 2 files changed, 32 insertions(+), 7 deletions(-) diff --git a/consumers/README.md b/consumers/README.md index 95cddf9..a5f3e94 100644 --- a/consumers/README.md +++ b/consumers/README.md @@ -7,7 +7,7 @@ executes them: a suite nobody runs is a document, not a test. | Consumer | Status | Location | | --------------------------------- | ---------------------------------------------- | ----------- | | Charon (Go) | All 9 suites, 314 subtests, verified at anchor `6054bcb2` | [`go/`](go) | -| Pluto (Rust) | in progress — see plans/pluto-conformance.md | [rust/](rust) | +| Pluto (Rust) | All 9 suites checked against pluto `67088a2` — 2 FAIL (`qbft_hashing`, `cluster_hashing`), rest PASS, ABSENT-OK, or UNREACHABLE; see plans/pluto-conformance.md | [rust/](rust) | These files live here rather than in Charon because this repository cannot merge into Charon. They are laid out to mirror Charon's own tree, so placing them is a @@ -90,10 +90,36 @@ placed as a sibling of this repository (`../pluto` relative to Unlike the Go consumer, nothing is copied into pluto and pluto is never modified — the harness reads pluto's crates in place. +### Running it + ```bash +# pluto checked out as a sibling of this repo (default: ../pluto) cd consumers/rust && cargo test ``` Vectors are read from `../../test_vectors/` at test time (no vendoring), or -from `SPEC_VECTORS_DIR` when set. Progress and per-suite verdicts are tracked -in `plans/pluto-conformance.md`. +from `SPEC_VECTORS_DIR` when set (same variable as the Go consumer), for +running against an unreleased build without moving files around. + +### Verdicts + +All nine published suites were run against pluto commit `67088a2`. This is a +snapshot of one pluto commit, not a continuously tracked target: `PASS` means +pluto reproduced every case the vector covers at that commit; `ABSENT-OK` and +`UNREACHABLE` are not passes — they record, respectively, a divergence excused +by pluto's charon-v1.7.1 parity pin, and a coverage gap where no public pluto +API reaches the behaviour at all. Full per-case detail, findings, and pluto +file:line citations are in `plans/pluto-conformance.md`'s Results table and +Findings section. + +| Suite | Verdict | +| --- | --- | +| `secp256k1_signatures` | PASS (2/2) | +| `qbft_hashing` | **FAIL** — 23/25 cases agree; 2 pinned divergences (prost omits default-valued protobuf map-entry key/value fields that charon's Go marshaler always emits) | +| `bls_threshold` | PASS (all 5 groups) | +| `cluster_hashing` | **FAIL** — pluto rejects charon-legitimate JSON shapes (`operators: null`, an absent `partial_deposit_data`, and a masked third gap on `timestamp`) | +| `priority_scoring` | PASS (18/18) | +| `timer_deadlines` | `round_timeout_nanos` PASS **only with `ProposalTimeout` explicitly enabled** (9 PROPOSER/round-1 cases diverge under pluto's default config); `deadline_nanos` ABSENT-OK; `duty_start_delay_nanos` UNREACHABLE | +| `qbft_msg_limits` | `counts`: 5 MATCH + 6 ABSENT-OK; `wire_size`: 4/4 MATCH | +| `qbft_decided_resends` | ABSENT-OK (pluto has no DECIDED-rebroadcast limiter) | +| `parsigex_sender_binding` | `cases` ABSENT-OK; `peer_map` UNREACHABLE | diff --git a/plans/spec-completion.md b/plans/spec-completion.md index 62ab806..c292703 100644 --- a/plans/spec-completion.md +++ b/plans/spec-completion.md @@ -382,10 +382,9 @@ Also fixed in this pass: ~/charon/`. All nine suites, **314 subtests**, run green against a charon worktree at the anchor `6054bcb2`. Not a PR yet — this repo cannot merge into charon, so `consumers/README.md` documents placement and the pinning rules. - - [ ] **Pluto (Rust)** — not started. Pluto's crate layout mirrors charon's - (`crates/{cluster,consensus,priority,parsigex,crypto,k1util,...}`), and it - already carries `crates/consensus/testdata/vectors/hashproto.json`, which - `qbft_hashing.json` is meant to replace. + - [x] **Pluto (Rust), written and verified 2026-08-01.** See + `plans/pluto-conformance.md` and its Results table for the full + per-suite verdicts against pluto commit `67088a2`. Correction to the plan's wording: it cannot be "a Go test package". Almost everything the vectors cover is unexported in charon — `hashProto`, From 515bc64985fc5f9ec160e55746d036b66bfeb43f Mon Sep 17 00:00:00 2001 From: Andrei Smirnov Date: Sat, 1 Aug 2026 07:12:05 +0300 Subject: [PATCH 21/27] Record Results row 10 for the coverage guard and docs task --- plans/pluto-conformance.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/plans/pluto-conformance.md b/plans/pluto-conformance.md index 53be50b..4dae1e5 100644 --- a/plans/pluto-conformance.md +++ b/plans/pluto-conformance.md @@ -76,7 +76,7 @@ pluto commit actually tested. Statuses: `todo` / `in progress` / `done`. | 7 | QBFT message limits | `qbft_msg_limits` | done | ABSENT-OK (counts, both directions); PASS (wire_size) | `counts` (11/11 cases run): pluto has neither the spec's `2n` justification cap nor any `2*(j+1)` values cap. Its own cap is `4 * node_count()` (component.rs `MAX_JUSTIFICATIONS_PER_NODE = 4`, strict `>`), and there is no values-length check anywhere in `crates/consensus/src`/`crates/core/src`. Every counts case's `justification_count` stays at/under pluto's `4n` (proven structurally in the test, not by case name), so pluto accepts all 11 cases: 5 match the spec's `accepted: true` cases outright, 6 are ABSENT-OK divergences (spec says reject — `too_many_justifications` or `too_many_values` — pluto accepts) against ladder entry "QBFT DECIDED-resend rate limit and message size/count limits" (`first_charon_release: null`). Two extra tests pin pluto's own real boundary directly, since no vector case reaches it: `4 * nodes` justifications accepted, `4 * nodes + 1` rejected with `Error::TooManyJustifications { count, max }` asserted exactly (not just "any error"). `wire_size` (4/4 cases): `MAX_CONSENSUS_MSG_SIZE` (32 MiB) and `MAX_MESSAGE_SIZE` (128 MiB) constants pinned; enforcement runs the real `pluto_p2p::proto::read_protobuf_with_max_size` reader (mirroring `qbft/p2p.rs`'s own unit tests) over `futures::io::Cursor` — 1024 bytes and exactly 32 MiB accepted (round-trip decode verified byte-for-byte), 32 MiB+1 and 128 MiB rejected with a "too large" reason, matching `msg_too_large` exactly. `DeadlinerHandle::always` from the probe doc turned out to be `#[cfg(test)]`-gated inside pluto_core (not reachable externally); worked around with the real public `DeadlinerTask::start` and a custom future-deadline `DeadlineCalculator` (pluto's own public `NeverExpiringCalculator` returns `NoDeadline`, which `handle`'s strict `!= Scheduled` check treats as `Error::DutyExpired` — a real gotcha, not a doc typo). See Findings for the `4n` cap as its own hardening decision. | | 8 | DECIDED-resend limiting | `qbft_decided_resends` | done | ABSENT-OK (pin reading PASS; spec-reading mismatches reported per case) | Both cases run against the real `pluto_core::qbft::run` state machine (not code inspection alone), driven to DECIDED at round 1 via the minimal justified PRE-PREPARE + quorum PREPARE + quorum COMMIT sequence, then fed each case's ROUND-CHANGE event sequence one at a time. Pin reading (pluto's actual, unconditional rule — rebroadcast DECIDED on every post-decision ROUND-CHANGE from a source other than the deciding process, no cap, no round check) holds for all 28 events across both cases: PASS. Spec reading (the vector's per-event `rebroadcast` expectations, which encode a 16-per-source cap plus a strictly-increasing-round-per-source dedup) diverges on 4/7 events in `dedup_duplicates_and_stale_rounds` (the duplicate-round and stale-round entries) and 5/21 in `resend_cap_per_source` (the 17th-through-21st events, past pluto's absent 16-cap) — reported, not asserted, matching ladder entry "QBFT DECIDED-resend rate limit and message size/count limits" (`first_charon_release: null`). `SomeMsg`/`Definition`/`Transport`/`QbftLogger` were externally implementable exactly as the probe doc claimed, confirming `run` is fully reachable from an external crate. | | 9 | Sender binding + peer map | `parsigex_sender_binding` | done | `cases`: ABSENT-OK; `peer_map`: UNREACHABLE | `cases` (6/6 run) driven for real through `pluto_parsigex::new_eth2_verifier` (a genuine BLS-signed `SignedVoluntaryExit`, resolved against a `BeaconMock`): its `Verifier` closure has no sender parameter at all, so acceptance depends solely on whether the claimed `share_idx` is a registered key with a genuinely valid signature. 4/6 cases coincide with the vector (`own_share_index_accepted`, `assigned_non_contiguous_share_index_accepted`, `mismatched_share_index_rejected`, `non_positive_share_index_rejected`); 2/6 diverge (`another_peers_share_index_rejected`, `unknown_sender_rejected`) because pluto accepts a genuinely-signed but wrongly-attributed share index — pinned against ladder entry "Sender-bound share indices in the DKG lock-hash exchange" (`first_charon_release: null`). `peer_map` (3/3 enumerated, not runnable): every pluto function that builds or validates a peer/share-index map is either arithmetically position-derived (`Peer::share_idx` = `position + 1`, `crates/p2p/src/peer.rs`, used by the reachable `Definition::node_idx`, `crates/cluster/src/definition.rs`) and so cannot even represent the vectors' non-contiguous assignment (share index 4 for the second of two peers), or private/`pub(crate)` with no external entry point (`crates/dkg/src/node.rs::setup_p2p`, `crates/dkg/src/frostp2p/transport.rs::validate_peer_share_indices`, `crates/app/src/node/mod.rs::build_pub_shares_by_key`). This predates charon v1.7.1 with no ladder cover, so it is a genuine coverage gap, not excused as ABSENT-OK. | -| 10 | Coverage guard + docs | all | todo | — | | +| 10 | Coverage guard + docs | all | done | — | `consumers/rust/tests/coverage.rs` asserts the published `test_vectors/*.json` suites match the `COVERED` table and that each declared test file exists, mirroring the Go consumer's `TestEverySuiteIsCovered`. `tests/test_consumers.py` mirrors the same check from this side (the Rust code is not compiled here, same as the Go side). `consumers/README.md` and this plan's Results table now carry the final per-suite verdicts against pluto `67088a2`. | ## Findings From 6a96c8d80bfc70b1feaa58e976d25f94fa66a9f9 Mon Sep 17 00:00:00 2001 From: Andrei Smirnov Date: Sat, 1 Aug 2026 07:16:36 +0300 Subject: [PATCH 22/27] Skip Cargo build output in the spellcheck gate --- tox.ini | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tox.ini b/tox.ini index 9acbe08..769bf0d 100644 --- a/tox.ini +++ b/tox.ini @@ -45,7 +45,7 @@ commands = mypy src tests scripts [testenv:spellcheck] description = Run spell checking (codespell) extras = docs -commands = codespell src tests scripts docs test_vectors consumers proto charon_anchor.json README.md CLAUDE.md --skip="*.lock,*.svg,.git,__pycache__,.mypy_cache,.pytest_cache" --ignore-words=.codespell-ignore-words.txt +commands = codespell src tests scripts docs test_vectors consumers proto charon_anchor.json README.md CLAUDE.md --skip="*.lock,*.svg,.git,__pycache__,.mypy_cache,.pytest_cache,*/target/*" --ignore-words=.codespell-ignore-words.txt [testenv:pytest] description = Run tests (pytest) From ac81aab5c8f430b3af2b8e14c197be6b9bebb388 Mon Sep 17 00:00:00 2001 From: Andrei Smirnov Date: Sat, 1 Aug 2026 07:41:38 +0300 Subject: [PATCH 23/27] Add proposal_timeout entry to the behaviour ladder The timer_deadlines vectors assume charon's proposal_timeout feature is enabled (stable since v1.9.0, charon #4296), but the behaviour ladder had no entry for it, so an implementation pinned to an older charon (pluto at v1.7.1) had nothing excusing the 9 diverging PROPOSER/round-1 cases. Record it in charon_anchor.json, the README compatibility table, and the consensus.md timer prose, which omitted the proposer round-1 exception the reference implementation and vectors already carried. Co-Authored-By: Claude Fable 5 --- README.md | 1 + charon_anchor.json | 6 ++++++ docs/dv-spec/consensus.md | 5 ++++- plans/pluto-conformance.md | 5 +++++ 4 files changed, 16 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index bc8fdd3..664dfe1 100644 --- a/README.md +++ b/README.md @@ -43,6 +43,7 @@ targets `v1.7.1`): some specified behaviors landed after `v1.7.1`: | ------------------------------------------------------------------ | -------------------- | | `MsgSync.nickname` field (DKG sync) | `v1.9.0` | | Deterministic (genesis-derived) eager double linear round deadlines | `v1.9.0` | +| Extended 1.5s proposer round-1 timeout (`proposal_timeout`) | `v1.9.0` | | Linear round timer subsequent-round timeout fix | unreleased (`v1.11.0` RCs) | | QBFT DECIDED-resend rate limit and message size/count limits | unreleased (`v1.11.0` RCs) | | Preferred priority protocol ID `/charon/priority/2.0.0` | unreleased (`main`) | diff --git a/charon_anchor.json b/charon_anchor.json index 56f15c0..fc40d53 100644 --- a/charon_anchor.json +++ b/charon_anchor.json @@ -83,6 +83,12 @@ "first_charon_release": "v1.9.0", "spec": "docs/dv-spec/consensus.md" }, + { + "name": "Extended 1.5s proposer round-1 timeout (proposal_timeout)", + "first_charon_release": "v1.9.0", + "spec": "docs/dv-spec/consensus.md", + "note": "The behaviour itself is older (charon #3739) but sat behind the alpha proposal_timeout feature flag; v1.9.0 (charon #4296) enabled it by default. The timer_deadlines vectors assume it is enabled." + }, { "name": "Linear round timer subsequent-round timeout fix", "first_charon_release": null, diff --git a/docs/dv-spec/consensus.md b/docs/dv-spec/consensus.md index 6e101af..55a2acd 100644 --- a/docs/dv-spec/consensus.md +++ b/docs/dv-spec/consensus.md @@ -66,7 +66,10 @@ leader_index = (duty.slot + duty.type + round) % n - On timeout, nodes trigger a round change and increment the round - Timer/backoff strategy does not affect safety, but must guarantee eventual progress - The default strategy is the *eager double linear* timer: round `r` lasts - `r` seconds, and the first deadline of each round is computed + `r` seconds — except round 1 of a proposer duty, which lasts 1.5 seconds + (block proposals need longer to fetch and agree on a value; see + `PROPOSAL_ROUND_TIMEOUT` in the reference implementation). The first + deadline of each round is computed deterministically from the chain (`genesis_time + slot * slot_duration + duty_start_delay + timeout`) rather than the local clock, so round boundaries — and hence leader election — stay aligned across all peers. diff --git a/plans/pluto-conformance.md b/plans/pluto-conformance.md index 4dae1e5..3452207 100644 --- a/plans/pluto-conformance.md +++ b/plans/pluto-conformance.md @@ -215,6 +215,11 @@ whether a ladder entry covers it, and whether it was reported upstream._ compatibility table documents the ladder and is enforced by `tests/test_release.py`, so the entry and the README update need to land together, in their own commit — not folded into this conformance task. + **Addressed on this branch:** `charon_anchor.json` now carries "Extended + 1.5s proposer round-1 timeout (proposal_timeout)" (`v1.9.0`), README's + compatibility table has the matching row, and `docs/dv-spec/consensus.md`'s + timer section now states the proposer round-1 exception the vectors and + reference implementation (`PROPOSAL_ROUND_TIMEOUT`) already carried. - **Pluto's justification-count cap is `4n`, not the spec's `2n` (task 7, `qbft_msg_limits/counts`).** `Consensus::handle` From 1a5a5813864305832d066d7b510e74dba0c9643a Mon Sep 17 00:00:00 2001 From: Andrei Smirnov Date: Sat, 1 Aug 2026 07:56:02 +0300 Subject: [PATCH 24/27] Correct review-found comment inaccuracies, sharpen findings Re-validation of every recorded divergence (differential Go probes in the charon checkout, layered serde probes, and a source audit of pluto 67088a2) confirmed all findings. Three documentation defects surfaced in review: - qbft_msg_limits.rs presented a paraphrase of test_vectors/README.md's slug rule as a verbatim quote; reworded as a paraphrase. - qbft_decided_resends.rs claimed pluto's own test reaches quorum COMMIT; that test cancels at the node's own COMMIT broadcast, so the external COMMIT quorum is this harness's extension. Both comments now say so. - The plan attributed DKG's no-op parsigex verifier to Exchanger; the production wiring is in dkg/src/node.rs::setup_p2p (Exchanger's no-op verifiers are cfg(test)-only). Also sharpen the cluster_hashing finding: a layered probe proved the all_empty_lists blocker set is exactly operators/validators null-handling plus the missing timestamp default, and tick the plan's completed step checkboxes. Co-Authored-By: Claude Fable 5 --- consumers/rust/tests/cluster_hashing.rs | 8 +- consumers/rust/tests/qbft_decided_resends.rs | 21 +++-- consumers/rust/tests/qbft_msg_limits.rs | 3 +- plans/pluto-conformance.md | 95 +++++++++++--------- 4 files changed, 70 insertions(+), 57 deletions(-) diff --git a/consumers/rust/tests/cluster_hashing.rs b/consumers/rust/tests/cluster_hashing.rs index 7366704..40142e4 100644 --- a/consumers/rust/tests/cluster_hashing.rs +++ b/consumers/rust/tests/cluster_hashing.rs @@ -22,9 +22,11 @@ use spec_vectors_pluto::load_suite; /// Whoever sees this test go red after a pluto change should expect to find /// the *next* blocker, not assume the case now passes end to end — only after /// (1) and (2) both resolve does `all_empty_lists` belong back in the strict -/// `definition_hashes` loop. See the cluster_hashing Results row / Findings -/// entry for the full field list (a lower bound, not a total — later fields -/// are never reached while an earlier one blocks parsing). +/// `definition_hashes` loop. A layered probe confirmed (1) and (2) are the +/// complete blocker set *for this case* (`name` and `deposit_amounts` are +/// already handled), but charon `omitempty` fields this case does not +/// exercise may hide more instances of the same root cause. See the +/// cluster_hashing Results row / Findings entry. const DEFINITION_KNOWN_DIVERGENCE: &str = "all_empty_lists"; /// `validator_without_deposit_data` is the lock-side counterpart: charon's diff --git a/consumers/rust/tests/qbft_decided_resends.rs b/consumers/rust/tests/qbft_decided_resends.rs index 3ceabc2..c78236c 100644 --- a/consumers/rust/tests/qbft_decided_resends.rs +++ b/consumers/rust/tests/qbft_decided_resends.rs @@ -154,12 +154,14 @@ impl SomeMsg for TestMsg { } } -/// Runs a `qbft::run` instance on a background thread, feeds it the minimal -/// justified message sequence pluto's own `broadcast_request_maps_protocol_fields` -/// test uses (`crates/core/src/qbft/internal_test.rs`) to reach quorum -/// PREPARE and quorum COMMIT at round 1, then confirms decision was actually -/// reached before handing control back -- so a harness bug (failure to -/// decide) can never be mistaken for a conformance result. +/// Runs a `qbft::run` instance on a background thread and drives it to a +/// round-1 decision: a justified PRE-PREPARE plus a quorum of PREPAREs, +/// built the way pluto's own `broadcast_request_maps_protocol_fields` test +/// (`crates/core/src/qbft/internal_test.rs`) builds them -- that test stops +/// at the node's own COMMIT broadcast, so the quorum of external COMMITs +/// that actually reaches DECIDED is this harness's own extension. Decision +/// is then confirmed before handing control back -- so a harness bug +/// (failure to decide) can never be mistaken for a conformance result. struct Harness { receive_tx: mpmc::Sender>, decided_rx: mpmc::Receiver<()>, @@ -251,9 +253,10 @@ fn spawn_decided_instance(nodes: i64, decided_round: i64) -> Harness { }); // Minimal justified sequence to reach DECIDED at round 1: 1 PRE-PREPARE - // from the leader, a quorum of PREPAREs, then a quorum of COMMITs - // (mirrors `broadcast_request_maps_protocol_fields`, - // `crates/core/src/qbft/internal_test.rs`). + // from the leader and a quorum of PREPAREs (message construction as in + // pluto's `broadcast_request_maps_protocol_fields`, + // `crates/core/src/qbft/internal_test.rs`), then a quorum of COMMITs, + // which that test never sends -- it cancels at the node's own COMMIT. receive_tx .send(TestMsg::pre_prepare(leader, 1, VALUE)) .expect("receive channel open"); diff --git a/consumers/rust/tests/qbft_msg_limits.rs b/consumers/rust/tests/qbft_msg_limits.rs index 4b7b50d..e8d39e5 100644 --- a/consumers/rust/tests/qbft_msg_limits.rs +++ b/consumers/rust/tests/qbft_msg_limits.rs @@ -394,7 +394,8 @@ fn qbft_consensus_msg_of_encoded_len(target_len: usize) -> pbconsensus::QbftCons /// `futures::io::Cursor` in-memory stream type. /// /// Matching on `err.to_string()` below is in tension with -/// `test_vectors/README.md`'s "the slugs are the contract, wording is not": +/// `test_vectors/README.md`'s rule that the slugs are the contract and the +/// wording of an error message is not: /// it is unavoidable here because every rejection path in /// `read_protobuf_with_max_size` reports through the same `io::ErrorKind::InvalidData` /// with no structured reason code, and pluto's own `p2p.rs` tests assert the diff --git a/plans/pluto-conformance.md b/plans/pluto-conformance.md index 3452207..755c730 100644 --- a/plans/pluto-conformance.md +++ b/plans/pluto-conformance.md @@ -158,8 +158,13 @@ whether a ladder entry covers it, and whether it was reported upstream._ `timestamp`". Confirmed directly: mutating `all_empty_lists`'s `operators`/ `validators` from `null` to `[]` (simulating a fix to (1)) still fails to parse on the missing `timestamp` key; adding an empty `timestamp` string on - top of that mutation parses successfully, with no further blocker found in - that specific probe (not a proof no more exist elsewhere in the struct). No + top of that mutation parses successfully. Re-confirmed 2026-08-01 by a + layered probe: for *this case* the blocker set is exactly these three + fields — `name` (absent from the JSON) is already covered by + `#[serde(default)]` and `deposit_amounts: null` by `DepositAmountsSerde`, + so the fix for `all_empty_lists` is three `#[serde(default)]` attributes. + Charon `omitempty` fields this case does not exercise may still hide more + instances of the same root cause elsewhere in the structs. No `charon_anchor.json` ladder entry covers any of this — the ladder tracks *protocol behaviour* that postdates charon v1.7.1, and all of these field tags are unchanged, long-standing charon serialization behaviour, not a recent @@ -294,8 +299,10 @@ whether a ladder entry covers it, and whether it was reported upstream._ (`unknown_sender_rejected`). Ladder entry "Sender-bound share indices in the DKG lock-hash exchange" (`first_charon_release: null`) covers this exactly — no released charon carries the check either, so this is ABSENT-OK, not reported upstream. - Note that DKG's own lock-hash exchange does not even go through this verifier: `Exchanger` - (`crates/dkg/src/exchanger.rs`) wires parsigex with a no-op verifier and instead checks + Note that DKG's own lock-hash exchange does not even go through this verifier: DKG wires + parsigex with a no-op verifier in `crates/dkg/src/node.rs::setup_p2p` (`Exchanger` itself, + `crates/dkg/src/exchanger.rs`, receives the already-built parsigex handle; its own no-op + verifiers are `#[cfg(test)]`-only) and instead checks signatures post-hoc in `crates/dkg/src/aggregate.rs::agg_lock_hash_sig`/ `verify_threshold_partials` (same no-sender-parameter shape) — but `crates/dkg/src/lib.rs` declares `mod aggregate;` without `pub`, so that code is not reachable from an external @@ -372,7 +379,7 @@ The smallest suite against the most-public pluto API (`pluto-k1util`: everything - Produces: `spec_vectors_pluto::load_suite(name: &str) -> serde_json::Value` and `spec_vectors_pluto::unhex(s: &str) -> Vec` — every later task consumes these. -- [ ] **Step 1: Write the package scaffold** +- [x] **Step 1: Write the package scaffold** `consumers/rust/Cargo.toml` (copy the exact version requirements for `serde_json`, `hex`, `k256` from `~/pluto/Cargo.toml` `[workspace.dependencies]`): @@ -445,7 +452,7 @@ pub fn unhex(s: &str) -> Vec { } ``` -- [ ] **Step 2: Write the failing test** +- [x] **Step 2: Write the failing test** `consumers/rust/tests/secp256k1_signatures.rs`. The suite carries one secret key and cases of `{input.hash_hex, signature_hex, recovered_pubkey_hex}` — signing is RFC 6979 @@ -496,7 +503,7 @@ print(json.load(open('test_vectors/secp256k1_signatures.json'))['pubkey_hex'])"` if it is 33 bytes it is SEC1 compressed and `to_sec1_bytes()` matches; if 65 bytes, use `k256::EncodedPoint::from(&rec).to_untagged_bytes()` accordingly. -- [ ] **Step 3: Run it** +- [x] **Step 3: Run it** ```bash cd consumers/rust && cargo test --test secp256k1_signatures @@ -504,11 +511,11 @@ cd consumers/rust && cargo test --test secp256k1_signatures Expected: compiles against `~/pluto` and all cases pass (charon and pluto share RFC 6979 + low-S via k256). Any red case is a FAIL finding — triage per the taxonomy. -- [ ] **Step 4: Record the verdict** +- [x] **Step 4: Record the verdict** Fill the pluto commit under test and row 1 of the Results table in this plan. -- [ ] **Step 5: Update `consumers/README.md` and commit** +- [x] **Step 5: Update `consumers/README.md` and commit** Add to the consumer table: `| Pluto (Rust) | in progress — see plans/pluto-conformance.md | [rust/](rust) |`, plus a short "Pluto consumer" section: sibling-checkout requirement, the @@ -536,7 +543,7 @@ git commit -m "Add Rust consumer scaffold and secp256k1 suite for pluto" `~/pluto/crates/core/src/lib.rs` before writing; the golden test at `~/pluto/crates/consensus/src/qbft/msg.rs:420` shows working imports to copy). -- [ ] **Step 1: Add dependencies** +- [x] **Step 1: Add dependencies** ```toml pluto-consensus = { path = "../../../pluto/crates/consensus" } @@ -545,7 +552,7 @@ prost = "" prost-types = "" ``` -- [ ] **Step 2: Write the test — four groups, one test fn each** +- [x] **Step 2: Write the test — four groups, one test fn each** ```rust use prost::Message; @@ -665,7 +672,7 @@ fn any_string_hashing() { } ``` -- [ ] **Step 3: Run it** +- [x] **Step 3: Run it** ```bash cargo test --test qbft_hashing @@ -675,12 +682,12 @@ golden file and none of these values moved after v1.7.1. If the `any_string` *encoding* diverges (type URL prefix differs), decode the vector's `encoding_hex` to see charon's actual type URL and fix the constructed `type_url`, not the assertion. -- [ ] **Step 4: Record verdict in Results row 2; note in the row that pluto's own +- [x] **Step 4: Record verdict in Results row 2; note in the row that pluto's own `crates/consensus/testdata/vectors/hashproto.json` is now redundant with this suite (the stated goal of the qbft_hashing suite) — a candidate upstream cleanup, not something this repo changes.** -- [ ] **Step 5: Commit** +- [x] **Step 5: Commit** ```bash git add consumers/rust plans/pluto-conformance.md @@ -701,7 +708,7 @@ git commit -m "Run qbft_hashing suite against pluto" (all trait methods pub); `pluto_crypto::types::{PublicKey, PrivateKey, Signature, Index}` are plain byte arrays. Share indices are 1-based, matching the suite. -- [ ] **Step 1: Write the test** +- [x] **Step 1: Write the test** The suite pins: per-share pubkeys and partial signatures over `message_hex`, four different quorums whose threshold aggregates must equal the group signature, secret @@ -794,10 +801,10 @@ fn bls_threshold() { } ``` -- [ ] **Step 2: Run** `cargo test --test bls_threshold`. Expected: PASS — pluto's +- [x] **Step 2: Run** `cargo test --test bls_threshold`. Expected: PASS — pluto's `blst_impl` cites charon v1.7.1 parity and BLS did not change after it. -- [ ] **Step 3: Record verdict in Results row 3; commit** +- [x] **Step 3: Record verdict in Results row 3; commit** (`git commit -m "Run bls_threshold suite against pluto"`). --- @@ -817,7 +824,7 @@ fn bls_threshold() { async `verify_signatures(&EthClient)`). Raw hash functions are `pub(crate)`; going through set-then-compare covers the same code. -- [ ] **Step 1: Write the test** +- [x] **Step 1: Write the test** ```rust use spec_vectors_pluto::load_suite; @@ -881,7 +888,7 @@ Before writing the field names for expected hashes, check the suite's actual key print(list(c['definition'][0]), list(c['lock'][0]))"`) — the plan's `config_hash_hex`/`definition_hash_hex`/`lock_hash_hex` names must match what is there. -- [ ] **Step 2: Add the full-verification check for `real_keys_3_of_4`** +- [x] **Step 2: Add the full-verification check for `real_keys_3_of_4`** In the same file — this is the case that chains lock hash, public shares, plain BLS aggregation and node signatures; only it carries real signatures: @@ -907,13 +914,13 @@ async fn real_keys_lock_verifies_end_to_end() { name and constructor; if `verify_signatures` demands a live EL for these operators' sig type, record that limitation in the Results row instead of mocking one.) -- [ ] **Step 3: Run** `cargo test --test cluster_hashing`. Expected: v1.10 files parse +- [x] **Step 3: Run** `cargo test --test cluster_hashing`. Expected: v1.10 files parse (pluto supports V1_0…V1_10) and all hashes match — pluto's own testdata includes charon's `cluster_lock_v1_10_0.json` golden. Parse *failures* here are findings: the inputs are verbatim charon files, so a rejection means pluto cannot load a real charon artifact. -- [ ] **Step 4: Record verdict in Results row 4; commit** +- [x] **Step 4: Record verdict in Results row 4; commit** (`git commit -m "Run cluster_hashing suite against pluto"`). --- @@ -937,7 +944,7 @@ libp2p network — pluto's own `crates/priority/tests/prioritiser_test.rs` is an `Consensus` trait (implement a mock capturing the proposed `PriorityResult`), `MsgVerifier` (`Box::new(|_| Ok(()))`), proto types `PriorityMsg`, `PriorityTopicProposal`. -- [ ] **Step 1: Write the harness** +- [x] **Step 1: Write the harness** Shape (helpers copied from `prioritiser_test.rs` — that file compiles against public API only, so everything it does is available here): @@ -972,7 +979,7 @@ if present, order-only otherwise. Cases where `result` is `[]` assert the topic absent (below `min_required`) — an empty expected list is a *rejection* pin, not a skip. -- [ ] **Step 2: Add the protocol-ID pin (ladder item 6)** +- [x] **Step 2: Add the protocol-ID pin (ladder item 6)** ```rust #[test] @@ -986,12 +993,12 @@ fn priority_protocol_ids() { } ``` -- [ ] **Step 3: Run** `cargo test --test priority_scoring`. Expected: scoring cases +- [x] **Step 3: Run** `cargo test --test priority_scoring`. Expected: scoring cases PASS (the algorithm predates v1.7.1 and pluto's stable sort matches the spec). Watch the ties: charon v1.7.1 used an unstable sort — if any tie-case diverges, that is ladder entry "Stable sort of scored priorities" → triage before verdicting. -- [ ] **Step 4: Record verdict in Results row 5; commit** +- [x] **Step 4: Record verdict in Results row 5; commit** (`git commit -m "Run priority_scoring suite against pluto"`). --- @@ -1017,7 +1024,7 @@ The vectors give `duty_start_delay_nanos`, `round_timeout_nanos` and absolute add `pluto-featureset = { path = "../../../pluto/crates/featureset" }` if `FeatureSet` lives there) -- [ ] **Step 1: Write the test** +- [x] **Step 1: Write the test** ```rust use std::time::Duration; @@ -1054,7 +1061,7 @@ the timer future under a paused clock and copy that pattern — if the future re to its deadline `Instant` without needing `tokio::time::advance`, the above works; if it sleeps, wrap with `tokio::time::timeout` + `advance`. -- [ ] **Step 2: Determine the two remaining columns' verdicts** +- [x] **Step 2: Determine the two remaining columns' verdicts** Read `EagerDoubleLinearRoundTimer::with_duty` and its callers: if no genesis/slot timing enters the deadline, mark `deadline_nanos` ABSENT-OK (ladder v1.9.0 entry) @@ -1062,13 +1069,13 @@ and add a pinned-divergence comment in the test naming the entry. Confirm `delay_slot_offset` is still private → mark `duty_start_delay_nanos` UNREACHABLE with file:line. -- [ ] **Step 3: Run** `cargo test --test timer_deadlines`. Triage: mismatches on +- [x] **Step 3: Run** `cargo test --test timer_deadlines`. Triage: mismatches on linear rounds ≥ 2 are the known nanosecond bug (ladder "Linear round timer subsequent-round timeout fix", `null`) — if the vectors' proposer/linear cases hit it, split those cases into a pinned-divergence check asserting pluto's 200ns schedule, named for the ladder entry, and keep the rest strict. -- [ ] **Step 4: Record verdict in Results row 6 (verdict per column: +- [x] **Step 4: Record verdict in Results row 6 (verdict per column: timeout / deadline / start-delay); commit** (`git commit -m "Run timer_deadlines suite against pluto"`). @@ -1088,7 +1095,7 @@ with file:line. `sign_msg`: clear signature → `hash_proto` → `pluto_k1util::sign` (Task 2 proved the root; Task 1 proved the signature). -- [ ] **Step 1: Write the `counts` half** +- [x] **Step 1: Write the `counts` half** ```rust // Per `counts` case (input: nodes, justification_count, value_count): @@ -1116,7 +1123,7 @@ with file:line. // breaks liveness and no ladder entry can excuse it. ``` -- [ ] **Step 2: Write the `wire_size` half** +- [x] **Step 2: Write the `wire_size` half** ```rust #[test] @@ -1137,11 +1144,11 @@ async fn wire_size_enforcement() { } ``` -- [ ] **Step 3: Run** `cargo test --test qbft_msg_limits`, triage every red case +- [x] **Step 3: Run** `cargo test --test qbft_msg_limits`, triage every red case against the pre-registered expectations — anything *outside* them (e.g. an at-limit case rejected) is a FAIL finding. -- [ ] **Step 4: Record verdict in Results row 7 and the 4n-cap finding under +- [x] **Step 4: Record verdict in Results row 7 and the 4n-cap finding under Findings; commit** (`git commit -m "Run qbft_msg_limits suite against pluto"`). --- @@ -1161,7 +1168,7 @@ rebroadcasts on every post-decision ROUND-CHANGE from another source). Ladder en MSG_ROUND_CHANGE, MSG_DECIDED, SomeMsg}` — all public with public fields; implement `QbftTypes` and `SomeMsg` for a minimal test message type. -- [ ] **Step 1: Write the harness** +- [x] **Step 1: Write the harness** ```rust // Per case (input: nodes, decided_round, events[{source, round}], rebroadcast[i]): @@ -1187,7 +1194,7 @@ rebroadcasts on every post-decision ROUND-CHANGE from another source). Ladder en // The test passes on the pin reading and prints the spec-reading mismatch count. ``` -- [ ] **Step 2: Run** `cargo test --test qbft_decided_resends`. +- [x] **Step 2: Run** `cargo test --test qbft_decided_resends`. Fallback, only if driving `run` externally proves impractical after a real attempt (e.g. `SomeMsg` cannot be implemented outside the crate): verdict the row @@ -1195,7 +1202,7 @@ UNREACHABLE, cite `crates/core/src/qbft/mod.rs:494-503` as the inspection eviden for ABSENT-OK, and leave the test file in place with `#[ignore]` and a comment explaining what blocked it. -- [ ] **Step 3: Record verdict in Results row 8; commit** +- [x] **Step 3: Record verdict in Results row 8; commit** (`git commit -m "Run qbft_decided_resends suite against pluto"`). --- @@ -1216,7 +1223,7 @@ construction is reachable. - Modify: `consumers/rust/Cargo.toml` (add `pluto-parsigex`, and `pluto-dkg` if the exchanger lives there) -- [ ] **Step 1: Probe and write the test** +- [x] **Step 1: Probe and write the test** ```rust // cases (input: share_idx_by_peer {self, other}, sender, share_idx; accepted, reason): @@ -1234,11 +1241,11 @@ construction is reachable. // (crates/app/src/node/mod.rs:800), verdict that half UNREACHABLE with file:line. ``` -- [ ] **Step 2: Run** `cargo test --test parsigex_sender_binding`, triage: a missing +- [x] **Step 2: Run** `cargo test --test parsigex_sender_binding`, triage: a missing *peer-map* validation (accepting an incomplete map) is a FAIL finding — charon v1.7.1 already rejects it, no ladder cover. Sender-binding absence is ABSENT-OK. -- [ ] **Step 3: Record verdict in Results row 9; commit** +- [x] **Step 3: Record verdict in Results row 9; commit** (`git commit -m "Run parsigex_sender_binding suite against pluto"`). --- @@ -1251,7 +1258,7 @@ construction is reachable. - Modify: `tests/test_consumers.py` (extend to the Rust consumer) - Modify: `plans/spec-completion.md` (Phase 3 item 2 pluto bullet → point here) -- [ ] **Step 1: Write the coverage guard** +- [x] **Step 1: Write the coverage guard** The Go consumer's `TestEverySuiteIsCovered` has an equivalent here: an uncovered suite is indistinguishable from a passing one. @@ -1292,7 +1299,7 @@ fn every_suite_is_covered() { } ``` -- [ ] **Step 2: Extend `tests/test_consumers.py`** +- [x] **Step 2: Extend `tests/test_consumers.py`** Mirror the existing Go mapping check: assert each of the nine suites appears in `consumers/rust/tests/coverage.rs`'s `COVERED` table and that each declared test @@ -1300,7 +1307,7 @@ file exists — read the file with a regex over the `("suite", "file")` pairs, t same technique the file already uses for the Go side (the Rust code is not compiled here either). Run `uv run pytest tests/test_consumers.py` and confirm green. -- [ ] **Step 3: Finalize docs** +- [x] **Step 3: Finalize docs** - `consumers/README.md`: Pluto row lists suites run / subcheck count / pluto commit / verdict summary, and a "running it" snippet: @@ -1311,14 +1318,14 @@ here either). Run `uv run pytest tests/test_consumers.py` and confirm green. - `plans/spec-completion.md` Phase 3 item 2: change the pluto bullet from "not started" to a pointer at this plan and its Results table. -- [ ] **Step 4: Full run + quality checks** +- [x] **Step 4: Full run + quality checks** ```bash cd consumers/rust && cargo fmt --check && cargo clippy --all-targets && cargo test cd ../.. && uv run tox -e all-checks && uv run pytest tests/test_consumers.py ``` -- [ ] **Step 5: Record Results row 10, finalize the Findings section (each finding: +- [x] **Step 5: Record Results row 10, finalize the Findings section (each finding: divergence, pluto file:line, ladder cover or not, reported upstream or not), commit, and open the PR** From f6dfe27a0116b9dba7476d8b3f495ceae9b40639 Mon Sep 17 00:00:00 2001 From: Andrei Smirnov Date: Sat, 1 Aug 2026 07:58:37 +0300 Subject: [PATCH 25/27] Clarify what the Rust coverage guard checks vs the Go one Go's TestEverySuiteIsCovered also asserts each suite ships cases, read from its vendored release manifest. The Rust consumer reads test_vectors/ live with no manifest; each suite test pins its exact case count instead. Say so rather than claiming unqualified equivalence. Co-Authored-By: Claude Fable 5 --- consumers/rust/tests/coverage.rs | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/consumers/rust/tests/coverage.rs b/consumers/rust/tests/coverage.rs index 1683f7e..cb239f8 100644 --- a/consumers/rust/tests/coverage.rs +++ b/consumers/rust/tests/coverage.rs @@ -2,7 +2,11 @@ //! //! An uncovered suite is indistinguishable from a passing one: the Go consumer's //! `TestEverySuiteIsCovered` (`consumers/go/testutil/specvectors/specvectors.go`) -//! catches this for Charon; this is the equivalent for the Rust side. +//! catches this for Charon; this is the equivalent for the Rust side. Go's +//! version also asserts each suite ships cases, read from its vendored +//! release manifest; this consumer reads `test_vectors/` live and has no +//! manifest, so non-emptiness is enforced by each suite test pinning its +//! exact case count instead. use std::collections::BTreeSet; From f58f1e6592b65b68a5e21375fe7a5e48d0545292 Mon Sep 17 00:00:00 2001 From: Andrei Smirnov Date: Sat, 1 Aug 2026 08:08:19 +0300 Subject: [PATCH 26/27] Reclassify the default-config timer divergence as ABSENT-OK The proposal_timeout ladder entry added in ac81aab formally covers the 9 PROPOSER/round-1 cases that diverge under pluto's default FeatureSet, so the plan's Results row, its finding, the consumer README, and the timer test's comments now name that entry per the verdict taxonomy instead of calling it an uncovered known-divergence. Also record the 2026-08-01 re-validation pass in the plan: full suite re-run, both FAIL findings re-proven by differential execution, all pluto citations re-audited. Co-Authored-By: Claude Fable 5 --- consumers/README.md | 2 +- consumers/rust/tests/timer_deadlines.rs | 14 +++++++++----- plans/pluto-conformance.md | 22 ++++++++++++++++++++-- 3 files changed, 30 insertions(+), 8 deletions(-) diff --git a/consumers/README.md b/consumers/README.md index a5f3e94..2d7b1cd 100644 --- a/consumers/README.md +++ b/consumers/README.md @@ -119,7 +119,7 @@ Findings section. | `bls_threshold` | PASS (all 5 groups) | | `cluster_hashing` | **FAIL** — pluto rejects charon-legitimate JSON shapes (`operators: null`, an absent `partial_deposit_data`, and a masked third gap on `timestamp`) | | `priority_scoring` | PASS (18/18) | -| `timer_deadlines` | `round_timeout_nanos` PASS **only with `ProposalTimeout` explicitly enabled** (9 PROPOSER/round-1 cases diverge under pluto's default config); `deadline_nanos` ABSENT-OK; `duty_start_delay_nanos` UNREACHABLE | +| `timer_deadlines` | `round_timeout_nanos` PASS **only with `ProposalTimeout` explicitly enabled** (9 PROPOSER/round-1 cases diverge under pluto's default config — ABSENT-OK, ladder entry "Extended 1.5s proposer round-1 timeout (proposal_timeout)"); `deadline_nanos` ABSENT-OK; `duty_start_delay_nanos` UNREACHABLE | | `qbft_msg_limits` | `counts`: 5 MATCH + 6 ABSENT-OK; `wire_size`: 4/4 MATCH | | `qbft_decided_resends` | ABSENT-OK (pluto has no DECIDED-rebroadcast limiter) | | `parsigex_sender_binding` | `cases` ABSENT-OK; `peer_map` UNREACHABLE | diff --git a/consumers/rust/tests/timer_deadlines.rs b/consumers/rust/tests/timer_deadlines.rs index 8fc7da2..9c5a564 100644 --- a/consumers/rust/tests/timer_deadlines.rs +++ b/consumers/rust/tests/timer_deadlines.rs @@ -36,9 +36,11 @@ //! So a default-configured pluto disagrees with the vectors on every //! `PROPOSER`/round-1 case (pluto: 1s; vectors: 1.5s), and that disagreement //! is the same class of gap as the `deadline_nanos` column below: a v1.9.0 -//! charon behaviour that postdates pluto's pin. `round_timeouts_pluto_default_feature_set` -//! pins it explicitly instead of letting the enabled-feature test's PASS -//! stand in for it. +//! charon behaviour that postdates pluto's pin, ABSENT-OK against ladder +//! entry "Extended 1.5s proposer round-1 timeout (proposal_timeout)" +//! (`charon_anchor.json`, `first_charon_release: v1.9.0`). +//! `round_timeouts_pluto_default_feature_set` pins it explicitly instead of +//! letting the enabled-feature test's PASS stand in for it. use std::collections::HashSet; use std::sync::Arc; @@ -160,8 +162,10 @@ async fn round_timeouts_pluto_default_feature_set() { // v1.7.1 parity (pluto's default) is 1s, not the vectors' 1.5s. See the // module doc comment ("Fix round 1") for why -- charon promoted // `ProposalTimeout` to enabled-by-default in v1.9.0 (#4296), after - // pluto's anchor. This must flip loudly (the `assert_eq!` below fails) - // the moment pluto's default configuration changes in either direction. + // pluto's anchor; ladder entry "Extended 1.5s proposer round-1 timeout + // (proposal_timeout)" covers it. This must flip loudly (the `assert_eq!` + // below fails) the moment pluto's default configuration changes in + // either direction. const PINNED_DEFAULT_PROPOSER_ROUND1_TIMEOUT: Duration = Duration::from_secs(1); let mut divergent_cases = 0usize; diff --git a/plans/pluto-conformance.md b/plans/pluto-conformance.md index 755c730..c6e2f05 100644 --- a/plans/pluto-conformance.md +++ b/plans/pluto-conformance.md @@ -65,6 +65,14 @@ pluto commit actually tested. Statuses: `todo` / `in progress` / `done`. **Pluto commit under test:** `67088a2` +**Re-validated 2026-08-01:** the full Rust suite re-run green against `67088a2`; +both FAIL-class findings re-proven by fresh differential execution (a charon +`hashProto` probe for the map-entry divergence, a layered serde probe for the +cluster-parsing divergence); every pluto file:line claim in the Findings +re-audited against source, with two attribution corrections folded back in +(the Exchanger/`setup_p2p` no-op-verifier wiring, and the `all_empty_lists` +blocker set now proven complete for that case). + | # | Check | Suite | Status | Verdict | Notes | | --- | --- | --- | --- | --- | --- | | 1 | Harness scaffold + secp256k1 | `secp256k1_signatures` | done | PASS | Both cases pass: sign, recover, and verify_65 all match the vector via `pluto-k1util` (33-byte SEC1 pubkey). | @@ -72,7 +80,7 @@ pluto commit actually tested. Statuses: `todo` / `in progress` / `done`. | 3 | BLS threshold aggregation | `bls_threshold` | done | PASS | All 5 groups PASS: `keys` (2/2), `partials` (4/4, pubshare + partial signature), `threshold_aggregates` (4/4 distinct 3-of-4 quorums, each reproducing the one `group_signature_hex`), `recovery` (1/1, `recover_secret` yields the exact `group_secret_hex`), `plain_aggregate` (1/1, matches the pinned bytes and does **not** verify under `group_pubkey_hex`, confirming plain aggregation is not a threshold-signature substitute). `group_signature_hex` also verifies under `group_pubkey_hex`. `pluto-crypto`'s `BlstImpl`/`Tbls` API matched the brief's guesses exactly; no divergence found. | | 4 | Cluster hashes + lock verification | `cluster_hashing` | done | FAIL (2 pinned known-divergence cases; at least 3 distinct affected fields — a lower bound, not a total) | `definition` 5/6 strict-PASS (`config_hash` and `definition_hash` both match on 5 cases, including the unsigned/signed pair proving config_hash is signature-independent); `all_empty_lists` is a real divergence, pinned (see Findings) — it carries **at least two independent parse blockers** (`operators`/`validators` null-rejection, then a masked `timestamp` missing-field rejection once the first is hypothetically fixed), so a partial upstream fix will not make it pass. `lock` 3/4 strict-PASS (`lock_hash` matches and `verify_hashes` succeeds on 3); `validator_without_deposit_data` is a second real divergence, pinned (see Findings). `real_keys_3_of_4`: hashes verify; the full `verify_signatures(&EthClient::new(""))` chain is asserted to fail specifically at the definition EIP-712 stage (`LockError::DefinitionSignaturesVerificationFailed`) because its operator/creator signatures are unavoidable placeholders (charon's EIP-712 signing helpers aren't exported for vector generation) — `Lock::verify_signatures` short-circuits there, so its private BLS-aggregate and node-signature checks are never reached from this external crate; that half is a coverage gap (not a failure), noted in the test's doc comment rather than mocked around. | | 5 | Priority scoring | `priority_scoring` | done | PASS | All 18/18 cases PASS end-to-end through the public `Prioritiser` API over an in-process libp2p (`MemoryTransport` + noise + yamux) network, with a mock `Consensus` capturing the proposed `PriorityResult` (`calculate_result` itself is `pub(crate)`, unreachable directly). Covers below-quorum rejection (empty-result pins), single-priority and two-priority scoring, count-then-relative-priority ordering, and the two `test-case`-name-only "deterministic ordering" cases (score ties resolved by first-seen order in peer-id-sorted input — confirmed stable, no divergence from Charon v1.7.1's own stable-sort assumption). Peer `PeerId`s are generated then sorted ascending and mapped onto the vector's peer indices, since scoring tie-breaks are first-seen over peer-id-sorted input. Also pins `PROTOCOL_ID`/`protocols()` to the legacy slash-less `charon/priority/2.0.0` (ladder: preferred slash form is unreleased). Full 18-case suite runs in under 1s (well under the 2-minute budget). | -| 6 | Round timer deadlines | `timer_deadlines` | done | round_timeout: PASS **only under `ProposalTimeout` enabled**, a documented non-default precondition (pluto's *default* config has a pinned, real divergence on `PROPOSER`/round-1 cases — see Notes and Findings); deadline: ABSENT-OK, with a real code-level slot-invariance pin (not just a comment); duty_start_delay: UNREACHABLE | `round_timeouts_with_proposal_timeout_enabled`: 216/216 PASS, but only with `Feature::ProposalTimeout` explicitly enabled (it is `Status::Alpha`, off under pluto's `Config::default()`). `round_timeouts_pluto_default_feature_set` runs the same 216 cases under pluto's actual default `FeatureSet` and pins the resulting divergence: the 207 non-`PROPOSER`-round-1 cases still match the vectors, but all 9 `PROPOSER`/round-1 cases (3 slots x 3 slot durations) diverge — pluto's default gives 1s, the vectors (which assume `ProposalTimeout` enabled) expect 1.5s. This is a real, pinned known-divergence (see Findings), not swept under the enabled-feature test's PASS. `deadline_nanos`: `round_timeout_is_duty_slot_invariant` builds two timers per (duty_type, round) combination (24 total, covering all cases) differing only in `slot` (0 vs. 7231) and asserts bit-identical measured timeouts — a real code pin for "no timer reads slot", not a doc comment, so it fails loudly if `with_duty`/`RoundTimer::timer` ever grows slot-dependence; genesis-invariance is structurally guaranteed (the API has no genesis parameter at all) and is stated as such rather than tested. Matches ladder entry "Deterministic (genesis-derived) eager double linear round deadlines" (`first_charon_release: v1.9.0` > pluto's v1.7.1 anchor). `duty_start_delay_nanos` is UNREACHABLE: `delay_slot_offset` (`crates/core/src/scheduler.rs`) is a non-`pub` `async fn` with no external caller, confirmed still private. | +| 6 | Round timer deadlines | `timer_deadlines` | done | round_timeout: PASS **only under `ProposalTimeout` enabled**, a documented non-default precondition (pluto's *default* config has a pinned divergence on `PROPOSER`/round-1 cases, ABSENT-OK since 2026-08-01 against ladder entry "Extended 1.5s proposer round-1 timeout (proposal_timeout)" — see Notes and Findings); deadline: ABSENT-OK, with a real code-level slot-invariance pin (not just a comment); duty_start_delay: UNREACHABLE | `round_timeouts_with_proposal_timeout_enabled`: 216/216 PASS, but only with `Feature::ProposalTimeout` explicitly enabled (it is `Status::Alpha`, off under pluto's `Config::default()`). `round_timeouts_pluto_default_feature_set` runs the same 216 cases under pluto's actual default `FeatureSet` and pins the resulting divergence: the 207 non-`PROPOSER`-round-1 cases still match the vectors, but all 9 `PROPOSER`/round-1 cases (3 slots x 3 slot durations) diverge — pluto's default gives 1s, the vectors (which assume `ProposalTimeout` enabled) expect 1.5s. This is a real, pinned known-divergence (see Findings), not swept under the enabled-feature test's PASS. `deadline_nanos`: `round_timeout_is_duty_slot_invariant` builds two timers per (duty_type, round) combination (24 total, covering all cases) differing only in `slot` (0 vs. 7231) and asserts bit-identical measured timeouts — a real code pin for "no timer reads slot", not a doc comment, so it fails loudly if `with_duty`/`RoundTimer::timer` ever grows slot-dependence; genesis-invariance is structurally guaranteed (the API has no genesis parameter at all) and is stated as such rather than tested. Matches ladder entry "Deterministic (genesis-derived) eager double linear round deadlines" (`first_charon_release: v1.9.0` > pluto's v1.7.1 anchor). `duty_start_delay_nanos` is UNREACHABLE: `delay_slot_offset` (`crates/core/src/scheduler.rs`) is a non-`pub` `async fn` with no external caller, confirmed still private. | | 7 | QBFT message limits | `qbft_msg_limits` | done | ABSENT-OK (counts, both directions); PASS (wire_size) | `counts` (11/11 cases run): pluto has neither the spec's `2n` justification cap nor any `2*(j+1)` values cap. Its own cap is `4 * node_count()` (component.rs `MAX_JUSTIFICATIONS_PER_NODE = 4`, strict `>`), and there is no values-length check anywhere in `crates/consensus/src`/`crates/core/src`. Every counts case's `justification_count` stays at/under pluto's `4n` (proven structurally in the test, not by case name), so pluto accepts all 11 cases: 5 match the spec's `accepted: true` cases outright, 6 are ABSENT-OK divergences (spec says reject — `too_many_justifications` or `too_many_values` — pluto accepts) against ladder entry "QBFT DECIDED-resend rate limit and message size/count limits" (`first_charon_release: null`). Two extra tests pin pluto's own real boundary directly, since no vector case reaches it: `4 * nodes` justifications accepted, `4 * nodes + 1` rejected with `Error::TooManyJustifications { count, max }` asserted exactly (not just "any error"). `wire_size` (4/4 cases): `MAX_CONSENSUS_MSG_SIZE` (32 MiB) and `MAX_MESSAGE_SIZE` (128 MiB) constants pinned; enforcement runs the real `pluto_p2p::proto::read_protobuf_with_max_size` reader (mirroring `qbft/p2p.rs`'s own unit tests) over `futures::io::Cursor` — 1024 bytes and exactly 32 MiB accepted (round-trip decode verified byte-for-byte), 32 MiB+1 and 128 MiB rejected with a "too large" reason, matching `msg_too_large` exactly. `DeadlinerHandle::always` from the probe doc turned out to be `#[cfg(test)]`-gated inside pluto_core (not reachable externally); worked around with the real public `DeadlinerTask::start` and a custom future-deadline `DeadlineCalculator` (pluto's own public `NeverExpiringCalculator` returns `NoDeadline`, which `handle`'s strict `!= Scheduled` check treats as `Error::DutyExpired` — a real gotcha, not a doc typo). See Findings for the `4n` cap as its own hardening decision. | | 8 | DECIDED-resend limiting | `qbft_decided_resends` | done | ABSENT-OK (pin reading PASS; spec-reading mismatches reported per case) | Both cases run against the real `pluto_core::qbft::run` state machine (not code inspection alone), driven to DECIDED at round 1 via the minimal justified PRE-PREPARE + quorum PREPARE + quorum COMMIT sequence, then fed each case's ROUND-CHANGE event sequence one at a time. Pin reading (pluto's actual, unconditional rule — rebroadcast DECIDED on every post-decision ROUND-CHANGE from a source other than the deciding process, no cap, no round check) holds for all 28 events across both cases: PASS. Spec reading (the vector's per-event `rebroadcast` expectations, which encode a 16-per-source cap plus a strictly-increasing-round-per-source dedup) diverges on 4/7 events in `dedup_duplicates_and_stale_rounds` (the duplicate-round and stale-round entries) and 5/21 in `resend_cap_per_source` (the 17th-through-21st events, past pluto's absent 16-cap) — reported, not asserted, matching ladder entry "QBFT DECIDED-resend rate limit and message size/count limits" (`first_charon_release: null`). `SomeMsg`/`Definition`/`Transport`/`QbftLogger` were externally implementable exactly as the probe doc claimed, confirming `run` is fully reachable from an external crate. | | 9 | Sender binding + peer map | `parsigex_sender_binding` | done | `cases`: ABSENT-OK; `peer_map`: UNREACHABLE | `cases` (6/6 run) driven for real through `pluto_parsigex::new_eth2_verifier` (a genuine BLS-signed `SignedVoluntaryExit`, resolved against a `BeaconMock`): its `Verifier` closure has no sender parameter at all, so acceptance depends solely on whether the claimed `share_idx` is a registered key with a genuinely valid signature. 4/6 cases coincide with the vector (`own_share_index_accepted`, `assigned_non_contiguous_share_index_accepted`, `mismatched_share_index_rejected`, `non_positive_share_index_rejected`); 2/6 diverge (`another_peers_share_index_rejected`, `unknown_sender_rejected`) because pluto accepts a genuinely-signed but wrongly-attributed share index — pinned against ladder entry "Sender-bound share indices in the DKG lock-hash exchange" (`first_charon_release: null`). `peer_map` (3/3 enumerated, not runnable): every pluto function that builds or validates a peer/share-index map is either arithmetically position-derived (`Peer::share_idx` = `position + 1`, `crates/p2p/src/peer.rs`, used by the reachable `Definition::node_idx`, `crates/cluster/src/definition.rs`) and so cannot even represent the vectors' non-contiguous assignment (share index 4 for the second of two peers), or private/`pub(crate)` with no external entry point (`crates/dkg/src/node.rs::setup_p2p`, `crates/dkg/src/frostp2p/transport.rs::validate_peer_share_indices`, `crates/app/src/node/mod.rs::build_pub_shares_by_key`). This predates charon v1.7.1 with no ladder cover, so it is a genuine coverage gap, not excused as ABSENT-OK. | @@ -95,7 +103,13 @@ whether a ladder entry covers it, and whether it was reported upstream._ `empty_value` (`set: {"0xaabb": ""}`) encodes to `0a0a0a063078616162621200` per charon but `0a080a06307861616262` per pluto (missing the empty-value field 2); `empty_key` (`set: {"": "01"}`) encodes to `0a050a00120101` per charon but - `0a03120101` per pluto (missing the empty-key field 1). No `charon_anchor.json` + `0a03120101` per pluto (missing the empty-key field 1). Re-confirmed + 2026-08-01 by direct differential execution: a scratch Go probe in the + charon checkout ran `hashProto` itself + (`core/consensus/qbft/msg.go:140`, `proto.MarshalOptions{Deterministic: + true}` at `:151`) on both cases and reproduced the vector bytes exactly — + including for a nil (not just empty) map value — while the pinned pluto + outputs were re-verified by the suite against `67088a2`. No `charon_anchor.json` ladder entry covers map-entry presence, so this is not `ABSENT-OK` — it is a real interop risk: two clusters running pluto vs. charon nodes would compute different QBFT value hashes for an `UnsignedDataSet` containing an empty pubkey string or an @@ -207,6 +221,10 @@ whether a ladder entry covers it, and whether it was reported upstream._ the task now tests and documents as an explicit precondition rather than presenting as an unqualified default-config PASS. Not reported upstream to pluto (this is pluto correctly tracking its own v1.7.1 anchor, not a defect). + **Since 2026-08-01 this is formally ABSENT-OK:** the ladder entry "Extended + 1.5s proposer round-1 timeout (proposal_timeout)" (`v1.9.0`) added by the + recommendation below now covers it, and the test's comments name that entry + per the taxonomy. - **Recommendation for this repo (not pluto): `charon_anchor.json`'s `behaviours` ladder has no entry for the proposal-timeout feature-flag stabilization From 998f04a800d96252cbf985ed488d2a0eed993427 Mon Sep 17 00:00:00 2001 From: Andrei Smirnov Date: Sat, 1 Aug 2026 08:12:24 +0300 Subject: [PATCH 27/27] Mark the ladder-entry finding as addressed in its lead The bullet's bold lead still read as an open recommendation ("has no entry") even though the entry landed in ac81aab; retitle it as an addressed gap and fix the timer finding's cross-reference to it. Co-Authored-By: Claude Fable 5 --- plans/pluto-conformance.md | 36 +++++++++++++++++------------------- 1 file changed, 17 insertions(+), 19 deletions(-) diff --git a/plans/pluto-conformance.md b/plans/pluto-conformance.md index c6e2f05..8bb8698 100644 --- a/plans/pluto-conformance.md +++ b/plans/pluto-conformance.md @@ -222,26 +222,24 @@ whether a ladder entry covers it, and whether it was reported upstream._ presenting as an unqualified default-config PASS. Not reported upstream to pluto (this is pluto correctly tracking its own v1.7.1 anchor, not a defect). **Since 2026-08-01 this is formally ABSENT-OK:** the ladder entry "Extended - 1.5s proposer round-1 timeout (proposal_timeout)" (`v1.9.0`) added by the - recommendation below now covers it, and the test's comments name that entry - per the taxonomy. - -- **Recommendation for this repo (not pluto): `charon_anchor.json`'s `behaviours` - ladder has no entry for the proposal-timeout feature-flag stabilization - (task 6).** The promotion of `ProposalTimeout` from alpha to stable-by-default - in charon v1.9.0 (`06b6371b`, #4296) is exactly the kind of postdating-v1.7.1 - behaviour change the ladder exists to track, and the `timer_deadlines` vectors + 1.5s proposer round-1 timeout (proposal_timeout)" (`v1.9.0`), added on this + branch (see the next finding), now covers it, and the test's comments name + that entry per the taxonomy. + +- **Repo-side gap found by task 6, addressed on this branch: + `charon_anchor.json`'s `behaviours` ladder had no entry for the + proposal-timeout feature-flag stabilization.** The promotion of + `ProposalTimeout` from alpha to stable-by-default in charon v1.9.0 + (`06b6371b`, #4296) is exactly the kind of postdating-v1.7.1 behaviour + change the ladder exists to track, and the `timer_deadlines` vectors already depend on it (they assume `ProposalTimeout` enabled), but no ladder - entry names it — unlike the sibling "Deterministic (genesis-derived) eager - double linear round deadlines" entry, which does. Adding an entry is a - separate, self-contained change to this repo's release surface: `README`'s - compatibility table documents the ladder and is enforced by - `tests/test_release.py`, so the entry and the README update need to land - together, in their own commit — not folded into this conformance task. - **Addressed on this branch:** `charon_anchor.json` now carries "Extended - 1.5s proposer round-1 timeout (proposal_timeout)" (`v1.9.0`), README's - compatibility table has the matching row, and `docs/dv-spec/consensus.md`'s - timer section now states the proposer round-1 exception the vectors and + entry named it — unlike the sibling "Deterministic (genesis-derived) eager + double linear round deadlines" entry, which does. Fixed in its own commit + (the ladder and README's compatibility table must move together — + `tests/test_release.py` enforces that): `charon_anchor.json` now carries + "Extended 1.5s proposer round-1 timeout (proposal_timeout)" (`v1.9.0`), + README has the matching row, and `docs/dv-spec/consensus.md`'s timer + section now states the proposer round-1 exception the vectors and reference implementation (`PROPOSAL_ROUND_TIMEOUT`) already carried. - **Pluto's justification-count cap is `4n`, not the spec's `2n` (task 7,