feat: canonical singleton-lineage walk composed from primitive reads

Cargo.toml

@@ -8,7 +8,7 @@
 # aggregating canonical source). See SPEC.md for the normative contract.
 [package]
 name = "dig-chainsource-interface"
-version = "0.3.0"
+version = "0.3.1"
 edition = "2021"
 rust-version = "1.75.0"
 license = "Apache-2.0 OR MIT"
@@ -22,10 +22,42 @@ categories = ["cryptography::cryptocurrencies", "api-bindings"]
 chia-protocol = "0.36.1"
 thiserror = "2"
 
+# --- `lineage-walk` only (see [features]) ------------------------------------------------------
+# The canonical launcher -> tip singleton walk DERIVES each successor by running the parent's inner
+# puzzle, so it needs a CLVM evaluator and the vetted singleton layer/puzzle types. Every version is
+# pinned to the same coherent chia-* set the rest of the DIG on-chain line rides (chia-protocol
+# 0.36 / chia-wallet-sdk 0.34); the SDK's own SingletonLayer is reused rather than re-implemented so
+# the walk cannot byte-drift from the puzzle it authenticates against.
+chia-puzzle-types = { version = "0.36.1", optional = true }
+chia-puzzles = { version = "0.20", optional = true }
+chia-sdk-driver = { version = "0.34", optional = true }
+chia-sdk-types = { version = "0.34", optional = true }
+clvm-traits = { version = "0.36.1", optional = true }
+clvm-utils = { version = "0.36.1", optional = true }
+clvmr = { version = "0.16", optional = true }
+
 [features]
 default = []
 testing = []
+# The canonical singleton lineage walk (`walk_singleton_lineage`). NON-DEFAULT: it pulls in a CLVM
+# evaluator, and a consumer that only depends on the trait should not pay for one. Enabling it is
+# purely additive — no existing item changes shape.
+lineage-walk = [
+    "dep:chia-puzzle-types",
+    "dep:chia-puzzles",
+    "dep:chia-sdk-driver",
+    "dep:chia-sdk-types",
+    "dep:clvm-traits",
+    "dep:clvm-utils",
+    "dep:clvmr",
+]
 
 [dev-dependencies]
 chia-traits = "0.36.1"
 hex = "0.4"
+# The in-process Chia Simulator: the lineage-walk tests authenticate against REAL singleton spends
+# (launcher, eve, recreation, melt) rather than hand-built fixtures, so the adversarial cases are
+# genuine chain data.
+chia-sdk-test = "0.34"
+chia-bls = "0.36.1"
+anyhow = "1"

README.md

@@ -47,6 +47,36 @@ walking `parent_spend` back toward the real launcher — a spoofed curried-puzzl
 recreation parent-spend, so the walk fails closed. `SingletonLineage` follows suit: authority is
 **membership** (`contains`), never tip-equality.
 
+## The canonical lineage walk (feature `lineage-walk`)
+
+`resolve_singleton_lineage` is the one method with no default body, and it is the most
+trust-critical: its result IS the authority set consumers test membership against. A source backed
+only by primitive reads can borrow the whole walk instead of hand-rolling it:
+
+```toml
+dig-chainsource-interface = { version = "0.3", features = ["lineage-walk"] }
+```
+
+```rust
+fn resolve_singleton_lineage(
+    &self,
+    launcher_id: Bytes32,
+) -> Result<Option<SingletonLineage>, Self::Error> {
+    resolve_singleton_lineage_via_walk(self, launcher_id)
+}
+```
+
+The walk starts at the launcher coin and **derives** each successive coin by running the previous
+coin's own spend — it never recognises a coin by its puzzle hash, its curried launcher id, or its
+presence in a child list, because all three are attacker-chosen. It refuses rather than truncating
+past either of its two bounds — `MAX_LINEAGE_DEPTH` spends and `DEFAULT_WALK_BUDGET` of wall-clock
+time — so a hostile source serving an endless chain of valid recreations can neither hang the
+calling thread nor grow the walk's memory without limit. Both bounds come with the one-line
+delegation above; `walk_singleton_lineage_within` chooses others. See SPEC.md §4a.
+
+The feature is off by default: the walk needs a CLVM evaluator, and a consumer that only depends on
+the trait should not pay for one.
+
 ## Implementing a provider
 
 Implement `ChainSource` over your backend, choosing `type Error` (`ChainSourceError` is recommended

SPEC.md

@@ -1,4 +1,4 @@
-# dig-chainsource-interface — normative specification (v0.1.0)
+# dig-chainsource-interface — normative specification (v0.4.0)
 
 This is the authoritative contract for the DIG Network canonical `ChainSource` interface. An
 independent reimplementation of this crate, of a provider, or of a consumer MUST conform to this
@@ -56,6 +56,13 @@ Every fallible method distinguishes:
 - `TooManyRecords { count, limit }` — the backend returned more records than the consumer's
   hostile-input bound allows; distinct from `Malformed` (each record may be well-formed, but the
   count exceeds the cap) — the consumer fails closed the same as every other variant.
+- `Timeout` — also carries a lineage walk that exceeded its wall-clock budget (§4a).
+- `RevealTooLarge { limit }` — a puzzle reveal expands, once its CLVM back-references are unfolded,
+  beyond the walk's bound (§4a). Distinct from `Malformed`: the reveal may be entirely well-formed
+  chain data and merely larger than the walk will authenticate.
+- `LineageTooDeep { limit }` — a singleton lineage walk exceeded its hop bound (§4a). The lineage it
+  could build is INCOMPLETE, so it is refused rather than truncated: a partial member set would make
+  `contains` answer `false` for genuine members, which is a fail-OPEN membership answer.
 
 Absence MUST NOT be encoded as an error; an error MUST NOT be degraded to a value.
 
@@ -72,6 +79,141 @@ echo a caller-supplied coin into the lineage. Echoing would make `contains` mean
 foreign coin to claim authority. Consumers authenticate coins by walking `parent_spend` toward the
 real launcher and testing lineage membership — never by puzzle-hash equality.
 
+## 4a. The canonical lineage walk (feature `lineage-walk`)
+
+`ChainSource::resolve_singleton_lineage` has no default body, so a source backed only by primitive
+reads would have to hand-roll the §4 money-critical requirement. The optional, NON-DEFAULT
+`lineage-walk` feature supplies that walk once, as free functions:
+
+```rust
+pub const MAX_LINEAGE_DEPTH: usize = 100_000;
+pub const DEFAULT_WALK_BUDGET: Duration = Duration::from_secs(45);
+pub const MAX_REVEAL_EXPANDED_BYTES: usize = 4 * 1024 * 1024;
+pub const MAX_HOP_CLVM_COST: u64 = 100_000_000;
+
+// Fields are PRIVATE and `max_hops` is clamped to MAX_LINEAGE_DEPTH; the guards cannot be
+// disabled through a struct literal. Default: the two bound constants above.
+pub struct WalkBounds { /* private */ }
+impl WalkBounds {
+    pub fn hops(max_hops: usize) -> Self;   // clamped to MAX_LINEAGE_DEPTH
+    pub fn within(self, budget: Duration) -> Self;
+    pub fn max_hops(self) -> usize;
+    pub fn budget(self) -> Duration;
+}
+
+pub enum LineageWalkError<E> {
+    Source(E), Malformed(String), NotASingleton { coin_id },
+    RevealTooLarge { coin_id, limit }, TooDeep { limit }, DeadlineExceeded { budget },
+}
+
+pub fn walk_singleton_lineage<S: ChainSource>(source: &S, launcher_id: Bytes32)
+    -> Result<Option<SingletonLineage>, LineageWalkError<S::Error>>;
+
+pub fn walk_singleton_lineage_bounded<S: ChainSource>(source: &S, launcher_id: Bytes32, max_hops: usize)
+    -> Result<Option<SingletonLineage>, LineageWalkError<S::Error>>;
+
+pub fn walk_singleton_lineage_within<S: ChainSource>(source: &S, launcher_id: Bytes32, bounds: WalkBounds)
+    -> Result<Option<SingletonLineage>, LineageWalkError<S::Error>>;
+
+pub fn resolve_singleton_lineage_via_walk<S: ChainSource<Error = ChainSourceError>>(
+    source: &S, launcher_id: Bytes32) -> Result<Option<SingletonLineage>, ChainSourceError>;
+```
+
+A conforming walk MUST:
+
+1. **Derive, never recognise.** At each hop it reads the current coin's own spend, requires the
+   returned spend to BE that coin's spend, requires the puzzle reveal to hash to that coin's puzzle
+   hash, parses the reveal as a singleton curried to the launcher under resolution, runs the inner
+   puzzle, and RECONSTRUCTS the odd-amount successor's full puzzle hash from the launcher id and the
+   successor's inner puzzle hash. It MUST NOT select the successor by puzzle-hash equality, by
+   curried launcher id alone, or from `coin_records_by_parent` — every one of those is spoofable,
+   because a coin's `puzzle_hash` is attacker-chosen.
+2. **Bind each derived coin to chain state.** A CLVM solution is not committed to by a coin's puzzle
+   hash, so a dishonest source could pair a genuine reveal with a fabricated solution. Each derived
+   successor MUST be confirmed to exist via `coin_record` before it enters the lineage.
+3. **Decode the `CREATE_COIN` amount as SIGNED, and decode it BEFORE the puzzle hash.** CLVM atoms
+   carry no sign, so the singleton melt marker `-113` decodes into a `u64` as `143` — an odd,
+   positive amount indistinguishable from an ordinary recreation. A walk that made that mistake
+   would invent a phantom successor for every melted singleton instead of reporting the melt. The
+   amount is also the DISCRIMINANT for the puzzle hash: the canonical melt condition is
+   `(51 () -113)`, carrying a NIL puzzle hash, which is what standard chia-wallet-sdk tooling emits.
+   A walk that required a 32-byte puzzle hash before testing the melt marker would refuse every such
+   melt, making a singleton melted with standard tooling permanently unanswerable. Both the nil and
+   the 32-byte melt forms MUST decode as a melt.
+4. **Deserialize programs with the BACK-REFERENCE reader.** A puzzle reveal or solution may be
+   serialized in the CLVM back-reference form — the compressed encoding full nodes accept and block
+   generators emit, which a curried singleton reveal exercises heavily. A walk that reads only the
+   non-backref form reports a genuine singleton as `Malformed`, blaming an honest source.
+5. **Refuse, never truncate, past ANY bound** — `MAX_LINEAGE_DEPTH` spends and
+   `DEFAULT_WALK_BUDGET` of wall-clock time by default — and reject a repeated coin id as a cycle.
+   The hop cap alone is insufficient: it bounds neither elapsed time nor per-hop cost, so a hostile
+   source serving a structurally valid, ever-advancing chain of DISTINCT recreations trips no other
+   guard. `ChainSource` is synchronous, so that is the caller's thread. A budget overrun MUST report
+   as `LineageWalkError::DeadlineExceeded` (projecting to `ChainSourceError::Timeout`), never as
+   `TooDeep` or `Malformed` — the source may have been entirely honest.
+
+   The wall-clock budget is checked BETWEEN hops, so it is a **backstop**, not a hard deadline: a
+   conforming walk returns within `budget + one worst-case hop`. That guarantee is vacuous unless
+   the cost of ONE hop is itself bounded, which is what §5a and §5b require.
+
+5a. **Bound the EXPANDED size of a puzzle reveal, before hashing, parsing or running it.** CLVM
+   back-references are a compression: the bytes on the wire describe a shared DAG, while every
+   consumer of that DAG — the reveal-binding hash, curried-puzzle parsing, the evaluator — sees the
+   tree it unfolds into, and a `k`-level self-referential DAG unfolds into `2^k` nodes. A conforming
+   walk MUST refuse a reveal whose expansion exceeds `MAX_REVEAL_EXPANDED_BYTES`, reporting
+   `LineageWalkError::RevealTooLarge` (projecting to `ChainSourceError::RevealTooLarge`) — never
+   `Malformed`, because the reveal may be valid chain data that is merely too large.
+
+   Two properties are normative, and each closes an attack the other does not. The bound MUST be on
+   the EXPANSION, not on the serialized length: a serialized-length cap large enough for an honest
+   singleton is orders of magnitude above the bomb, which is about a kilobyte. And the bound MUST be
+   applied BEFORE the reveal-binding hash, so that every later use of those bytes is downstream of
+   it: a bomb curried as the INNER puzzle of an otherwise genuine singleton passes the binding check
+   truthfully — the walk derived that coin's puzzle hash from the bomb's own tree hash — and
+   detonates in whatever parses the reveal next. Memoizing the binding hash does not close this;
+   only bounding the expansion ahead of all of it does.
+
+5b. **Bound the CLVM cost of ONE hop.** A spend's puzzle reveal is hash-bound to its coin, but its
+   SOLUTION is bound to nothing — the source chooses it freely. A conforming walk MUST evaluate with
+   an explicit per-hop ceiling (`MAX_HOP_CLVM_COST`), never the whole-block cost limit, or one hop
+   may legitimately burn an entire block's worth of evaluation.
+6. **Bound per-hop CLVM memory.** A CLVM allocator is an arena that frees nothing until dropped, so
+   one shared across hops accumulates every hop's puzzle, solution and evaluation. A conforming walk
+   MUST start each hop with a fresh allocator (or restore a checkpoint). Sharing one both costs
+   memory linear in the chain length and exhausts the arena's own node ceiling BEFORE
+   `MAX_LINEAGE_DEPTH` is reached, which makes the documented `TooDeep` refusal unreachable and
+   misreports the exhaustion as `Malformed`.
+7. **Preserve the three-valued discipline of §3.** `Ok(None)` means the launcher id names no coin,
+   names a coin that is not wearing `SINGLETON_LAUNCHER_HASH`, was never spent into an eve, or the
+   singleton was melted. Every read failure surfaces as `LineageWalkError::Source(_)` carrying the
+   source's OWN error unchanged, so *unsupported* stays distinguishable from *unreadable* and
+   neither is ever collapsed into an absence.
+8. **Treat an unreadable spend as unknown, never as the tip.** `coin_spend` answers `Ok(None)` for
+   "unspent **or** unknown" (§3), so a walk MUST consult the coin's own `spent_height` before
+   concluding it has reached the tip. A coin recorded as SPENT whose spend the source does not serve
+   MUST fail closed with `LineageWalkError::Malformed`. Reporting it as the tip would present a
+   superseded state as current — and if the unserved spend was the melt, a dead singleton would
+   authenticate as live; at the launcher it would degrade an unknown into "never launched",
+   violating §3.
+9. **Refuse an unreadable `CREATE_COIN`.** Once a condition's opcode is known to be `CREATE_COIN`,
+   arguments the walk cannot decode (including an amount outside `i64`, or a negative amount that is
+   not the melt marker) MUST be a refusal. Skipping such a condition makes a spend look as though it
+   emitted no odd-amount child — a phantom melt, i.e. requirement 8's defect reached through the
+   condition decoder.
+
+`MAX_LINEAGE_DEPTH` is the ecosystem's SINGLE source of truth for this bound. A DIG crate that
+bounds a singleton lineage walk MUST import it from this crate rather than re-declare the literal;
+this crate is `00-foundation`, so every such consumer sits strictly above it.
+
+**Stated limit — the unspent eve.** An eve coin that has never been spent is admitted on the evidence
+of the launcher's own spend, which is the strongest evidence that exists: the eve is by definition the
+coin the launcher created, and the launcher's `CREATE_COIN` carries the eve's FULL puzzle hash, which
+is non-invertible — so nothing an attacker supplies reaches the decision. Consequently a launcher
+spent into an ORDINARY coin resolves to `Ok(Some(_))` with that coin as the tip, not `Ok(None)`. The
+eve's inner structure is constrained the moment it is itself spent, at which point the curried
+launcher id is checked and a non-singleton yields `NotASingleton`. A consumer that requires a *proven*
+singleton rather than a *launched* one MUST require a tip beyond the eve.
+
 ## 5. `CoinRecord` and `CoinState` conversion
 
 `CoinRecord { coin: Coin, confirmed_height: Option<u32>, spent_height: Option<u32>,
@@ -114,5 +256,6 @@ A conforming provider MUST:
    `chia-protocol`).
 3. Map `CoinState` per §5.
 4. Return a genuine forward-walked lineage from `resolve_singleton_lineage` per §4 — never an echo.
+   A source backed only by primitive reads SHOULD delegate to the §4a walk rather than hand-roll one.
 5. Report a `ProviderInfo` per §6.
 6. Remain reads-only: expose no broadcast/spend path through this interface.

src/error.rs

@@ -60,6 +60,31 @@ pub enum ChainSourceError {
         /// The maximum number of records the consumer will accept.
         limit: usize,
     },
+
+    /// A puzzle reveal expanded to more than the walk's decompressed-size bound.
+    ///
+    /// Distinct from [`Malformed`](Self::Malformed) on purpose: the reveal may be perfectly
+    /// well-formed chain data — it is simply larger, once its CLVM back-references are expanded,
+    /// than this walk will authenticate. Blaming the source for corruption would be a lie, and
+    /// would hide the one thing a consumer can act on: the payload was too big, not wrong.
+    #[error("puzzle reveal expands beyond the {limit}-byte bound")]
+    RevealTooLarge {
+        /// The expanded-size bound the walk refused to exceed.
+        limit: usize,
+    },
+
+    /// A singleton lineage walk exceeded its hop bound before reaching the tip.
+    ///
+    /// Distinct from every other variant, and deliberately NOT a silent truncation: the walk found
+    /// more hops than it will follow, so the lineage it could build is INCOMPLETE and must never be
+    /// presented as the whole lineage (a partial member set would make
+    /// [`SingletonLineage::contains`](crate::SingletonLineage::contains) answer `false` for genuine
+    /// members — a fail-OPEN membership answer on a money path). The answer is unknown → fail closed.
+    #[error("singleton lineage walk exceeded its {limit}-hop bound")]
+    LineageTooDeep {
+        /// The hop bound the walk refused to exceed.
+        limit: usize,
+    },
 }
 
 #[cfg(test)]
@@ -78,6 +103,22 @@ mod tests {
         );
     }
 
+    /// "Too big" must never read as "corrupt": the reveal may be perfectly valid chain data, and a
+    /// consumer that cannot tell the two apart cannot tell a hostile source from a heavy one.
+    #[test]
+    fn reveal_too_large_is_distinct_from_malformed() {
+        let too_large = ChainSourceError::RevealTooLarge { limit: 4_194_304 };
+        assert_eq!(
+            too_large.to_string(),
+            "puzzle reveal expands beyond the 4194304-byte bound"
+        );
+        assert_ne!(
+            too_large,
+            ChainSourceError::Malformed("undecodable program".to_string())
+        );
+        assert!(!matches!(too_large, ChainSourceError::Malformed(_)));
+    }
+
     #[test]
     fn too_many_records_is_distinct_from_malformed() {
         let too_many = ChainSourceError::TooManyRecords { count: 5, limit: 1 };

src/lib.rs

@@ -35,13 +35,31 @@
 //! actual reveal+solution — a spoofed curried-puzzle coin has no genuine recreation parent-spend,
 //! so the walk fails closed. This crate supplies that primitive (and [`SingletonLineage`], whose
 //! authority is MEMBERSHIP, not tip-equality); consumers supply the trust logic on top.
+//!
+//! ## The canonical lineage walk (feature `lineage-walk`)
+//!
+//! [`ChainSource::resolve_singleton_lineage`] is the one method with no default body, so a source
+//! backed only by primitive reads would have to hand-roll that money-critical authentication. Enable
+//! the non-default `lineage-walk` feature and the whole walk is supplied — the method body becomes a
+//! one-line delegation to [`resolve_singleton_lineage_via_walk`]:
+//!
+//! ```toml
+//! dig-chainsource-interface = { version = "0.3", features = ["lineage-walk"] }
+//! ```
+//!
+//! The feature is OFF by default because the walk needs a CLVM evaluator (it runs each parent's
+//! inner puzzle to DERIVE its successor), and a consumer that only depends on the trait should not
+//! pay for one.
 
 mod error;
 mod lineage;
 mod provider;
 mod record;
 mod source;
 
+#[cfg(feature = "lineage-walk")]
+mod walk;
+
 #[cfg(feature = "testing")]
 mod testing;
 
@@ -51,6 +69,13 @@ pub use provider::{ProviderId, ProviderInfo, ProviderKind};
 pub use record::CoinRecord;
 pub use source::{ChainSource, ChainSourceProvider};
 
+#[cfg(feature = "lineage-walk")]
+pub use walk::{
+    resolve_singleton_lineage_via_walk, walk_singleton_lineage, walk_singleton_lineage_bounded,
+    walk_singleton_lineage_within, LineageWalkError, WalkBounds, DEFAULT_WALK_BUDGET,
+    MAX_HOP_CLVM_COST, MAX_LINEAGE_DEPTH, MAX_REVEAL_EXPANDED_BYTES,
+};
+
 #[cfg(feature = "testing")]
 pub use testing::MockChainSource;