Skip to content

feat(formal): Clean/Lean model of BITWISE, LT, EQ and SHIFT with proo… - #1027

Open
ColoCarletti wants to merge 26 commits into
mainfrom
formal/clean-model
Open

ColoCarletti wants to merge 26 commits into
mainfrom
formal/clean-model

Conversation

@ColoCarletti

Copy link
Copy Markdown
Collaborator

…fs and a differential check against the prover

@MauroToscano

Copy link
Copy Markdown
Contributor

/review-ai

@MauroToscano

Copy link
Copy Markdown
Contributor

/ai-review

Comment thread prover/src/tests/lean_diff.rs Resolved
Comment thread .github/workflows/formal.yml Resolved
@claude

claude Bot commented Oct 30, 2026

Copy link
Copy Markdown
Contributor

Review summary

No critical, high or medium issues. The Rust changes are test-only, apart from making Multiplicity::evaluate_with public. The differential harness in lean_diff.rs is sound:

  • eval_program_base asserts that a constraint reads only the current row, so a constraint that also reads the next row fails loudly rather than being skipped.
  • Unknown export nodes or channels panic.
  • Column counts are asserted to match.
  • Honest traces include a padding row.
  • The expected LT differences (μ·(1−μ) vs. ARE_BYTES[μ,0]) are pinned exactly, so any further drift fails the test.

Minor notes (two are posted inline):

  • Low: formal.yml cache key ends in github.sha, which saves a new cache entry on every run and adds pressure on the repo's cache quota.
  • Low: the seed 0xC1EA_0000 ^ label.len() is the same for LT and EQ, so the label salt has no effect.
  • Low: git diff --exit-code -- export does not catch untracked new export files. If a chip is added to Export.lean without committing its JSON, the check still passes. git status --porcelain -- export with an empty-output check would cover that.
  • Low: Multiplicity::evaluate_with became pub only for this test. That's fine, but #[doc(hidden)] or a short note would make the intent clear.
  • Note: BITWISE is modeled in Lean but is not exported or diff-checked like LT, EQ and SHIFT. Worth saying so in the README so readers don't assume it's covered.

🤖 Generated with Claude Code

…el to its channel API.

Clean removed StaticLookupChannel in favour of Air.Flat.Lookup, so the structure moves to
LambdaVM/StaticLookup.lean and BITWISE reads its tables through toChannel. Interactions now carry a
direction and a nonnegative multiplicity, and a pull must have multiplicity 0 or 1: EQ, LT and SHIFT
discharge that from their existing bit constraints, and their guarantees are stated for
multiplicity 1 instead of -1. EqLookup is rewritten for the current SoundEnsemble API (a balance
parameter, channels added before tables, unique table names, a demand generation mode per
component), and eq_rows_correct takes the InteractionsFit premise that TableSoundness now has.
Channels/Counted.lean is ported to directions; not_consistent needs only two interactions, since a
pull of multiplicity N >= 2 counts as a push. A grind call in IsByte that no longer closes its goal
is replaced by a direct proof. The export now records each interaction's direction, so the JSON
files are regenerated and lean_diff.rs signs Lean multiplicities by direction. Nothing that is
proven changes.
…p fallback the linter flags.

if_true, if_false, if_pos, if_neg, dif_pos and dif_neg become ite_true, ite_false, ite_eq_left,
ite_eq_right, dite_eq_left and dite_eq_right, as in Clean. In SHIFT, first | ... | skip becomes
try first | ..., which behaves the same. The build has no warnings left.
… the bump, and the renames

left in SHIFT's completeness proof. Both lemmas discharged a pull requirement stated with the old
multiplicity -1 (`¬ -e = -1 → ¬ e = 0 → P`), which the directed interactions no longer produce.
In SHIFT, the hypotheses that needed neg_inj.mp to go from `-x = -1` to `x = 1` now arrive as
`x = 1`, so they are introduced under their final names instead of being copied.
…tates the sign the

comparison uses: a sender (Clean's pull) counts −m and a receiver (Clean's push) +m. The old
first sentence described the prover's LogUp sum, where a sender adds +m, and read as if it
contradicted the code below it. Comment only, no code change.
Clean's Demand.direction is the net sign of a message, pushes minus
pulls, so a demand of direction .pull is one that was pulled more often
than pushed and needs a component that pushes to answer it. EQ pushes
its result on the ALU bus, so its generation mode has to trigger on
.pull, the same way pushStaticComponent does and the same way Clean's
own add8Component and lookup provider do. The field is inert today,
since generation is not part of TableSoundness and nothing here runs
Clean's witness generator, but .push described a component that answers
pushes by pulling, which EQ never does.
The Clean bump changed the things this file documents: pulls carry a
nonnegative multiplicity with an explicit direction instead of a
negative one, a pull owes a multiplicity of 0 or 1, BalancedInteractions
no longer carries the interaction count bound (it moved to the Balance's
fits, reaching soundness as the InteractionsFit premise), and
not_consistent is now a two-interaction witness instead of the p/N
construction. Also lists LambdaVM/StaticLookup.lean, which the bump
added, and records that the export format carries the direction.
Bump Clean to current main (f3abad7c) and Lean to v4.35.0 in formal/
@ColoCarletti
ColoCarletti marked this pull request as ready for review November 11, 2026 18:08
jotabulacios added a commit that referenced this pull request Nov 13, 2026
… credit EqLookup to its PR.

The section linked the fork by branch name while lakefile.toml pins by sha, so the two could drift,
and the link dies when the branch is renamed or deleted once the upstream PRs land. It also said
"Clean main f3abad7c plus three commits", which was true of the old pin and not of 73309694: that
one is the three commits plus the four review fixes answering the audit of them. And EqLookup,
described there as "in this same PR", is already in the model from PR #1027.
@jotabulacios

Copy link
Copy Markdown
Collaborator

/ai-review

@github-actions

Copy link
Copy Markdown

AI Review

PR #1027 · 35 changed files

Warning: the diff was truncated before review.

Findings

No non-rejected structured findings were reported.

Reviewer Lanes

Lane Model Prompt Status Findings

Native Codex and Claude reviews run separately and post their own comments. They are not included in this structured provenance report.

Raw lane outputs, candidates, final issues, and model metrics are uploaded as workflow artifacts.

@claude

claude Bot commented Nov 13, 2026

Copy link
Copy Markdown
Contributor

Review: Lean formal model + lean_diff differential test

No blocking issues. The Rust changes are test-only, apart from making Multiplicity::evaluate_with public, which is harmless. pr_main.yaml runs on every PR, so lean_diff catches Rust table changes that drift from the Lean model. formal.yml keeps export/ in sync with the Lean sources. I found no sorry, axiom, admit or native_decide in the Lean sources.

Minor points:

  • Medium (scope, already documented): The LT model forces μ ∈ {0,1}, but Rust dedups rows with μ up to MU_MAX. So the LT soundness and completeness theorems don't cover the rows the prover actually produces when μ > 1. The test pins this delta by content, which is good. Still, readers shouldn't take "LT is proven" at face value until counted pulls land. A one-line caveat next to the theorem statement in Lt.lean would help, in addition to the README.
  • Low (test coverage): compare evaluates constraints with eval_program_base on a single main row. That works for LT, EQ and SHIFT, which only constrain the current row. If one of these tables ever gains a constraint that reads the next row or the aux trace, this harness won't model it. A short note in the module doc would prevent a false sense of coverage.
  • Low (doc rot): formal/README.md has a dated results snapshot ("Result on main after that merge, 2026-10-28" plus the counts). CI already enforces these results, so the snapshot will go stale. Consider removing it or making it date-free.
  • Low (CI hygiene): formal.yml pipes an unpinned elan-init.sh from curl into sh. It's acceptable with contents: read, but pinning the elan version or the script hash would make builds reproducible.
  • Nit: In Export.lean, channelJson silently writes [] when an index is out of range. Rust would still flag the mismatch, but failing loudly at export time (panic!) would make the cause obvious.

…ed elan, export fails on a missing table entry
@MauroToscano

Copy link
Copy Markdown
Contributor

Review notes, checked at f927f3e.

Not proven: fine to leave out, but please say so in the README

These are scope limits, not bugs. Please add a short "What is intentionally not proven" section to formal/README.md that lists them in one place. Some are already mentioned in passing, but a reader should not have to assemble them.

  1. The lookups a row makes are assumed true, not proven. Each theorem is about one row of one chip, given that every range check or lookup it pulls is true. Nothing discharges that for the real system. The only ensemble theorem (eq_rows_correct) is trivially true because no ALU consumer is modelled, and LT and SHIFT have no ensemble. A negative multiplicity in an unmodelled table that shares IS_HALF (the class of bug Fix: Constrain table multiplicities #1016 fixed) would leave every theorem and lean_diff green.
  2. LT is proven for μ ∈ {0, 1} only. The deduplicated rows that lt.rs produces (μ = 2…255, present on every real block) are covered by neither soundness nor completeness. The new doc comment on LT.circuit helps; the README section should state it as a scope limit too. This may already be within reach: Counted.Demo suggests the LT row can be written with the exact Rust interactions, with only the ensemble step needing upstream Clean.
  3. The Lean BITWISE table is compared on 656 sampled rows. The "all 2^20 rows" loop compares generate_bitwise_row against a Rust transcription of Bitwise.row, not against Lean.
  4. The meaning side is not checked. The ALU guarantee has no consumer, since the CPU is not modelled. "Proved against spec/" means spec/main@2eaf1f080, not the spec/ in this repository.

The differential test

  1. Direction is never compared on its own. lean_diff.rs:165-166 folds is_sender into the sign of the multiplicity, so a Lean pull of m and a push of −m compare equal. The README rule that Lean multiplicities are nonnegative is never checked either. The sign does carry the direction, so nothing is wrong today. Still, please compare the Lean direction with is_sender and the unsigned multiplicities separately: a flipped direction turns a fact the chip must prove into one it assumes.
  2. The test checks the trace-building copy of the multiplicity math, not what the verifier enforces. It evaluates Multiplicity::evaluate_with and BusValue::combine_from (lean_diff.rs:165-170). The constraints the verifier enforces come from emit_multiplicity / emit_fingerprint (lookup.rs:1975, lookup.rs:2164), which are separate copies. If the two drift apart, the test still passes. Comparing against the emitted expressions would also remove the need to make evaluate_with public.
  3. One LT delta is pinned by position. The comment at lean_diff.rs:413 says both deltas are "pinned by content, not position". The Lean-only constraint is pinned to index 0 (vec![0], lean.asserts[0]) and then checked by content; only the Rust-only interaction is pinned by content alone. Moving isBit r.mu in LT.main would fail the test. Either look the assert up by content or fix the comment.

CI

  1. The cache stores a full copy of Mathlib. The !formal/.lake/packages/mathlib exclusion at formal.yml:49 has no effect, because actions/cache archives formal/.lake recursively. A warm run restores about 2.3 GB, and then lake exe cache get downloads Mathlib again. Each new manifest adds another entry of about 2.2 GB against the repository's 10 GB cache quota, which the Rust caches share.
  2. Not suitable as a required check yet. The workflow has a paths filter and no merge_group trigger. If it becomes required, PRs that don't touch formal/ and merge-queue entries would wait for a check that never runs. lake build also only warns on sorry; a step that fails on sorry / axiom in formal/ would make the check mean "fully proven". The Rust lean_diff tests already run in the normal prover shards.

Minor

  • Shift padding docstring. The Padding docstring (Shift.lean:231) says it covers more than "the all-zero Rust padding row", but Rust padding rows set ZBS = 1 (that is why Padding requires zbs = 1). An all-zero row pulls HWSL with multiplicity 1 − zbs = 1 and is not covered.
  • Placeholder witness generation. eqComponent (EqLookup.lean:74) and pushStaticComponent fill every column with the demand's multiplicity. That is a placeholder, but the comment reads as if generation works. If the generator is ever run, the rows break SUB and ZERO.
  • Repeated code:
    • About 15 copies of the same limb-value proof ((a + 65536*b).val = a.val + 65536*b.val) across Lt.lean, Eq.lean and Shift.lean; eval_shiftedPoly_1..4 differ only in the tuple position. One lemma in Field.lean would cover them.
    • Three copies of SplitMix64 remain (constraint_program_device_tests.rs:37, constraint_set_tests_a.rs:32, constraint_set_tests_b.rs:36) after the move to test_rng.rs.
    • Alu.Guarantees is one conjunction with a clause per opsel, so every chip's soundness proof must disprove every other chip's clause. Adding MUL, DVRM or BYTEWISE breaks LT, EQ and SHIFT, and the work grows with the square of the number of chips. Keying the guarantee on the decoded opsel, with one disjointness lemma, keeps each chip to its own clause.
    • Counted.lean defines its own Normal class next to Clean's RawChannel.Normal, used through dot notation in the same namespace; a distinct name (e.g. CountedNormal) would avoid confusion.
  • Slowness:
    • lean_diff calls eval_program_base (which evaluates the whole program) once per constraint, so the cost grows with the square of the constraint count. Evaluating once per row and reading all roots gives the same check.
    • The export builds every channel's full table with slc.table.toArray (about 10M entries) to output 656 sampled rows, which costs about 60 s per CI run. The sampled entries can come from Bitwise.row i directly.

…n in README, slimmer lake cache, review fixes
@ColoCarletti

ColoCarletti commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator Author

Review notes, checked at f927f3e.

Not proven: fine to leave out, but please say so in the README

These are scope limits, not bugs. Please add a short "What is intentionally not proven" section to formal/README.md that lists them in one place. Some are already mentioned in passing, but a reader should not have to assemble them.

1. **The lookups a row makes are assumed true, not proven.** Each theorem is about one row of one chip, given that every range check or lookup it pulls is true. Nothing discharges that for the real system. The only ensemble theorem (`eq_rows_correct`) is trivially true because no ALU consumer is modelled, and LT and SHIFT have no ensemble. A negative multiplicity in an unmodelled table that shares `IS_HALF` (the class of bug [Fix: Constrain table multiplicities #1016](https://github.com/yetanotherco/lambda_vm/pull/1016) fixed) would leave every theorem and `lean_diff` green.

2. **LT is proven for μ ∈ {0, 1} only.** The deduplicated rows that `lt.rs` produces (μ = 2…255, present on every real block) are covered by neither soundness nor completeness. The new doc comment on `LT.circuit` helps; the README section should state it as a scope limit too. This may already be within reach: `Counted.Demo` suggests the LT row can be written with the exact Rust interactions, with only the ensemble step needing upstream Clean.

3. **The Lean BITWISE table is compared on 656 sampled rows.** The "all 2^20 rows" loop compares `generate_bitwise_row` against a Rust transcription of `Bitwise.row`, not against Lean.

4. **The meaning side is not checked.** The ALU guarantee has no consumer, since the CPU is not modelled. "Proved against `spec/`" means `spec/main@2eaf1f080`, not the `spec/` in this repository.

The differential test

5. **Direction is never compared on its own.** [`lean_diff.rs:165-166`](https://github.com/yetanotherco/lambda_vm/blob/f927f3e7007a50c37f28114dff0d10dea63d9556/prover/src/tests/lean_diff.rs#L165-L166) folds `is_sender` into the sign of the multiplicity, so a Lean pull of _m_ and a push of −_m_ compare equal. The README rule that Lean multiplicities are nonnegative is never checked either. The sign does carry the direction, so nothing is wrong today. Still, please compare the Lean direction with `is_sender` and the unsigned multiplicities separately: a flipped direction turns a fact the chip must prove into one it assumes.

6. **The test checks the trace-building copy of the multiplicity math, not what the verifier enforces.** It evaluates `Multiplicity::evaluate_with` and `BusValue::combine_from` ([`lean_diff.rs:165-170`](https://github.com/yetanotherco/lambda_vm/blob/f927f3e7007a50c37f28114dff0d10dea63d9556/prover/src/tests/lean_diff.rs#L165-L170)). The constraints the verifier enforces come from `emit_multiplicity` / `emit_fingerprint` ([`lookup.rs:1975`](https://github.com/yetanotherco/lambda_vm/blob/f927f3e7007a50c37f28114dff0d10dea63d9556/crypto/stark/src/lookup.rs#L1975), [`lookup.rs:2164`](https://github.com/yetanotherco/lambda_vm/blob/f927f3e7007a50c37f28114dff0d10dea63d9556/crypto/stark/src/lookup.rs#L2164)), which are separate copies. If the two drift apart, the test still passes. Comparing against the emitted expressions would also remove the need to make `evaluate_with` public.

7. **One LT delta is pinned by position.** The comment at [`lean_diff.rs:413`](https://github.com/yetanotherco/lambda_vm/blob/f927f3e7007a50c37f28114dff0d10dea63d9556/prover/src/tests/lean_diff.rs#L413) says both deltas are "pinned by content, not position". The Lean-only constraint is pinned to index 0 (`vec![0]`, `lean.asserts[0]`) and then checked by content; only the Rust-only interaction is pinned by content alone. Moving `isBit r.mu` in `LT.main` would fail the test. Either look the assert up by content or fix the comment.

CI

8. **The cache stores a full copy of Mathlib.** The `!formal/.lake/packages/mathlib` exclusion at [`formal.yml:49`](https://github.com/yetanotherco/lambda_vm/blob/f927f3e7007a50c37f28114dff0d10dea63d9556/.github/workflows/formal.yml#L49) has no effect, because `actions/cache` archives `formal/.lake` recursively. A warm run restores about 2.3 GB, and then `lake exe cache get` downloads Mathlib again. Each new manifest adds another entry of about 2.2 GB against the repository's 10 GB cache quota, which the Rust caches share.

9. **Not suitable as a required check yet.** The workflow has a `paths` filter and no `merge_group` trigger. If it becomes required, PRs that don't touch `formal/` and merge-queue entries would wait for a check that never runs. `lake build` also only warns on `sorry`; a step that fails on `sorry` / `axiom` in `formal/` would make the check mean "fully proven". The Rust `lean_diff` tests already run in the normal prover shards.

Minor

* **Shift padding docstring.** The `Padding` docstring ([`Shift.lean:231`](https://github.com/yetanotherco/lambda_vm/blob/f927f3e7007a50c37f28114dff0d10dea63d9556/formal/LambdaVM/Tables/Shift.lean#L231)) says it covers more than "the all-zero Rust padding row", but Rust padding rows set ZBS = 1 (that is why `Padding` requires `zbs = 1`). An all-zero row pulls HWSL with multiplicity 1 − zbs = 1 and is not covered.

* **Placeholder witness generation.** `eqComponent` ([`EqLookup.lean:74`](https://github.com/yetanotherco/lambda_vm/blob/f927f3e7007a50c37f28114dff0d10dea63d9556/formal/LambdaVM/Ensembles/EqLookup.lean#L74)) and `pushStaticComponent` fill every column with the demand's multiplicity. That is a placeholder, but the comment reads as if generation works. If the generator is ever run, the rows break SUB and ZERO.

* **Repeated code:**
  
  * About 15 copies of the same limb-value proof (`(a + 65536*b).val = a.val + 65536*b.val`) across `Lt.lean`, `Eq.lean` and `Shift.lean`; `eval_shiftedPoly_1..4` differ only in the tuple position. One lemma in `Field.lean` would cover them.
  * Three copies of `SplitMix64` remain (`constraint_program_device_tests.rs:37`, `constraint_set_tests_a.rs:32`, `constraint_set_tests_b.rs:36`) after the move to `test_rng.rs`.
  * `Alu.Guarantees` is one conjunction with a clause per opsel, so every chip's soundness proof must disprove every other chip's clause. Adding MUL, DVRM or BYTEWISE breaks LT, EQ and SHIFT, and the work grows with the square of the number of chips. Keying the guarantee on the decoded opsel, with one disjointness lemma, keeps each chip to its own clause.
  * `Counted.lean` defines its own `Normal` class next to Clean's `RawChannel.Normal`, used through dot notation in the same namespace; a distinct name (e.g. `CountedNormal`) would avoid confusion.

* **Slowness:**
  
  * `lean_diff` calls `eval_program_base` (which evaluates the whole program) once per constraint, so the cost grows with the square of the constraint count. Evaluating once per row and reading all roots gives the same check.
  * The export builds every channel's full table with `slc.table.toArray` (about 10M entries) to output 656 sampled rows, which costs about 60 s per CI run. The sampled entries can come from `Bitwise.row i` directly.

Thanks for the review. Addressed in the latest push, checked at 04449f1:

Not proven (1–4): new README section "What is intentionally not proven" listing the four limits in one place.

Differential test

  • 5: direction, multiplicity and fingerprint are now matched separately; a flipped direction fails the test.
  • 6: the Rust side of an interaction is now captured from emit_multiplicity / emit_fingerprint with the same CaptureBuilder as the constraints and evaluated at fixed z, α (eval_program_roots, new in stark); the Lean side folds the exported message into the same fingerprint. evaluate_with and combine_from are no longer used by the test, and evaluate_with is private again. BITWISE messages are compared the same way.
  • 7: both LT deltas are now found by content.
  • Slowness: the program is evaluated once per row, all roots read at once.

CI

Minor: Shift.Padding docstring fixed; eqComponent / pushStaticComponent docstrings say the generation is a placeholder; the three SplitMix64 copies now use test_rng.

Follow-up, not in this PR: the shared limb lemma, keying Alu.Guarantees on the decoded opsel, the Counted.Normal rename and the export speed-up all touch files that #1031 and the branches stacked on it rewrite (Lt/Eq/Shift, Alu, Counted, Bitwise), so I'd do them there rather than create conflicts here.

jotabulacios added a commit that referenced this pull request Nov 16, 2026
… credit EqLookup to its PR.

The section linked the fork by branch name while lakefile.toml pins by sha, so the two could drift,
and the link dies when the branch is renamed or deleted once the upstream PRs land. It also said
"Clean main f3abad7c plus three commits", which was true of the old pin and not of 73309694: that
one is the three commits plus the four review fixes answering the audit of them. And EqLookup,
described there as "in this same PR", is already in the model from PR #1027.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants