Repository navigation
WHIR no-epoch (prove-and-retire) prover: block 25368371 — work in progress - #1014
Draft
MauroToscano wants to merge 2146 commits into
Draft
MauroToscano wants to merge 2146 commits into
MauroToscano wants to merge 2146 commits into
Conversation
|
Benchmark Results for modified programs 🚀
|
|
Benchmark Results for unmodified programs 🚀
|
MauroToscano
force-pushed
the
noepoch/whir
branch
from
October 1, 2026 14:57
e9107c6 to
97ba486
Compare
added 30 commits
November 15, 2026 14:41
Under BlockFormat::memory = Fused, the walk makes one MEM_LOAD (MEM_STORE) row of each CPU load (store) whose MEMW message classify_memw sends to MEMW_A, in place of the LOAD (STORE) op and the MEMW_A row; any other stays as today. Every store is routed by the walk under the format (a STORE op for each one not fused), so no router routes one, and each window's STORE segment is its walk's. The fused rows' BITWISE lookups are in-walk lookups, counted where LOAD's are. Both tables stream as their own families (StreamTable::MemLoad and MemStore, after STORE), kept as tails when the streamed ops are dropped, rebuilt by the regenerator, and built by the finish at MaxRowsConfig::mem_load / mem_store rows; they sit right after STORE's tables in the AIR order. The block proof gains a last field, fused_memory: the two tables' counts. The statement absorbs a format word and the pair right after the table counts under the fused format alone, and the split format's verifier refuses any pair but [0, 0], so the field never rides unbound; a split proof's statement, transcript and archive before the field are unchanged. The fused format needs MEMW_A's limbs; the whole-run build and the block tree refuse it. BLOCK_WHIR_MEMORY=fused is the harness's word for it (prove-block-base takes it; prove-block refuses it).
…hecks Blocks under the fused memory format prove and verify (every width, sign and alignment with bytes of mixed timestamps, every instruction, KECCAK, the commit syscall); each memory format refuses the other's proof; a split proof stating any fused pair ((1, 0), (0, 1), (2^32 - 1, 0)) is refused, and with that refusal taken out verifies; and a split proof's archive is the archive of the proof before the fused field, then the sixteen bytes of [0, 0].
The check that every streamed family's chunk takes the AIR the verifier builds for its table now covers MEM_LOAD's and MEM_STORE's.
Two kernels (fill_mem_load, fill_mem_store) fill the tables that check an aligned CPU load or store in one row with its memory tokens, from one record a row: MemLoadRow (32 B: address, timestamp, the step's limbs as ts - old - 1, the result's bytes, width, sign) and MemStoreRow (40 B: address, timestamp, step, width, the bytes written and the bytes they replace). The columns are the address as MEMW_A's (half, half, word), the timestamp's two words, the width flags, the bytes, the sign bit (loads), the step's two 16-bit limbs and mu; padding rows are zero. The records kinds, RecordsFill and RecordsInput variants and the upload plumbing are added; nothing builds these records yet, and the column order follows the design until the tables' AIRs fix it.
…their checks card_load_rows / card_store_rows read MEM_LOAD's and MEM_STORE's records in place as the card's structs (the same offsets, asserted on both sides). The Rust mirrors drop the named padding: the prover's records leave theirs unwritten, and no side reads it. Checks: - each field the card reads is the prover's, on rows over every width, sign and step; - a host replay of fill_mem_load and fill_mem_store (mem_fill_host_kat.cpp, vectors from an ignored test: tables of 1 to 4096 rows, padded and whole) writes the host fill's words in every cell, and two mutants (the sign bit read from byte 7; the step's limbs swapped) each move a word; - card tests (box only): the group upload fills each table with the host fill's words, counted as card fills, and a changed record makes other words. The streamed hand-out of these chunks to the card follows once the walk makes the rows.
… rows MEM_LOAD's and MEM_STORE's full streamed chunks are held as the rows the walk wrote (RowFamily impls over MemLoadRow and MemStoreRow, read in place by the card's fill), under the same room rule as MEMW_A's: no host work on the upload path, the rows in the page-locked pool when it is on, and a chunk the room rule refuses filled on the host in the tail job. A chunk dropped as its rows is rebuilt with their digest. RowFamilies, CardFillStamps and LAMBDA_VM_BLOCK_CARD_FILL's families gain mem_load and mem_store; they hold nothing outside the fused memory format. The tables' words are the same. Tests: under the fused format, the whole build with every family held is the build without the card, table for table, over windows and walkers; each fused family asked alone is the only one held; the room rule sends fused chunks to the host; the test fault on their rows moves the asked family's tables only; a dropped chunk comes back as its rows.
…rows On the card (cuda builds): every chunk the fused build streams with every family held is, word for word, the chunk the same build makes without the card, each records chunk as the card fills it (counted as a card fill and equal to its host fill), MEM_LOAD's and MEM_STORE's in one window part and in several; every held row moved four cycles later makes the card fill other words. A box test reads a real block's first MEM_LOAD and MEM_STORE chunks at the production height against the build without the card. A fused block with every streamed family filled from its records, MEM_LOAD and MEM_STORE included, proves with the partition, tables and statement of the block built on the host, its bytes equal under the deterministic grind, and verifies.
… default The block's production format now checks an aligned CPU load or store in one row with its memory tokens (MEM_LOAD / MEM_STORE) instead of a LOAD or STORE row plus a MEMW message to MEMW_A. On the BLAKE3 base this removes about half a billion GKR leaves and a billion cells from a median block and shortens its prove. The default moves only where the format is proved without recursion: `prove-block-base` (a base-only process) reads an unset BLOCK_WHIR_MEMORY as the production format's fused tables. Every other process, the block tree behind `prove-block` among them, still reads it as the pinned split format, the only one the tree recurses over, and the verifier's pinned preset stays split. BLOCK_WHIR_MEMORY=split proves the split format on the base, the rollback. The tests that derive tree plans from the production format name the split format, and tests pin the defaults: production fused, pinned split, the knob's unset reading per process, and the fused format off the tree's presets.
…fier With MEM_LOAD's or MEM_STORE's streamed chunks held as their rows and each row moved four cycles later (its step kept), every fill writes the same wrong table and the prover makes a proof of a run that did not happen. The unchanged verifier refuses it as a bus imbalance, and accepts it with the balance check alone switched off, so the balance is the check that refuses; the same block with no fault verifies.
The pin on the block-wide LogUp bound builds its census with MEM_LOAD's and MEM_STORE's AIRs too (and asserts they are in it), so a fused table with more interactions a row or a longer message would move the pinned 1475 and 208 and fail the test.
… residual rows in the frame Two tests that fail if a verifier check under the in-row register format is removed: - the CPU and REGISTER AIRs built under the in-row format hand the verifier each CPU table's clock column and REGISTER's six columns (register index, initial and final files, PC flag); dropping either AIR's preprocessed columns fails it. - block_frame refuses a statement whose residual register tables state 2^28 + 4 rows and frames one at 2^28; removing the cap's call from block_frame, or selecting other tables for it, fails it.
The proved in-row forgery test counted any verifier error as a refusal and its forged builds kept stale BITWISE halfword counts, so a forgery could be refused for a count it never targeted. Each forged build now has its halfword counts recounted as its prover would, and each refusal must be the check the forgery targets (the bus balance, or the residual table's constraint), with the build verifying once that one check is switched off. Adds a proved block whose untouched register starts and ends at another value (initial and final words edited alike, every bus balanced): the verifier refuses it at REGISTER's preprocessed openings and accepts it with that check off, so the files bind only through those columns.
BlockStatement gained the fused memory pair after this test was written; the test frames a statement under the split memory format, where the verifier takes only [0, 0].
The production format is fused now, so the pin's off-default memory word is `split`, and the child process also runs under `fused`, off the verifier's pinned split format. Under each word the production format stays fused and the pinned format split (neither reads the knob), while the harness's and a base-only process's readings follow it.
MEMW checks no value, and several tables read RAM bytes with no range check of their own (ECREC's signature and hash, ECSM's base point, LOAD's result bytes). A census over the block verifier's AIR set now requires every value a MEMW sender writes to RAM to be a constant byte or a column the same table sends to the byte table, except a pinned allow-list that says why each exception is a byte anyway: COPY's value lanes (a copy of bytes already in RAM), ECREC's key and status, ECSM's result point and KECCAK's output state (replies of tables that check them). A second test shows the census fails when STORE loses its byte checks or a table gains an unchecked RAM write.
A memset row with dst equal to src writes its lanes at T and reads them back at T + 1 from the same cell, so the memory argument closes on its own write and the lanes are free. The new test builds such a row with non-byte lanes, shows it fails only the dst = src + 8 constraint and its no-wrap carry, and shows its write and read are the same cell at adjacent timestamps carrying the same non-bytes. The control, the same lanes at dst = src + 8, satisfies every constraint and reads another cell.
Two forged rows, each consistent in every relation, byte and carry: r read as r + N (with r = 2, N + 2 is a curve x below p, so x2, yR and their quotients are recomputed for it), and the tiny-y wrong root (the point (r, 1) requested with the even root, claimed as p + 1 instead of p - 1). Each is refused by its bound's closing carry and nothing else, like the existing tests for s, u1, u2, xA and xB. The two bounds' docs now say what each stops.
The keccak256 sponge keys its round-chip requests by (call timestamp, block index), with the block index in the slot where a permute call's key carries its timestamp's high word. The keys stay distinct only while every CPU timestamp has a zero high word, which the CPU row cap gives today. A compile-time assert beside the sponge's row cap now fails the build if the cap is raised so far that a timestamp reaches 2^32, and says why.
Nothing proves a bail: the prover may answer status 1 for any call, including a valid signature. The chip, the guest syscall and its one caller now say so where a new caller would read them: on status 1 the key must be recovered in software, which alone decides whether the signature is invalid.
Each test edits a hand-built KECCAK row (one keccak256 call of two blocks, plus padding rows, built without a guest run) so that exactly one check fails and every other constraint still holds: FIRST on a padding row, a read on a padding row, a first row off block 0, a PART bit of 2 (with the count and message byte moved to match), and a FIRST flag of 2. For the four checks the tests can leave out, the edited row satisfies every constraint once that check alone is left out.
…g mass Two tests on the small cut fixture of the packed LogUp-GKR: - The combination challenge of a cut table's chunk claims must be drawn after the bin's whole ladder; drawn earlier on both sides, every round trip and tamper test would still pass. A recording transcript now pins its place in the prover's transcript: one draw between the bus outputs and the ladder, and exactly the line challenge, every cut table's combination coordinates and the constraint point between the ladder's last halves and the tables' claims. - A new prover fault puts the bus's whole sum into a padding slot of a short last chunk of an unbalanced trace, with honest trees and ladder over that list, so the bus balances. The verifier, which defines the padding as 0/1 itself, refuses it in the constraint phase; with that check off the forgery is accepted, so it is that check alone.
The caps on COPY, KECCAK, KECCAK_RND, ECSM and ECDAS tables were tested only by calling the count helper directly. A frame-level test now hands the block frame a hand-built statement (no proof) with one family a table over its cap and expects the cap's refusal by name, with the same statement at one table of each framing as the control.
The forged-nonce tests lived only in the multilinear crate, and the block test of 18 against 20 grind bits refuses on the absorbed format, so it would pass with the nonce check deleted. A test-only prover fault, armed process-wide at one bit count by a guard, now makes every grind at that count absorb a nonce that fails the check and keeps the rest of the proof consistent with it. A block proved that way at 17 bits (a count no other test uses) must be refused while the honest block verifies; a chain-level test does the same at 13 bits and expects the grind's own refusal. With the nonce check removed the chain-level forgery is accepted.
The existing refusal test runs at a non-production format and only drops each cut family's last table. A box-tier test now proves at the pinned format and leaves out a family's last real-row table, a middle one where there are three or more, and all of them; each must be refused on the bus balance and accepted with the balance check off.
The block statement now carries the fused memory tables' counts; the frame test states none under the split format and one of each under the fused one.
The fused memory format's MEM_LOAD and MEM_STORE tables put their RAM tokens on the memory bus themselves, which the census of MEMW senders did not read, and the pinned AIR set holds no fused table. The census now adds one table of each fused kind and requires every new RAM token a table produces to carry a constant byte, a byte-checked column, or the value of a token it consumes; MEMW and MEMW_A, whose tokens relay the MEMW messages, are pinned with that reason. A mutant drops MEM_STORE's byte checks and must break the census.
The traces were built in the builder's default forms (32-bit register tokens) and proved under the pinned format (in-row registers), so the prover refused them before any table was left out. They are now built in the pinned format's register format, MEMW_A order and KECCAK_RND form, and each program first proves and verifies with nothing left out as the control.
The census AIR set now holds one MEM_LOAD and one MEM_STORE table, so every census in the file reads them. Their IS_HALF lookups (the span and the step's two limbs) weigh MU, a bit column in each; and since they check memory order themselves, sending no MEMW message, the high word of each RAM token they put on the memory bus is pinned as their own column received from the CPU's memory-op dispatch, where the CPU puts 0.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
One WHIR (multilinear) proof per block, with no epochs (the prove-and-retire / VADCOP shape). This is a second prover next to #1010's epoch-based one. #1010 stays the reference and this branch does not touch it. The branch starts from #1010's head
f3d359998; compare againstf3d359998to see only this work.What it is
prover::block_whir::prove_block_whir/verify_block_whir. The whole block is oneMultiProof:(z, α, β)are drawn once.S_post ‖ g: upload again, argue (LogUp-GKR), re-encode with the NTT alone, open. The dropped tree levels are rebuilt from the queried cosets and checked against the kept nodes.WindowedTraceBuilder(noepoch/windowed-builder, also used by STARK no-epoch (prove-and-retire) prover: block 25368371 — work in progress #1013). Every full chunk of CPU, MEMW_R, MEMW_A, MEMW, LOAD, LT, SHIFT and STORE is laid out and committed as soon as it exists.⚠ Format changes (all approved by Mauro, 10-02 (§7 on 10-05), but §6, the port of #1013's S0a; for the cryptography team's end review; ledger rows W-0…W-6, W-8, W-9 in CRYPTO-REVIEW.md)
The block is one WHIR proof (
BlockWhirProof, oneMultiProof) over all of the block's tables.z, α, β.S_post ‖ g).Every parameter below is a verifier-side constant. None is read from the proof.
BLOCK_GROUP_POLYSBLOCK_MAX_GROUPSBLOCK_MAX_TABLE_VARSBLOCK_MAX_KECCAK_RNDBLOCK_KECCAK_RND_MAX_VARSBLOCK_MAX_ECDASBLOCK_ECDAS_MAX_VARSBLOCK_MAX_KECCAKBLOCK_KECCAK_MAX_VARSBLOCK_MAX_ECSMBLOCK_ECSM_MAX_VARSArgueFormat::BATCHEDPRODUCTION_WHIR_GRIND_BITSLEAF_PERMS_CAP,BLOCK_FAN_IN1. The group partition is in the statement
What. The streamed prover packs tables into groups in the order their chunks complete. It writes the partition (per group, its tables' indices) into the statement. The transcript absorbs it before any root.
Verifier. The verifier checks:
It then rebuilds every group's stack layout itself.
Security. Proven bits are unchanged.
BLOCK_MAX_TABLE_VARS(change 3), so no partition can make a chain taller than 2^27.Proof bytes vary from run to run. Arrival order depends on thread timing, and the hash-ordered tables' row order (LT, EQ, BYTEWISE, BRANCH, MUL, DVRM) follows a hash state that is random per process. Mauro, 10-02: "the proof not having the same bytes it's fine, they never had the same bytes anyways".
LAMBDA_VM_FIXED_TRACE_HASH=1(fixed hash keys) andLAMBDA_VM_DETERMINISTIC_GRIND=1. Under both, two processes prove the same bytes (FAST 423).2. Prepared openings, one stack per group (approved by Mauro, 10-02: "it's a standard technique, go ahead")
In short. This is #1010's existing prepared DECODE opening, carried into the block format. It is extended to the dense genesis pages and stacked per group.
What. The recursion guest cannot evaluate DECODE's five 2^20 preprocessed columns, nor the dense genesis pages (millions of rows), on its own. So each group's prepared tables (DECODE and the dense genesis pages chosen by
genesis_stack_plan) have their leading preprocessed columns stacked into one commitment.verify_block_whir, andverify_block_tree's plan. The prover's own roots only shortcut the prover's side.z.Negatives, on a guest with two dense genesis pages. Each is refused by the host verifier and by the recursion leaf, and admitted when the openings are skipped (the mutation):
Wrongly committed stacks are refused at the roots block: two pages swapped, a page holding another page's columns, another program's DECODE.
Cost. One stack per group costs +0.14 s of base (FAST 419; one commitment per table cost +0.49 s) and saves 0.57 s of recursion, by taking 90 k permutations out of the leaves.
BlockFormat::prepared = falseturns the openings off, for measurement only.3. ECDAS split; table heights capped
What.
split_ecdas).(round, op).Why. At a median block, one ECDAS table (2^19) argues on a 12 GiB tree. At p90 (2^20) it stacks into five polynomials on a 24 GiB tree.
Gates. FAST 534 and 535: the negatives, 3 mutations caught.
4. Recursion leaf: the leaf cap (G3) and the inverse share (W1)
LEAF_PERMS_CAP. With no fixed leaf count, it adds leaves while the heaviest leaf is over the cap. The block's own partition is unchanged.p · ediv(1, q), which has no satisfying assignment for q = 0 whatever p is. The formerediv(p, q)left the share free at p = q = 0 (≈ 2^-160 under GKR soundness).LFM_WHIR_SHARE_INVERSE=0keeps the former form.5. Batched argue, on by default since d9d0ac5 (
BlockFormat::argue = ArgueFormat::BATCHED; Mauro, 10-02: "batch the constraints")What. Each group's tables are argued together on the group's fork:
Verifier side. The verifier derives the bins from the statement's shapes and its own cap; the variant is never read from the proof. The proof carries one
BatchedArgueper group inBlockWhirProof.argues, andproof.tablesis empty under this format. The recursion leaves verify the same format (lane i-batch2, N-4).Security at the block's measured inputs (D-BATCH §3.2's formulas; re-checked at FAST 424's inputs: |T| ≤ 60 tables a group, ≤ 51 trees a bin, ≤ 2^28.87 input cells, bus messages ≤ 204 elements, N_C ≤ 413, D_max 4, n ≤ 21; L = 300):
Every term is above the WHIR phase, so the proof minimum is unchanged: 128.946 under the campaign accounting (130.393 for the WHIR fold under the calculator of record).
Measured.
The knob.
ArgueFormat::PerTable(one argument per table, the format before) stays measurable, withBLOCK_WHIR_ARGUE=per-tablein the real-block tests. At the per-table format, the MultiProof, the prepared openings and the partition are digest-equal to the head before the batched merge (BIG 393).6. KECCAK and ECSM split; their heights capped (the port of #1013's S0a; landed at 3cfffe2; approved by the lead under Mauro's any-block target, Mauro to confirm)
What.
split_keccak,split_ecsm), as ECDAS is at 2^17.Why. A keccak-heavy block at the gas limit makes about 2^21 permutation calls. One KECCAK table that tall stacks into eight polynomials of 2^27, against a group budget of three. At its cap each table fits one polynomial.
Bytes. Every block measured so far has KECCAK ≤ 2^17 rows and ECSM ≤ 2^11, so it builds the same tables and the same proof (FAST 830: digest 07d1bd43… unchanged on block 25368371).
Gates. FAST 830: KECCAK and ECSM forced into 4 tables each on block 25368371 prove and verify, base and tree; the count and height negatives; mutation C caught.
7. WHIR grinding 20 → 18 bits, queries 112 → 114 (landed at cbfa7d7; approved by Mauro, 10-05)
What.
Each WHIR chain grinds 18 bits instead of 20 before each round's queries, and opens 114 queries instead of 112 to buy the two bits back. At every block height (15 to 27 variables) Q =
num_queries(2, rounds, 128, 18).One site,
multilinear_prove::chain_config_under. Every side of the block reaches it throughBlockFormat::chain_config:verify_block_whir;The statement absorbs Q and the grind bits.
Scope: the block's WHIR proofs only. The recursion tree's own leaf, node and top proofs are STARK proofs (FRI) and keep their own grind.
It ports WHIR recursion on GPU (RPX) with ZisK-style proof formats: block 25368371 in 32.62 s #1010's grind 18 (landed there at 2687e48). Opt-out:
LAMBDA_VM_ZF_WHIR_GRIND_BITS=20, which reproduces the previous proofs byte for byte.Security. The calculator of record: Johnson bound, η = 1/300, the fold proof of work unground.
Measured behind the knob, on c55dec4, one binary per box:
Bytes. The proof bytes change. Under the fixed trace hash and deterministic grind, the 1× top proof's digest is b9b4826d… at the default and 0da6fea9… under
=20. Both trees are accepted (ULTRA 015, at this head).Known limit. The leaf cap (279,000 permutations) sits above 2^18 LFM_HASH rows, and 18 bits moves every leaf about 1.7 % closer to that doubling.
Gates.
zf_format::; the chain refusals and the legacy-bytes KAT; the pins the flip moved; the host block refusal.One-binary A/B: the block tree vs #1010's epoch tree (FAST 416, noepoch/whir @ 454dad6)
Eight arms E B B E E B B E on the same binary (md5 checked after every arm):
Base, block 25368371 on FAST (each run against #1010's epoch base on the same binary)
Recursion (W3): proved and verified on block 25368371 (FAST 413)
lfm::whir_block::WhirBlockPlan) runs the host verifier's own statement checks, derives every shape from the AIR and every prepared root from the program, and never reads a proof. It prices each group in permutations and partitions the groups over the leaves (heaviest first, onto the least-loaded leaf).block_node, shared byte for byte with STARK no-epoch (prove-and-retire) prover: block 25368371 — work in progress #1013). They check the id, the state and the output equal across children, add the shares, and the top asserts zero.verify_block_tree(elf, statement, top)derives the top program and checks the top proof against it. Every preset is pinned inside: the base's options, the block format, the tree's options, the leaf rule and the fan-in.Every proof was verified by the harness off the clock. The base is unchanged by the emission running beside phase B (20.98 s vs 21.18 s in the arms without the change).
Next:
Base levers after the A/B (FAST 417–422, each S/P on one binary)
Memory (D-MEMORY, with i-mem / i-mem2)
drop_streamed_ops)layout_ahead, D-EXEC)Narrow trace storage (M4, lane i-m4)
Prover-only; the proof bytes are the same. The proof digest equals the head's before M4 (07d1bd43…) under the fixed trace hash + deterministic grind (FAST 820).
BlockOptions::narrow, productionNarrowing::CARD).Max RSS, wide → narrow (BIG 560; the traces themselves shrink about 4×, e.g. 24.89 → 6.20 GiB at 1×):
Before M4, #1014 ran out of memory at 3.03× (i-m4); the narrow line puts the 120.7 GiB edge near 5.5–6×.
Phase A uploads the next group beside the commit (3′, lane i-noepoch-w2)
Prover-only; the proof bytes are the same (digest 07d1bd43… at the default, FAST 832; the commits, their order and their bytes are unchanged).
BlockOptions::upload_ahead, on by default;BLOCK_WHIR_UPLOAD_AHEAD=0is the control).Streamed chunks laid out on three threads (E3 on by default, lane i-noepoch-w2)
Prover-only; the proof bytes are the same (digest 07d1bd43… at the default, FAST 838 and 835; the groups, their order and the packing are unchanged).
layout_workers0 → 3 inBlockOptions::production(59e9890). The streamed chunks are laid out on three threads, with at most K + 1 = 3 of them unpacked at a time (layout_ahead, E3), instead of on the builder's one layout thread.BLOCK_WHIR_LAYOUT_WORKERS=0(the inline layout, 01da99f's) is the control.Phase B's kept-top paths re-hashed in parallel (lever 1, lane i-noepoch-w2)
Prover-only; the proof bytes are the same (digest 07d1bd43… at the default, FAST 840 at 733557a — 82f9046 adds only the knob; the paths, their order and the refusal are unchanged).
BLOCK_WHIR_REHASH_SERIAL=1restores the serial re-hash (a measurement knob).BLOCK OPEN SPLIT(withLAMBDA_VM_BASE_SPLIT=1), phase B's openings by host stage and the kept-top gather and re-hash seconds.The tree's leaves execute while their artifacts are built (R2-i, lane i-noepoch-w2)
Prover-only; the proofs are the same (the top proof's digest equal with and without it, 0da6fea9…, under the fixed trace hash + deterministic grind, FAST 841).
prove_tree_pipelined, the way a prover would run it; there is no production tree driver yet). Its level 0 built all three leaves' artifacts first (three serial card holds, 0.24 s at 1×) and only then let the leaves execute and fill their traces (0.46 s until the first was ready) while the card sat idle (nsys timeline, FAST 839). Execution and fill do not read the artifacts; only the prove does.lfm_proveis now cut into its host half (lfm_execute_and_fill) and its card half (LfmFilled::prove, which asserts the artifacts' hasher is the one the traces were filled for), in the same order; the harness builds the artifacts on a thread of their own and each leaf executes and fills beside them, waiting for its artifacts only to prove.W3_EXEC_BESIDE_ARTIFACTS=0is the control (the order before).A second whole-block field. The W3 readout prints, on the clock between the base and the tree, the block report, a second block frame for the LT heights and the group tables: 0.13–0.14 s at 1× that a prover would not spend. "Whole block" keeps its meaning (every number above includes them); from d169edd on,
W3 RECURSIONalso prints "whole excl. harness readouts" beside it (at d169edd, FAST 841 + 842 pooled: whole 18.54 s, whole excl. harness readouts 18.40 s).The rest laid out in waves; KECCAK_RND built as its tables (W, lane i-m4b)
Prover-only; the proof bytes are the same (digest 07d1bd43… under the fixed trace hash + deterministic grind: FAST 854 at 0bcc1fc, and FAST 855's W arm on this head's code).
BlockOptions::rest_layout_bytes). The tables, their order and the groups are the same (test).split_keccak_rndthen copied into its 2^16-row tables. At 4.13× that copy was a second peak of the same height. The finish now builds the 2^16-row tables directly (BlockOptions::finish_keccak_rnd_chunks), the same tables as the split's (test), and hands none out during the windows, which would change the groups.BLOCK_WHIR_REST_LAYOUT=allandBLOCK_WHIR_KR_FINISH_CHUNKS=0restore the old behaviour.pack_rest_as_laid_out) was re-measured with the waves: no gain (+0.01 s), so it stays off.LAMBDA_VM_BLOCK_MEMLOG=1(off by default). It prints the host memory term by term every half second and at each phase mark, with the line at jemalloc's active peak and each arena's bytes (those two in the lib tests, which install jemalloc).BLOCK REST TABLES(each rest table's layout end and group) and each group's commit start (from@).The tree's first finished leaf executes during phase B (R2-ii, lane i-noepoch-w2)
Prover-only; the proofs are the same (base digest 07d1bd43… and the top proof's digest equal with and without it, 0da6fea9… — FAST 841's — under the fixed trace hash + deterministic grind: FAST 845, and FAST 847 again on fac261f, the merge with W).
block_prove_on_forks_observed; the old entry point passes a no-op, and the proof is the same with any observer). A leaf's arena is built from its groups' words alone (group_arena_words+leaf_arena;block_leaf_arenais built from them). The W3 harness turns the groups into words as they arrive; the first leaf whose groups are all opened while another group is still to come (leaf 0 = groups 1, 5, 6 at 1×, done after group 6) executes and fills its traces on a thread of its own, about 1.5 s before the base ends, and the tree picks it up. Off the clock its arena is checked against the finished proof's.W3_LEAF_DURING_PHASE_B=0is the control.The finish's tables packed as they are built (b2, lane i-m4b)
Prover-only; the proof bytes are the same (digest 07d1bd43… under the fixed trace hash + deterministic grind at both arms: FAST 855 at 0cebe16, and FAST 857 again on 365e3ab, the merge with R2-ii).
BlockOptions::pack_finished). KECCAK_RND is packed in waves of four 2^16-row tables. Phase A takes those tables as narrow columns and uploads them as they are, so the card packs only the groups' other tables. Nothing packed is transposed.BLOCK REST PACKEDcounts the rest: 187 of 191 tables at 4.13×).BLOCK_WHIR_PACK_FINISHED=0restores W.Measurement posture: the card's pool now retains freed memory (baseline shift)
Not a prover change; the proofs are the same (the top proof's digest 0da6fea9… under both postures, fixed trace hash + deterministic grind, FAST 848; 351a773 adds only a test readout).
LAMBDA_VM_MEMPOOL_RELEASE_MB=0. The card's stream-ordered memory pool then hands its freed blocks back to the driver at each synchronize, so a VRAM sampler's total − free reads the live working set. The code default is to retain every freed block (DEFAULT_MEMPOOL_RELEASE_THRESHOLD_BYTES = u64::MAXin math-cuda), and that is what a prover runs.cuStreamSynchronizeandcuMemAllocAsyncwhile the card was idle was ≈ 0.25 s in phase A, 0.42 s in phase B (0.26 s of it in the encode: one 18–33 ms gap per group) and 0.34 s in the tree.W3 DEVICEline).The prover refuses a partition over the group maximum (lane i-m4b)
Prover-only; the proof bytes are the same (digest 07d1bd43…, FAST 858). The prover now refuses a block over
BlockFormat::max_groupsas its groups close, with the verifier's ownInvalidTableCountserror, instead of proving a block the verifier refuses. The median block (90 groups against the cap of 64) spent its whole ≈ 190 s base on such a proof (BIG 564). A test covers both prover paths, with a mutation for each check.The tree's nodes built beside it, each level proving as its nodes arrive (A, lane i-noepoch-w2)
Prover-only (the W3 harness); the proofs are the same (the top proof's digest 0da6fea9… — FAST 841's — with and without it, under the fixed trace hash + deterministic grind: FAST 889, and FAST 891 again after the permit fix below).
prove_tree_pipelined) built every node program and its artifacts on one thread, level after level, and level 0 joined that thread before any node proved, so level 1 waited for the whole tree's builds. At the median block, 32 s of serial node builds after the leaves' artifacts held level 0 open 10.7 s past its last leaf proof, and the card sat idle 11.4 s before level 1 (BIG 565 / 568's card trace). At 2.66× the wait was 1.5–2.1 s (FAST 888).W3_EMIT_THREADSthreads (default 4, the builder's own: on the global pool a prover's join can steal an emission and leave the card idle, STARK no-epoch (prove-and-retire) prover: block 25368371 — work in progress #1013's FAST 454), and the builder's thread builds each one's artifacts as it arrives. Each node proves once its children are proved and its slot is filled. A builder that stops fails every empty slot.W3_NODE_PIPE=0is the control.Published<T>serves as the slot; there is no per-group early emission (W3 builds every leaf's artifacts up front, so a level's programs are emitted together); and there is no host-only thread marking (WHIR no-epoch (prove-and-retire) prover: block 25368371 — work in progress #1014 has none).build_levels, an emit on the pool and a finish on the builder's thread). A toy tree at the median's shape puts the serial builder's programs in their slots whatever order a pool finishes them in, and finishes every node off the rayon workers (test; finishing on a worker, or publishing in arrival order, fails it). A failed build fails every slot above it without a hang (test).WhirBlockPlan::programsbuilds as rayon jobs.W3_NODE_TIMES=1(off by default). It prints, per program, when it was built, taken and proved; its prove's split (execute, fill, the prove net of the card wait, the wait); and each node level's start against the level below. The stamps are taken either way and cost a handful ofInstantreads.prove_tree_pipelinedis its de-facto driver. With the slots it now has a driver's shape: leaves from phase B, nodes from slots, levels as their nodes and children arrive. Moving it out of the test harness is a landing item for STARK no-epoch (prove-and-retire) prover: block 25368371 — work in progress #1013 and WHIR no-epoch (prove-and-retire) prover: block 25368371 — work in progress #1014 both.Each node proved as soon as its own children are (C, lane i-noepoch-w2)
Prover-only (the W3 harness); the proofs are the same (the 1× top proof's digest with and without it, under the fixed trace hash + deterministic grind, both trees accepted: 0da6fea9…, BIG 614 — the digest FAST 841 / 889 / 891 printed).
siblingsworkers proves every program of the tree, leaves and nodes, taking them in topological order (the leaves, then each node level) and publishing each proof into a slot of its own. A node waits only for its own children's slots, so it executes, fills and proves while the rest of the level below still proves.W3_DATAFLOW=0is the control: each node waits for its whole level below (tested never to start early).W3 LEVELwall now runs from the level below's last proof to its own.The held tables spill to disk, by default only when memory is short (lane i-m4b)
Prover-only; the proof bytes are the same: one shape and one proof digest (07d1bd43…) across the base and all four policies, under the fixed trace hash + deterministic grind (BIG 616). The 1× top proof's digest (0da6fea9…) and the tree verify at the landed merge (ULTRA 006).
crypto/stark/src/spill.rs) is imported byte-identical; it digests each slot and writes it with O_DIRECT on a writer thread, off the committer's path. A table moves only when the writer's queue has room at that moment, so the committer never waits.SpillFailed). After a group's opening its packed columns are let go.LAMBDA_VM_BLOCK_SPILL=auto(the default) |off|always|<GiB>(a resident budget).autospills a table when the host's bytes plus a reserve plus the table would pass the target.LAMBDA_VM_BLOCK_SPILL_TARGET_GIB, else min(the cgroup's limit, v2 or v1;MemTotal) − 10 GiB.BLOCK SPILLprints the policy, the slots and bytes, the writer's split, the read-back, and the largest host readingautodecided on.always(BIG 566, single runs): 22.74 GiB spilled, max RSS 52.53 → 40.12 GiB, whole block +1.15 s, drivers waited 0 s.auto(BIG 567): by default nothing spills (target 110.7 GiB). Under a 35 GiB target it spills 21.21 GiB, max RSS 41.04 GiB, +2.91 s.autospilled 26.98 GiB from group 44 of 90, and the tree verified.auto, O =off, S =always, T =autoat a 47.5 GiB target):autorun spilled 0 in both jobs, at 110.7 GiB and under the 47.5 GiB target (FAST's limit, kept as a real test on BIG's 128 GiB; the host read 25.3–26.8 GiB across those runs);alwaysandautoat 0 spilled within noise.autospilled 0 (host at most 27.65 GiB).Phase 4 counts the BITWISE lookups in slices (#1013's c369843, lane i-m4b)
Prover-only; the bytes are the same:
What changes:
LAMBDA_VM_P4_SLICED=0keeps the whole-source collectors. The knob is WHIR no-epoch (prove-and-retire) prover: block 25368371 — work in progress #1014's only, for the A/B.Measured: the median, BIG 619. One binary on the cap-128 exploration head, N P P N,
autospill on. N runs the whole-source path, P the slices.auto)autospills only once memory is short.The top executes each child's share as that child is proved, while a child is still unproved (B and B′, lane i-noepoch-w2)
Prover-only (the W3 harness and the LFM executor); the proofs are the same. The 1× top digest is 0da6fea9… with the streamed top on and off, under the fixed trace hash + deterministic grind, and both trees are accepted (BIG 617; ULTRA 008 and 009; ULTRA 011 at c55dec4 itself).
executor::StreamedExecutionruns an execution as its arena groups land: each instruction runs in the wave in which the last group it reads lands (each instruction's group set comes from one forward pass), in the level schedule's order, into the slot the schedule gives it. So the witness isexecute's, word for word.finishchecks that every instruction ran once. Dropping the group propagation (ReadBeforeWrite) or re-running instructions across waves (DoubleWrite) fails them.W3_STREAM_TOP, on by default), so only the last child's share and the fill remain after the last child.W3 TOP EXECUTEprints which. A test covers a program ready before the last child (streamed), after it (whole), and a failed child (whole, which then reports the failure). Inverting the predicate fails the test.A node executes from its program; only its prove waits for its artifacts (b′, lane i-noepoch-w2)
Prover-only (the W3 harness); the proofs are the same: the 1× top digest is 0da6fea9… with
W3_EXEC_EARLYon and off, under the fixed trace hash + deterministic grind, and both trees are accepted (ULTRA 012; ULTRA 013 at 806c211 itself).node_flow). The traces are filled under the block hasher, which the node artifacts are built for. The top's streaming choice (B′) is taken when the top can execute.W3_EXEC_EARLY=0is the control.LfmProveError::HasherMismatch, naming both) instead of failing a release assert, and the streamed execution's finish returns an error instead of asserting its public words' capacity. Each has a test and a mutation.