feat(autobahn): introduce EpochTrio for multi-epoch handling (CON-358)#3736
feat(autobahn): introduce EpochTrio for multi-epoch handling (CON-358)#3736wen-coding wants to merge 57 commits into
Conversation
PR SummaryHigh Risk Overview Proposal / QC verification now resolves AppQC committees from Prev or Current (lag-by-one), not only the current proposing epoch. Availability tracks Consensus restores and advances its own Restart: Reviewed by Cursor Bugbot for commit c82809f. Bugbot is set up for automated code reviews on this repo. Configure here. |
|
The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).
|
wen-coding
changed the title
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
![seidroid[bot]](https://avatars.githubusercontent.com/in/2745135?s=60&v=4){kind=small}
There was a problem hiding this comment.
The EpochTrio refactor is broad and mechanically clean, but bounding epoch 0 to a finite road range now forces a real epoch transition at road 108,000, and the consensus trio-rotation has an off-by-one that will stall consensus at that boundary; there are also committee-staleness and nil-deref latent defects in the epoch-transition machinery.
Findings: 5 blocking | 4 non-blocking | 4 posted inline
Blockers
- Cached placeholder committees survive real epoch registration (Codex P1). registry.TrioAt() generates genesis-committee placeholder epochs and inserts them; AddEpoch() later replaces only the map pointer, so EpochTrio values already cached in the avail/consensus/data
inners keep pointing at the stale placeholder *Epoch. advanceEpoch() additionally early-returns whenever the road index is still inside Current's range, so it never refreshes a staleNext. Once a future epoch's committee actually differs from genesis, next-epoch vote weighting (avail reweightForNextEpoch / pushVote uses trio.Next.Committee()) and QC verification will use the wrong committee. Consider having AddEpoch mutate the existing epoch object in place, or make advanceEpoch/TrioAt re-fetch Next from the registry rather than trusting the cached pointer. - Cursor's second-opinion review (cursor-review.md) and the repo REVIEW_GUIDELINES.md are both empty, so no Cursor findings or repo-specific standards were available to incorporate; Codex's two findings were reviewed and are corroborated above.
- Missing test coverage for the actual epoch-boundary transition: the new tests (TestInsertQCCrossEpochFallback, TestPushCommitQCCrossEpochFallback, TestAdvanceEpochTrio) only exercise the registry.TrioAt fallback with artificially-constructed trios, and every other test uses SingleEpochTrio. No test drives consensus across a real road-108,000 boundary (committing the last road of epoch e and proposing the first road of epoch e+1), which is exactly where the P0 off-by-one manifests.
- 2 blocking issue(s) flagged inline on specific lines.
Non-blocking
- Signature verification was moved under the hot inner lock in avail PushBlock and PushVote (previously done outside via VerifyInWindow). Holding s.inner across ed25519/BLS verification serializes all per-lane block/vote pushes. Prefer snapshotting the committee(s) under the lock and verifying outside, as PushCommitQC and data.PushQC already do.
- avail.PushAppQC verifies against inner.epochTrio directly without the cross-epoch registry.TrioAt fallback that PushCommitQC uses, so an appQC/commitQC whose epoch is outside the cached trio would fail verification; confirm this cannot occur or add the same fallback for consistency.
- 2 suggestion(s)/nit(s) flagged inline on specific lines.
| iSend.Store(inner{persistedInner: persistedInner{CommitQC: utils.Some(qc)}, epoch: i.epoch}) | ||
| // Reuse the cached trio when the QC stays within the current epoch. | ||
| trio := i.epochTrio | ||
| if !trio.Current.RoadRange().Has(qc.Proposal().Index()) { |
There was a problem hiding this comment.
[blocker] Off-by-one at the epoch boundary (Codex P0). This condition keys on the committed QC's index, but the trio must cover the next view, which is qc.Proposal().Index()+1 (= View().Index). When committing the last road of epoch e (Index() == Last_e), Has(Last_e) is true, so the trio stays at epoch e; but View() then stamps the next view (road Last_e+1, belonging to epoch e+1) with epoch e's index. That proposal fails Proposal.Verify's road-range check on every node → consensus stalls. Since this PR bounds epoch 0 to [0, EpochLength-1] (was OpenRoadRange), the chain will hit this at road 108,000. Advance based on NextIndexOpt(qc) / the new View's index instead: if !trio.Current.RoadRange().Has(qc.Proposal().Index()+1).
| return fmt.Errorf("unknown epoch_index %d", anchor.Proposal().EpochIndex()) | ||
| var anchorLanes func(func(types.LaneID) bool) | ||
| for inner := range s.inner.Lock() { | ||
| ep, _ := inner.epochTrio.EpochFor(anchor.Proposal().EpochIndex()) |
There was a problem hiding this comment.
[blocker] The ok from EpochFor is dropped and ep is dereferenced unconditionally. EpochFor returns (nil, false) when the epoch isn't in the trio, and Epoch.Committee() dereferences its receiver, so this nil-panics if the prune anchor's epoch has fallen outside the cached 3-epoch window. The previous code returned an explicit unknown epoch_index error here — restore an if !ok { return ... } guard.
| return fmt.Errorf("block.Verify(): %w", err) | ||
| } | ||
| for inner, ctrl := range s.inner.Lock() { | ||
| if err := p.Msg().Verify(inner.epochTrio.Current.Committee()); err != nil { |
There was a problem hiding this comment.
[suggestion] PushBlock verifies the proposal only against epochTrio.Current.Committee(), whereas PushVote (below) accepts votes from either Current or Next. newInner explicitly initializes block queues for next-epoch-only lanes "so that early votes and proposals from validators joining in the next epoch are not rejected" — but a next-epoch-only proposer's block fails p.Msg().Verify(current) (HasLane) here and is rejected, contradicting that intent. Verify against Next as a fallback too, or drop the next-epoch block-queue init if only current-epoch proposals are intended.
| if err := vote.Msg().Verify(currC); err != nil { | ||
| if err := vote.Msg().Verify(nextC); err != nil { | ||
| return fmt.Errorf("vote.Verify(): %w", err) | ||
| } |
There was a problem hiding this comment.
[suggestion] VerifySig (and Msg().Verify above) now run while holding the inner lock. These crypto checks were previously performed outside the lock; on the hot vote path this serializes all pushes behind signature verification. Snapshot currC/nextC under the lock and verify after releasing it.
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Additional findings (outside current diff — PR may have been updated during review):
-
🔴
sei-tendermint/internal/autobahn/avail/state.go:432-441— PushBlock/PushVote regressed on two fronts vs. pre-PR: (1) signature verification (Ed25519/BLS) now runs insides.inner.Lock(), serialising every peer block/vote against PushCommitQC, PushAppVote, ProduceLocalBlock, WaitForLaneQCs, RecvBatch, etc.; the pre-PR code intentionally routed this throughregistry.VerifyInWindowoutside the lock. (2) PushBlock verifies only againstinner.epochTrio.Current.Committee()— unlike sibling PushVote which does Current→Next fallback — so signed lane proposals from Next-only validators (whose queuesnewInnerexplicitly allocates) get rejected before enqueue. Fix by mirroring the pattern already used inPushCommitQC(state.go:277-287): snapshot the trio under a brief lock, run Verify/VerifySig outside the lock, then re-lock to enqueue.Extended reasoning...
The regression\n\nBefore this PR,
PushBlockandPushVoteverified signatures vias.data.Registry().VerifyInWindow(...)outsides.inner.Lock(). After this PR (state.go:435-441 and 486-497), bothp.Msg().Verify(committee)and the CPU-heavyp.VerifySig(committee)execute inside thefor inner, ctrl := range s.inner.Lock()iterator body.\n\ns.innerisutils.Watch[*inner](state.go:36).Watch.Lock()(libs/utils/mutex.go:214-220) acquiresw.ctrl.mu.Lock()— a plain exclusive mutex, not an RWMutex — and holds it for the entire iterator body. Ed25519 verify is order-of-tens-of-microseconds per call; BLS is heavier. That work now serialises against roughly 26 other call sites (PushCommitQC,PushAppQC,PushAppVote,WaitForLaneQCs,ProduceBlock,RecvBatch, the persister loop,fullCommitQC, …) which all take the same mutex.\n\n### Concrete failure scenario\n\nA validator receives sustained peer traffic — say 100 blocks/s from lane producers and 3000 votes/s from replicas across lanes. Each verify is ~50µs of Ed25519 (worse for BLS). Under the new code the mutex is held for roughly (100+3000)×50µs ≈ 155 ms of wall-clock per second across PushBlock/PushVote alone, blocking every consensus wake-up (ctrl.Updated()) and every others.innerwriter during those windows. Tail latency on QC advancement (PushCommitQC), lane-QC construction (laneQC via RecvBatch), and app-vote acceptance (PushAppVote) all spike. PushVote is worse than PushBlock — it may run Verify+VerifySig twice (Current, then Next fallback) inside the same lock scope.\n\n### The correct pattern already lives in this file\n\nPushCommitQC(state.go:277-287) snapshots the trio under a brief lock, runsqc.Verify(trio)OUTSIDE the lock, and then re-enters to store.PushAppVote(state.go:308-319) uses the same pattern to pull the committee out beforev.VerifySig.Committeeand*Epochvalues are immutable (onlyinneris atomically replaced), so snapshotting Current+Next committees before the verify is safe. Mirroring this pattern in PushBlock/PushVote is a few lines and closes the regression.\n\n### Secondary issue: PushBlock lacks the Next-committee fallback\n\nnewInner(inner.go:76-84) explicitly allocatesblocksandvotesqueues for lanes present only in the Next epoch, with the comment "so that early votes and proposals from validators joining in the next epoch are not rejected."PushVoteimplements the Current→Next fallback matching that intent (state.go:486-497).PushBlockdoes not — it verifies only againstCurrent(state.go:435-441).BlockHeader.Verify(block.go:75-80) fails on!c.HasLane(h.lane), so a signed lane proposal from a Next-only validator is rejected before it can be enqueued in the pre-allocatedinner.blocks[lane]slot.\n\nStep-by-step: trio at road 0 with Current=ep0 committee {A,B,C,D} and Next=ep1 committee {B,C,D,E} (E is new).newInnerallocatesinner.blocks[E]andinner.votes[E]via the Next-only lane loop. E signs and pushes aLaneProposalfor its lane.PushBlockreachesp.Msg().Verify(currC)→BlockHeader.Verify→!currC.HasLane(E)→ returns"E is not a lane".inner.blocks[E]stays empty; peers whose PushVote fallback accepted votes for that would-be block now hold votes referencing a header that will never exist. When the transition completes and Current advances to ep1, E has to re-propose and any prior votes are useless. Note:Registry.AddEpochis only invoked from tests today (production always fabricates Next as a placeholder sharing the genesis committee), so this doesn't manifest in production yet — but the plumbing this PR sets up is explicitly for real epoch transitions.\n\n### Suggested fix\n\nBoth issues fold into the same edit — mirrorPushCommitQC:\n\ngo\nvar trio types.EpochTrio\nfor inner := range s.inner.Lock() {\n trio = inner.epochTrio\n}\ncurrC, nextC := trio.Current.Committee(), trio.Next.Committee()\nif err := p.Msg().Verify(currC); err != nil {\n if err := p.Msg().Verify(nextC); err != nil {\n return fmt.Errorf("block.Verify(): %w", err)\n }\n}\nif err := p.VerifySig(currC); err != nil {\n if err := p.VerifySig(nextC); err != nil {\n return fmt.Errorf("block.VerifySig(): %w", err)\n }\n}\nfor inner, ctrl := range s.inner.Lock() {\n // ... existing enqueue logic ...\n}\n\n\nSame shape forPushVote. This restores the pre-PR concurrency behaviour and givesPushBlockthe Next-fallback thatnewInnerwas built to support.
There was a problem hiding this comment.
Well-structured infrastructure PR that introduces the EpochTrio abstraction and fixes the live-path epoch advancement / restart-trio derivation; the security-critical QC verification correctly binds epoch_index ↔ committee ↔ road-range, and I found no reachable-today blockers. The two second-opinion findings are either about not-yet-reachable future work or contained by downstream checks, so they land as non-blocking notes.
Findings: 0 blocking | 4 non-blocking | 1 posted inline
Blockers
- None at the file/PR level.
Non-blocking
- Cursor's second-opinion review file (cursor-review.md) is empty — that pass produced no output. Codex produced two findings, both incorporated below.
- Codex (High): the EpochTrio caching design means a cached trio retains placeholder (genesis-committee) epochs generated by Registry.TrioAt, and reweightForNextEpoch/advanceEpoch never create new per-lane queues (blocks/votes/nextBlockToPersist/persistedBlockStart) for validators that only join in a later committee — so once real committee changes exist, new members' lanes would be permanently rejected with ErrBadLane. This is a genuine limitation but is NOT reachable in this PR: Registry.AddEpoch is defined but never called anywhere in production (confirmed by grep), so all epochs currently resolve to the genesis committee placeholder, and inner.go:14-18 already documents this exact gap as an explicit TODO. Recommend ensuring that follow-up (dynamic committee wiring) adds lane-queue creation on epoch transition and the avail runPersist cross-epoch-lane union (state.go:769-771 TODO) before AddEpoch is ever invoked in production.
- avail.PushVote (state.go:497-531) verifies the LaneVote message and its signature against independently-chosen committees (message may validate against Current while the signature validates against Next), unlike PushBlock (state.go:443-453) which pins a single committee for both checks. Exploitability is limited today because currC==nextC in practice (no committee changes yet) and blockVotes.pushVote / inner.laneQC re-check membership via per-epoch weights, so no forged current-epoch LaneQC is possible; still worth tightening for consistency once committees can differ.
- 1 suggestion(s)/nit(s) flagged inline on specific lines.
| h := vote.Msg().Header() | ||
| for inner, ctrl := range s.inner.Lock() { | ||
| currC, nextC := inner.epochTrio.Current.Committee(), inner.epochTrio.Next.Committee() | ||
| if err := vote.Msg().Verify(currC); err != nil { |
There was a problem hiding this comment.
[suggestion] Consistency/robustness: unlike PushBlock (which pins a single committee for both the message and signature checks), PushVote validates vote.Msg().Verify and vote.VerifySig against independently-selected committees — the message can pass against currC while the signature only passes against nextC (or vice-versa), admitting a vote that is not valid under any single committee. It's not exploitable today (committees don't yet diverge, and blockVotes.pushVote/laneQC re-gate by per-epoch weight so no forged current-epoch QC results), but consider mirroring PushBlock: select one committee via Msg().Verify, then use that same committee for VerifySig.
Superseded: latest AI review found no blocking issues.
There was a problem hiding this comment.
A large, well-tested refactor replacing single-epoch *Epoch lookups with an EpochTrio/EpochTrioCursor abstraction across consensus/data/avail/p2p. The mechanics are sound for the current fixed-committee case, but the newly-added epoch-transition machinery has two latent correctness gaps (surfaced by Codex) that will bite once membership-changing transitions are actually wired up, plus a lock-scope regression in avail PushVote.
Findings: 1 blocking | 6 non-blocking | 2 posted inline
Blockers
- None at the file/PR level.
- 1 blocking issue(s) flagged inline on specific lines.
Non-blocking
- Performance/concurrency regression in
avail/state.goPushVote: signature verification (vote.Msg().Verifyandvote.VerifySig) now runs while holdings.inner.Lock(). The previous code verified outside the lock viaRegistry().VerifyInWindow(...).PushBlockin the same file still verifies outside the lock and only takes the trio under the lock —PushVoteshould follow the same pattern to avoid serializing expensive signature checks under the state mutex. cursor-review.mdis empty — the Cursor second-opinion pass produced no output; only Codex's review was available to merge.REVIEW_GUIDELINES.md(pulled from the base branch) is empty, so no repo-specific review standards could be applied.- Minor: several exported
Verifymethods lost their doc comments (BlockHeader.Verify,Block.Verify,LaneVote.Verify,LaneRange.Verify,View.Less,TimeoutVote.Verify, etc.). Not a lint failure under the configured linters, but the removals are incidental to the refactor and slightly reduce documentation. registry.TrioAtinserts generated placeholder epochs into the shared map permanently (epochAt); benign given road indices come from verified QCs, but worth a note that far-future lookups would grow the map.- 1 suggestion(s)/nit(s) flagged inline on specific lines.
| // TODO: filter votes per-epoch committee once epoch transitions are wired up. | ||
| // reweightForNextEpoch refreshes the per-epoch weight cache in every stored | ||
| // blockVotes entry for the given next epoch. Called once per epoch transition. | ||
| func (i *inner) reweightForNextEpoch(nextEpoch *types.Epoch) { |
There was a problem hiding this comment.
[blocker] reweightForNextEpoch only recomputes weights for queues that already exist; it never creates votes/blocks queues for lanes first introduced by the new Next epoch. newInner seeds queues for the starting Current ∪ Next lanes, but after an Advance (here and in PushCommitQC), the trio's Next becomes a further epoch whose brand-new lanes have no queue. A valid vote/block from such a lane then passes committee verification against Next but fails the inner.votes[h.Lane()] / inner.blocks[...] lookup with ErrBadLane (avail/state.go PushVote/PushBlock). This is latent today (no production AddEpoch caller, and the transition test uses identical committees across epochs so no new lane appears), but it's a real defect in the transition path and should be fixed — create queues for newly-introduced next-epoch lanes on transition — before membership-changing epoch transitions are enabled. (Confirms Codex finding.)
| } | ||
|
|
||
| // Trio returns the current cached trio. | ||
| func (c EpochTrioCursor) Trio() types.EpochTrio { return c.trio } |
There was a problem hiding this comment.
[suggestion] EpochTrioCursor caches *Epoch pointers in trio. registry.TrioAt may return placeholder epochs built from the genesis committee when AddEpoch hasn't yet supplied the real one. Once such a placeholder is cached here, Trio() and the current-epoch fast path in ForVerify keep returning the genesis committee even after AddEpoch overwrites the registry entry — the cursor only refreshes on Advance into a different epoch. This surfaces in data.CurrentTrio() → Committee().EvmShard(...) routing and in current-epoch QC/vote verification. Consider refreshing the cached trio from the registry (or not caching placeholder-backed epochs) so a later AddEpoch is observed. Latent until AddEpoch is wired to production, but worth addressing with the transition work. (Confirms Codex finding.)
There was a problem hiding this comment.
Well-structured refactor replacing single-Epoch verification with an EpochTrio/EpochTrioCursor model across the autobahn layers, with good new test coverage. No shipping-blocking bugs: the multi-epoch concerns raised are currently latent because AddEpoch/non-genesis committees have no production callers yet, so all epochs resolve to the genesis-committee placeholder. A few consistency and locking notes are worth addressing.
Findings: 0 blocking | 5 non-blocking | 2 posted inline
Blockers
- None at the file/PR level.
Non-blocking
- Cursor second-opinion pass produced no output (
cursor-review.mdis empty);REVIEW_GUIDELINES.mdis also empty, so no repo-specific guidelines were applied. - Codex (High) — cached trios retain overridden placeholder epochs:
Registry.epochAtinserts placeholders using the genesis committee, and cursors cache the resultingEpochTrio. OnceAddEpochoverwrites a registry entry with the real committee, existing cursors keep pointing at the stale placeholder (Nextin particular) until theyAdvance. This is currently latent (no productionAddEpochcaller and all committees are the genesis placeholder), but it will produce wrong committees for next-epoch vote weighting / QC verification once real epoch transitions are wired up. Consider a way for cursors/TrioAtto pick up committee updates, or document/enforce the ordering constraint. Registry.TrioAtmutates the map (inserts placeholders) as a side effect of a read-style lookup; over many distinct road indices this grows the map with placeholder epochs that are never GC'd. Minor, but worth a note since it also underlies the staleness issue above.- 2 suggestion(s)/nit(s) flagged inline on specific lines.
| inner.qcs[inner.nextQC] = qc | ||
| inner.nextQC += 1 | ||
| } | ||
| inner.cursor.Advance(qc.QC().Proposal().Index()) |
There was a problem hiding this comment.
[suggestion] Codex (Medium): data advances the cursor with qc.QC().Proposal().Index(), whereas consensus (pushCommitQC) and avail (PushCommitQC) advance with Index()+1. After finalizing the last QC of an epoch, insertQC (line 226) and this path leave the cursor on the old epoch until the first QC of the next epoch arrives, so CurrentTrio() — consumed by the p2p EVM shard router (giga_router_common.go/giga_router_validator.go) — reports the previous committee across the boundary window. Verification itself still works via the ForVerify Next-range fallback, and it's harmless today (all committees are the genesis placeholder), but the divergence from the other two layers looks unintentional; either align on Index()+1 or add a comment explaining why data intentionally lags.
| } | ||
| committee = nextC | ||
| } | ||
| if err := vote.VerifySig(committee); err != nil { |
There was a problem hiding this comment.
[suggestion] PushVote now runs both structural Verify and the cryptographic VerifySig inside the s.inner.Lock() scope, so signature verification serializes on the state mutex. Note PushBlock (just above) deliberately reads the committees under the lock, releases it, and verifies outside. Consider matching that pattern here to keep expensive crypto off the hot lock.
Superseded: latest AI review found no blocking issues.
c36ec91 to
d03b4f5
Compare
wen-coding
changed the title
wen-coding
changed the title
There was a problem hiding this comment.
This PR introduces EpochTrio to move autobahn from a single-epoch to a three-epoch (Prev/Current/Next) view, replacing LatestEpoch()/EpochByIndex with road-index-based window lookups and adding epoch seeding/advancement. The core mechanics look sound and are well tested, but two edge cases flagged by Codex are worth addressing: reweightForNextEpoch under-initializes lane queues, and PushQC now rejects out-of-window QCs; both are latent (masked while all epochs share the genesis committee) but will matter once real committee rotation lands.
Findings: 0 blocking | 6 non-blocking | 2 posted inline
Blockers
- None at the file/PR level.
Non-blocking
- Cursor produced no output:
cursor-review.mdis empty.REVIEW_GUIDELINES.mdis also empty, so no repo-specific standards were applied. (Codex did produce output — both its findings are incorporated below.) - The PR description references APIs/tests that are not present in the diff —
advanceEpoch(),data.State.CurrentTrio(),TestInsertQCCrossEpochFallback,TestPushCommitQCCrossEpochFallback,TestAdvanceEpochTrio, and a "cross-epoch verification fallback" indata.PushQC. The actual code exposesEpochTrio()and advances viaTrioAt(idx+1)with no fallback. The description appears stale relative to the committed changes; update it so reviewers/maintainers aren't misled about what shipped. - Nit: several error messages wrap
EpochForRoadfailures with the labelEpochAt(%d)(e.g. avail/state.go PushCommitQC/PushAppVote/PushAppQC). Rename for accurate diagnostics. - Nit:
TestPushAppQCPreviousEpoch(avail/state_test.go) buildsepochNbut discards it (_ = epochN) and never actually advances the state into epoch N, so it does not exercise the "late AppQC after an epoch boundary crossed" scenario its comment describes — it only pushes an epoch N-1 QC/AppQC into a genesis-seeded state. - 2 suggestion(s)/nit(s) flagged inline on specific lines.
There was a problem hiding this comment.
This PR wires multi-epoch handling via a new EpochTrio abstraction and a seeding-aware Registry; the core logic and test coverage are solid. No present-tense blockers (all epochs still use the genesis committee placeholder), but there are two latent correctness gaps that will bite once real committee rotation is enabled, plus minor notes.
Findings: 0 blocking | 6 non-blocking | 2 posted inline
Blockers
- None at the file/PR level.
Non-blocking
- Cursor second-opinion review (cursor-review.md) and the repo REVIEW_GUIDELINES.md were both empty — no Cursor output to merge and no repo-specific guidelines applied.
- Codex flag (confirmed, latent): fullnodes never seal the registry. SealSeeding() is only called from consensus.NewState (validator path); NewGigaFullnodeRouter builds only a data.State and no consensus/avail state, so on a fullnode
seedingstays true forever and EpochAt/TrioAt will auto-generate any missing epoch with the genesis committee, permanently bypassing AdvanceIfNeeded's advancement constraints. Harmless today (genesis committee everywhere) but silently defeats this PR's own safety gate. Consider sealing the registry on the fullnode path (or documenting why it must remain open). - After SealSeeding, Registry.EpochAt still acquires the write lock (state.Lock()) for pure reads because it may auto-generate during seeding; consensus hot paths (pushProposal midpoint gate, pushCommitQC) call it per-message. Minor serialization/perf concern worth a fast read path once seeding is sealed.
- Several wrapped errors are mislabeled: e.g. state.go PushCommitQC/PushAppVote/PushAppQC return
fmt.Errorf("EpochAt(%d): ...")while actually calling EpochForRoad. Cosmetic, but confusing in logs. - 2 suggestion(s)/nit(s) flagged inline on specific lines.
There was a problem hiding this comment.
This PR wires multi-epoch handling into autobahn via a new EpochTrio abstraction and an epoch Registry that seeds epochs on demand. The refactor is clean and well-tested at the unit level, but the epoch-advance path at data/avail has a correctness gap: crossing an epoch boundary requires epoch N+2 to be registered, which the seeding model (AdvanceIfNeeded seeds only N+1) can never satisfy in production, so nodes will error at the end of epoch 0.
Findings: 3 blocking | 5 non-blocking | 3 posted inline
Blockers
- Root cause of the boundary failure:
Registry.AdvanceIfNeeded(registry.go) seeds onlycurrentIdx+1from an epoch-N AppQC, and an AppQC is fundamentally downstream of its CommitQC (registry.go's own comment states "AppQC never runs ahead of consensus"). ButTrioAtrequires both Current and Next, so entering epoch N+1 needs epoch N+2 registered. N+2 is only seeded by an AppQC in epoch N+1, which cannot exist before the CommitQC that ends epoch N. The PR description's claim that "AppQC has already been processing roads from epoch N+1" by the midpoint of epoch N contradicts registry.go and the AppQC→CommitQC dependency. Fix options: have AdvanceIfNeeded seed two epochs ahead (N+1 and N+2), or relax the boundary path to tolerate a not-yet-seeded Next. This is also Codex's P0 and should be resolved (or the seeding invariant proven) before merge. Unit tests miss it because SealSeeding is only called in consensus.NewState / the fullnode router, so most tests run in the always-auto-generate seeding phase and never exercise a post-seal boundary crossing. - 2 blocking issue(s) flagged inline on specific lines.
Non-blocking
- Cursor's second-opinion pass (cursor-review.md) produced no output; only Codex's review was available to merge.
- Consider an integration/e2e test that seals seeding and then drives a full epoch-boundary crossing (road EpochLength-1 → EpochLength) through data/avail PushCommitQC/PushQC, which would catch the N+2 seeding gap that unit tests miss.
- Several EpochForRoad error messages in avail/state.go are mislabeled as
EpochAt(%d)/EpochForRoad(%d)inconsistently — harmless but worth aligning the wording with the method actually called. - consensus/inner.go pushCommitQC derives the next epoch from
qc.Proposal().EpochIndex()rather thani.epoch.EpochIndex(); since the removed explicit epoch-match check is gone, a QC with an inconsistent EpochIndex would be trusted for the lookup. Low risk given QC verification, but worth a defensive assert. - 1 suggestion(s)/nit(s) flagged inline on specific lines.
| } | ||
| idx := qc.QC().Proposal().Index() | ||
| if idx == s.epochTrio.Load().Current.RoadRange().Last { | ||
| nextTrio, err := s.cfg.Registry.TrioAt(idx + 1) |
There was a problem hiding this comment.
[blocker] Same boundary bug as avail/state.go: TrioAt(idx+1) at the end of epoch N requires epoch N+2 to be registered, but the seeding model only guarantees N+1 exists when entering N+1. Once seeding is sealed, PushQC will error at the first real epoch boundary. Fix alongside the avail path.
| // block of epoch N — at that point N+2 won't be seeded until execution | ||
| // finalizes end(N), and this gate becomes meaningful back-pressure. | ||
| if proposal.Proposal().Msg().Index() == i.epoch.RoadRange().MidPoint() { | ||
| if _, err := i.registry.EpochAt(types.RoadIndex(i.epoch.EpochIndex()+1) * epoch.EpochLength); err != nil { |
There was a problem hiding this comment.
[suggestion] The midpoint liveness gate checks epoch N+1, but the PR's stated safety condition (and the TODO here) is N+2. Because N+1 is pre-seeded at init, this gate is a no-op today and provides none of the intended back-pressure (Codex P1). This is acknowledged as intentional-for-now via the TODO, so not blocking — but note it does not mitigate the data/avail N+2 boundary failure above, since even an N+2 gate cannot make N+2 seedable under the current AppQC ordering.
There was a problem hiding this comment.
Solid, well-tested groundwork for multi-epoch handling via EpochTrio, but it introduces a liveness-breaking bug for full nodes: they seal registry seeding after only epochs 0/1 and never call AdvanceIfNeeded, so data.State.PushQC's boundary TrioAt(idx+1) fails at the first epoch boundary and full-node sync permanently stalls.
Findings: 3 blocking | 4 non-blocking | 2 posted inline
Blockers
- Full nodes cannot cross the first epoch boundary (confirmed; also raised by Codex).
AdvanceIfNeededis only invoked fromavail/state.go, but full nodes run onlydata.State(no avail/consensus).NewGigaFullnodeRoutercallsSealSeeding()afterdata.NewStatehas seeded only epochs 0 and 1, so no further epochs are ever created. When a QC for the last road of epoch 0 arrives,data.State.PushQCcallsRegistry().TrioAt(idx+1)which requires epoch 2 asNext; it is missing and PushQC returns an error, so the full node stops accepting QCs forever. Fix by either not sealing seeding on full nodes, or having the full-node data path advance/seed epochs from incoming (App)QCs. - Missing test coverage for the full-node epoch-boundary crossing. All tests use
GenRegistry/GenRegistryAt, which pre-seed 3 epochs, masking the bug above. Add a test that constructs a full-node-style data.State (seeded 0/1 + SealSeeding) and pushes a QC at the last road of epoch 0 to assert the boundary is handled. - 1 blocking issue(s) flagged inline on specific lines.
Non-blocking
- Cursor's second-opinion review file (cursor-review.md) was empty — that pass produced no output.
Registry.EpochAtnow always takes the write lock (r.state.Lock()) even for pure reads afterSealSeeding, whereas the oldEpochByIndexusedRLock.EpochAt/TrioAtare called from consensus (pushCommitQC, midpoint gate) and data/avail restore paths; consider aRLockfast-path for the common post-seal read case to avoid serializing epoch lookups.- Several wrapped errors read
EpochAt(%d)/EpochForRoadinconsistently with the function actually invoked (e.g. avail/state.go PushCommitQC/PushAppVote/PushAppQC wrapEpochForRoadfailures as"EpochAt(%d): %w"). Harmless but misleading in logs — align the message text with the call. - 1 suggestion(s)/nit(s) flagged inline on specific lines.
| // if the epoch has not been registered via AdvanceIfNeeded. | ||
| func (r *Registry) EpochAt(roadIndex types.RoadIndex) (*types.Epoch, error) { | ||
| epochIdx := types.EpochIndex(roadIndex / EpochLength) | ||
| for s := range r.state.Lock() { |
There was a problem hiding this comment.
[nit] EpochAt now takes the write lock unconditionally, including for pure reads after SealSeeding (the old EpochByIndex used RLock). Since this is hit on consensus/data hot-ish paths, consider an RLock fast-path that only escalates to Lock when a missing epoch must be auto-generated during seeding.
There was a problem hiding this comment.
A well-structured, heavily-tested refactor that wires the EpochTrio machinery for multi-epoch handling across the data/avail/consensus layers. No blocking correctness issues were found; the main caveat is that committee rotation is still a documented placeholder, so the feature is machinery-only for now.
Findings: 0 blocking | 7 non-blocking | 3 posted inline
Blockers
- None at the file/PR level.
Non-blocking
- Codex (P1): every seeded epoch reuses the genesis committee (
makeEpochcopiesgenesis.Committee()), so QC verification, vote weighting, and EVM shard routing remain effectively tied to genesis and no real validator-set change happens across epochs yet. This is intentional and explicitly documented via TODOs inregistry.go/giga_router_common.go(real rotation comes with snapshot/execution-layer committee derivation), and it is not a regression (the prior code also used the genesis epoch everywhere), so I do not treat it as a blocker — but reviewers/mergers should be aware this PR introduces the machinery only and the multi-epoch feature is not functionally complete. - Cross-epoch consistency: several
EpochByIndex(qc.Proposal().EpochIndex())lookups were replaced with road-index-based resolution, and the removed avail checkqc.Proposal().EpochIndex() != inner.epoch.EpochIndex()is gone. The QC's self-reportedEpochIndexfield is therefore no longer cross-checked against its road index. This is harmless while all committees are identical, but should be revisited when committees actually differ per epoch. - The Cursor second-opinion review file (
cursor-review.md) was empty — that pass produced no output. - Restore path in
avail/state.goNewStateonly handles a single epoch-boundary crossing (if inner.commitQCs.next > startTrio.Current.RoadRange().Last). This is fine given prune anchors are recent, but a restart whose loaded CommitQCs span more than one epoch past the anchor would not fully reweight; worth a comment or assertion documenting the single-boundary assumption. - 3 suggestion(s)/nit(s) flagged inline on specific lines.
Superseded: latest AI review found no blocking issues.
| return inner{persistedInner: persisted, registry: registry, epoch: viewEpoch}, nil | ||
| } | ||
|
|
||
| func (s *State) pushCommitQC(qc *types.CommitQC) error { |
There was a problem hiding this comment.
Either pushCommitQC should support cross epoch jumps (not just E -> E+1), or we should have a separate epoch transition function (my preferred option). In particular we are not guaranteed that our source of CommitQCs (avail state) will not serve us LatestCommitQC from some future epoch.
There was a problem hiding this comment.
The tricky part is that if we push last CommitQC of an epoch, we need to verify it against the old epoch, but then we need to receive the new epoch spec to start voting. At no point we should need both E and E+1, but sequencing the transitions correctly might get tricky.
Also note that we cannot just drop the last commitQC when entering the new epoch, because we need to know the state of lanes and the next GlobalBlockNumber, which are not known at the time the epoch spec is defined (at the start of epoch E).
There was a problem hiding this comment.
quick idea: we might create yet another type which would combine epoch spec and the last commitQC to provide full context for consensus state to participate in a new epoch.
There was a problem hiding this comment.
hmm, actually I have just realized that we need to split the current Epoch type into exactly that: the part which is known 1 epoch earlier and and the part which is not known until the previous epoch is fully sequenced.
There was a problem hiding this comment.
I'm a bit hesitant putting commitQC and epoch into the same data structure, they are pretty different concepts. I'm okay putting them into one struct for context, but I guess that's inner?
Are we willing to have avail/ state/ data/ switch epoch together with the latest commitQC? I remember you said you preferred for each one to have their own copy, so I kept everyone advancing Epoch or EpochTrio on its own. An alternative is have registry provide a new EpochDuo to everyone when it sees commitQC, then avail/ data/ state/ will share AtomicRecv(EpochDuo), but we don't have to put registry in state.go any more.
Thoughts?
Align with GlobalRange/LaneRange; document epoch-0 AppQC wait and Last-CommitQC trio advance. Co-authored-by: Cursor <cursoragent@cursor.com>
There was a problem hiding this comment.
No bugs found, but this PR makes substantive semantic and concurrency changes to consensus-critical epoch/CommitQC logic in autobahn, so I'm deferring for human review rather than approving.
Extended reasoning...
Overview
This PR (part of an ongoing autobahn refactor series) touches 35 files across sei-tendermint/autobahn/types and sei-tendermint/internal/autobahn/{avail,consensus,data,epoch,producer} plus the giga p2p router. Core changes: (1) RoadRange is redefined from an inclusive [First, Last] range to a half-open [First, Next) range — a semantic change to a fundamental consensus type used throughout epoch/committee/road indexing; (2) a new EpochTrio type formalizes a sliding Prev/Current/Next epoch window with dedicated resolution helpers (EpochForRoad, EpochForCommitRoad, VerifyInWindow); (3) Registry gains TrioAt/WaitForTrio/AdvanceIfNeeded/SetupInitialTrio, introducing a new locking/notification pattern (highestEpoch atomic kept outside the RWMutex-protected state to preserve the RLock fast path for EpochAt); (4) persistedInner.validate now takes two distinct epochs (commitEp, viewEp) to correctly verify state that straddles an epoch boundary.
Security risks
No injection/auth-style risks — this is internal consensus state-machine logic, not a network-facing trust boundary in the traditional sense. The real risk category here is consensus-safety: an off-by-one in the new half-open range semantics, a race in the highestEpoch/WaitForTrio notification path, or an incorrect epoch chosen in EpochForCommitRoad/validate could cause a validator to reject valid QCs, accept invalid ones, or panic (note the explicit panic() in EpochForCommitRoad for an assumed-impossible case) — any of which is a liveness or safety incident in production, not a typical software bug.
Level of scrutiny
This warrants a high level of scrutiny. It is squarely "touches critical code paths" and contains "large scale design decisions" (the trio window model, the boundary-epoch validation split, the new locking/wait pattern) that a human familiar with the autobahn consensus design should weigh in on, even though the bug-hunting pass found nothing. The range semantics change alone (inclusive → exclusive) is the kind of pervasive, easy-to-get-subtly-wrong change that benefits from a human who knows all call sites and invariants.
Other factors
The diff includes substantial test updates alongside the production code (committee_test.go, epoch_trio_test.go, registry_test.go, state_test.go, etc.), which is a positive signal, and this appears to be one PR in a deliberate, incremental refactor sequence (per recent commit history: rename advanceEpoch, add WaitForTrio, gate CommitQC on AppQC, drop stale AppQCs, half-open RoadRange). That incremental style is good practice, but each step still changes core consensus behavior and should get a human set of eyes before merging.
Make it a method on loadedAvailState per review feedback. Co-authored-by: Cursor <cursoragent@cursor.com>
Combine blocks, votes, and persistence cursors into inner.lanes per review feedback. Co-authored-by: Cursor <cursoragent@cursor.com>
Future AppVotes block until their CommitQC advances the window; stale votes still drop. Co-authored-by: Cursor <cursoragent@cursor.com>
Backpressure peer streams while catching up; drop only when the road is stale. Co-authored-by: Cursor <cursoragent@cursor.com>
Drop per-epoch byHash maps; reweight from byKey on epoch advance. Co-authored-by: Cursor <cursoragent@cursor.com>
newInner errors if Current.EpochIndex > 0 and no prune anchor is present. Co-authored-by: Cursor <cursoragent@cursor.com>
…aking it Align with Current-only LaneQC weighting: drop the ep argument and let the caller verify the returned epoch matches the view it intends to propose in. Co-authored-by: Cursor <cursoragent@cursor.com>
Iterate FullCommitQC headers instead of the committee so block collection stays correct as committees change. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
There was a problem hiding this comment.
No bugs found, but this PR makes non-trivial changes to the autobahn BFT consensus epoch-boundary logic (splitting commitEp/viewEp in persistedInner.validate, epoch-aware vote filtering in PushPrepareVote/PushCommitVote/PushTimeoutVote, the new EpochTrio sliding window, and WaitForLaneQCs returning the current epoch) — worth a human look given how safety-critical correct epoch handling is for consensus liveness and equivocation prevention.
Extended reasoning...
Overview
This PR (38 files) refactors epoch handling across the autobahn consensus/avail/p2p-giga stack in sei-tendermint. Key changes: persistedInner.validate now takes separate commitEp/viewEp epochs to correctly verify CommitQC vs. current-view artifacts across an epoch boundary; a new EpochTrio type models a sliding Prev/Current/Next epoch window with road-index and committee-membership resolution helpers; State.PushPrepareVote/PushCommitVote/PushTimeoutVote now explicitly drop (rather than mis-verify) votes whose epoch doesn't match the current epoch; WaitForLaneQCs no longer takes an epoch argument and instead returns the epoch it resolved against, with callers checking for epoch drift before proceeding; and the p2p giga router files add epoch-scoped routing logic. Companion test files were updated/added alongside each change.
Security risks
No injection/auth/crypto-vulnerability patterns were spotted directly, but this is exactly the kind of code where a subtle epoch/view-boundary bug could cause safety violations (e.g., verifying a vote/QC against the wrong epoch's committee, silently dropping legitimate votes, or accepting stale artifacts across an epoch transition) — the consequences of a logic error here are consensus safety or liveness bugs rather than typical web-app vulnerabilities.
Level of scrutiny
This warrants a high level of scrutiny. It's dense, non-mechanical logic changes to the BFT consensus core (vote verification, QC verification, epoch window resolution) — precisely the kind of critical path and large-scale design change called out as needing human review, regardless of whether the automated bug hunt found issues.
Other factors
The PR includes substantial new/updated unit tests (committee_test.go, epoch_trio_test.go, state_test.go, inner_test.go, etc.), which is a positive signal, and the commit history shows this was built up incrementally with descriptive commit messages. However, test coverage doesn't substitute for a domain expert's review of the epoch-boundary invariants themselves. The PR timeline/metadata endpoint was unavailable during this review, so I could not confirm whether a maintainer has already reviewed or commented on this revision.
Pure rename to preserve git history; content changes follow separately. Co-authored-by: Cursor <cursoragent@cursor.com>
Drop the resident Next epoch: new-committee lane traffic is admitted only
after CommitQC advances Current into the next epoch. CommitQC/block/vote
verification is now Current-only; the registry seeds {N-1, N} and fetches the
next epoch at the boundary via WaitForDuo (renamed from TrioAt/WaitForTrio).
Co-authored-by: Cursor <cursoragent@cursor.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
There are 3 total unresolved issues (including 2 from previous reviews).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit afcd76a. Configure here.
| headers := make([]*types.BlockHeader, lr.Next()-lr.First()) | ||
| for inner, ctrl := range s.inner.Lock() { | ||
| q := inner.votes[lr.Lane()] | ||
| q := inner.lanes[lr.Lane()].votes |
There was a problem hiding this comment.
Missing lane panics header rebuild
High Severity
headers indexes inner.lanes without ensuring the lane exists. After restart or epoch advance, newInner only seeds lanes for the operating duo’s current committee, while fullCommitQC can still walk lanes from a previous epoch via EpochForRoad. A nil *laneState triggers a panic when rebuilding older commit QCs.
Additional Locations (2)
Reviewed by Cursor Bugbot for commit afcd76a. Configure here.
There was a problem hiding this comment.
A large, well-tested refactor introducing multi-epoch handling (EpochDuo{Prev,Current}) across the autobahn consensus/data/avail layers, replacing single-epoch lookups with a sliding window. No present-day correctness bugs were confirmed — all epochs currently reuse the genesis committee, so the epoch-boundary races Codex flags are latent until dynamic committees land — but several forward-looking robustness concerns and a title/description-vs-implementation mismatch are worth addressing.
Findings: 0 blocking | 10 non-blocking | 3 posted inline
Blockers
- None at the file/PR level.
Non-blocking
- Title/description vs implementation mismatch: the PR title says "EpochTrio" and the description states "Lane votes/blocks verify against Current+Next so producers in the upcoming epoch can participate before AppQC", but the implemented struct is
EpochDuo{Prev, Current}and PushBlock/PushVote/PushCommitQC verify against Current only (per the deliberateepoch_duo.gocomment: "Next is intentionally not held"). Please reconcile the description/title with the shipped Duo design so future readers aren't misled. - Codex (High) — upcoming-epoch lane traffic rejected: PushBlock/PushVote (state.go:509, 561) verify only against Current, so next-epoch producers cannot pre-propagate before the boundary. This is intentional in the current design and harmless today because every epoch reuses the genesis committee (registry.go:104
genesis.Committee()), so no valid producer is actually excluded. It becomes a real concern (empty lanes / stalled rollover) only once dynamic committees are wired — worth a tracking TODO at the verification sites. - Codex (High) — WaitForDuo can wake early on a registry gap:
highest >= centerIdx(registry.go WaitForDuo) does not guarantee centerIdx itself is registered, so a disjoint seeded window above a gap would make DuoAt error instead of waiting. Under the current invariant (WaitForDuo targets are always exactly Current+1, seeded contiguously ahead of the tip) this cannot happen, but the coupling is fragile; consider waiting on "centerIdx registered" directly rather than on the monotonic high-water mark. - Codex (Medium) — PushVote verifies against Current loaded off-lock (state.go:561) but re-loads the duo and weights against Current under the lock (state.go:586-587); a concurrent boundary advance could weight a vote under a committee it was not membership/header-verified against. Latent (identical committees today); becomes relevant with dynamic committees. Same shape applies to blockVotes.reweight discarding an already-formed LaneQC on advance.
- consensus pushCommitQC (inner.go:160) hard-errors (fail-fast, can halt the node) if epoch N+1 is not yet registered at a boundary, relying on the invariant that sequential execution/AdvanceIfNeeded always seeds N+1 first. Please double-check this holds across restart timing (SetupInitialDuo seeds only {view-1, view}; N+1 depends on executeBlock having run) before merging, since a violated assumption crashes consensus rather than backpressuring like avail/data.
- Scaffolding limitation to keep visible: makeEpoch stamps every placeholder epoch with genesis.FirstBlock()/FirstTimestamp()/Committee(), so placeholder epochs do not carry their true first block. The tests acknowledge this; ensure the "verified snapshot / state-sync" TODOs are tracked so these placeholders are replaced before dynamic committees or per-epoch first-blocks are relied upon.
- cursor-review.md is empty (no Cursor output for this PR) and REVIEW_GUIDELINES.md is empty (no repo-specific standards were available to apply).
- 3 suggestion(s)/nit(s) flagged inline on specific lines.
| } | ||
| if _, ok := q.q[h.BlockNumber()].pushVote(inner.epoch, vote); ok { | ||
| duo := s.epochDuo.Load() | ||
| if q.q[h.BlockNumber()].pushVote(duo.Current, vote).IsPresent() { |
There was a problem hiding this comment.
[suggestion] Verify/weight epoch can diverge: the vote is signature/header-verified against Current loaded off-lock at the top of PushVote (state.go:561), but here the duo is re-loaded under the lock and weighting uses this (possibly newer) Current after a concurrent boundary advance. Harmless today since all epochs share the genesis committee, but once committees differ a vote verified under committee N could be counted under committee N+1. Consider carrying the epoch used for verification into the locked section (or re-verifying if the window advanced).
| iSend.Store(inner{persistedInner: persistedInner{CommitQC: utils.Some(qc)}, epoch: i.epoch}) | ||
| // Invariant: N+1 is registered before this CommitQC (setup / AdvanceIfNeeded). | ||
| // Hard-error if missing — do not WaitForDuo like avail/data do for N+1. | ||
| nextEp, err := i.registry.EpochAt(qc.Proposal().Index() + 1) |
There was a problem hiding this comment.
[suggestion] This hard-errors (and thus can terminate consensus) when epoch N+1 isn't registered at the boundary, unlike avail/data which WaitForDuo. The invariant that execution always seeds N+1 before consensus commits the last road of N is plausible in steady state, but please confirm it also holds immediately after a restart, where SetupInitialDuo only seeds {view-1, view} and N+1 relies on executeBlock having already run AdvanceIfNeeded. A failed assumption here halts the node.
| } | ||
| return p.VerifySig(c) | ||
| }); err != nil { | ||
| c := s.epochDuo.Load().Current.Committee() |
There was a problem hiding this comment.
[nit] PushBlock (and PushVote at 561) verify only against Current, so a validator in the upcoming epoch can't pre-propagate before the boundary advances the duo. This matches the deliberate EpochDuo design ("Next is intentionally not held") and is safe today because all epochs reuse the genesis committee, but note it contradicts the PR description's "verify against Current+Next". Worth a TODO tying this to the dynamic-committee work so lane rollover doesn't stall once committees actually change.
There was a problem hiding this comment.
I'm not approving this PR. A high-severity concern raised in earlier review rounds on this PR is still present, unchanged, in the current code: consensus.pushCommitQC (sei-tendermint/internal/autobahn/consensus/inner.go:150) verifies an incoming CommitQC against i.epoch — the epoch cached from the previous call — before resolving the incoming QC's own epoch from its road index. This run's automated bug hunt listed the same concern as a ruled-out candidate, but my own reading of the current code does not support that conclusion, so I think it still needs a maintainer's eyes rather than being closed out.
Extended reasoning...
Overview
This PR reworks epoch handling across the data, avail, and consensus layers of sei-tendermint's autobahn consensus (introducing EpochDuo, replacing the earlier EpochTrio design per the latest commits), plus registry seeding/waiting logic and EVM shard routing. It is a large, consensus-critical change spanning ~38 files.
The specific concern
consensus.pushCommitQC (inner.go:145-169) does:
i := s.innerRecv.Load()
if qc.Proposal().Index() < i.View().Index { return nil }
if err := qc.Verify(i.epoch); err != nil { ... } // verifies against the OLD cached epoch
for iSend := range s.inner.Lock() {
...
nextEp, err := i.registry.EpochAt(qc.Proposal().Index() + 1) // only decides epoch to STORE going forward
...
}The index check only rejects a QC that is behind the current view — it does not require the incoming QC's index to be exactly i.View().Index. pushCommitQC is driven by s.avail.LastCommitQC().Iter (consensus/state.go), which reads the latest value of a coalescing AtomicRecv populated by avail.State.markCommitQCsPersisted — itself called once per QC inside the persist-batch goroutine (runPersist), independent of pushCommitQC's own consumption rate. If several Store() calls land while pushCommitQC's prior call is still running (e.g. while catching up on a backlog), the next Iter wake can observe a QC whose road index has already crossed an epoch boundary that i.epoch (single-step-advanced via EpochAt(prevIdx+1)) was never advanced through. qc.Verify(i.epoch) then fails on the epoch-index mismatch, and since this runs inside scope.SpawnNamed, the error tears down the whole consensus scope.
I raised this exact issue three times in earlier review rounds on this PR (most recently confirming it was still present as of commit 9cdb3fe+). The maintainer's reply addressed a related-but-different property ("execution is sequential, can't reach N+2 without going through N+1"), which concerns registry seeding, not the coalescing-watch skip in the verify path itself. The code here is unchanged in this respect even after the subsequent EpochTrio→EpochDuo refactor. This run's bug-hunting pass flagged the same code as a candidate and its verifier concluded it wasn't a real bug; based on my own reading of the current Iter/AtomicSend semantics and the persist-batch call pattern, I don't think that refutation holds, so a maintainer should take another look specifically at this path rather than have it closed out.
Security / liveness risk
This is a liveness risk on a validator's consensus scope (not a safety/fork risk, since it only causes a hard error/teardown, not incorrect state) — but a full scope teardown on a validator during catch-up is exactly the kind of failure mode that's easy to hit under load or after an outage, which is when it matters most.
Level of scrutiny
Given this touches the core CommitQC verification path in consensus and has a plausible, previously-argued trigger condition that survived a large subsequent refactor unaddressed, I think this warrants explicit maintainer confirmation (either that the coalescing-skip scenario is impossible for some invariant I'm missing, or a fix mirroring how avail.PushCommitQC/data.PushQC resolve the epoch from the QC's own road index before verifying).
…ast road
Keep epoch transitions explicit via State, and restore SetupInitialDuo/{0,1}
plus AdvanceIfNeeded(N+2) only when roadIndex+1 == epoch Next.
Co-authored-by: Cursor <cursoragent@cursor.com>
There was a problem hiding this comment.
Large, well-tested refactor introducing multi-epoch windowing (EpochDuo) across the autobahn data/avail/consensus layers. One peer-triggerable liveness bug: PushCommitQC gates an unbounded wait on the CommitQC's unverified embedded EpochIndex before signature verification. A second, currently-latent non-atomic verify/weight window in PushVote. Everything else is sound.
Findings: 1 blocking | 4 non-blocking | 2 posted inline
Blockers
- None at the file/PR level.
- 1 blocking issue(s) flagged inline on specific lines.
Non-blocking
- Cursor second-opinion review (cursor-review.md) and REVIEW_GUIDELINES.md were both empty — no Cursor findings and no repo-specific guidelines to apply this pass.
- FYI (not fixable here): every registered epoch is a placeholder built from the genesis committee (makeEpoch uses genesis.Committee()), so the new cross-epoch verify/reweight paths (advanceEpoch→reweight, Prev/Current committee selection) are never exercised against a changed committee. The many TODOs acknowledge this, but it means the multi-committee correctness of PushVote/reweight/PushCommitQC is effectively untested until real committee derivation lands.
- data.State.PushQC returns EpochForRoad's error for any out-of-window road, while avail.PushCommitQC/PushAppQC/PushAppVote silently drop out-of-window roads (return nil). The asymmetry is probably fine given data.PushQC is fed in-window QCs by the avail Run loop, but a future road (node behind) would error/tear down rather than back-pressure — worth a comment confirming that's intended.
- 1 suggestion(s)/nit(s) flagged inline on specific lines.
| return fmt.Errorf("unknown epoch_index %d", qc.Proposal().EpochIndex()) | ||
| // Epochs are numbered from 0 (unlike global block numbers). Epoch 0 has no | ||
| // prior AppQC to wait for; N+1 requires AppQC of epoch N. | ||
| if qcEpoch := qc.Proposal().EpochIndex(); qcEpoch > 0 { |
There was a problem hiding this comment.
[blocker] waitForAppQCEpoch(ctx, qcEpoch-1) is gated on qc.Proposal().EpochIndex(), which is unverified at this point — qc.Verify(ep) doesn't run until ~9 lines below. A malicious/buggy peer can send a CommitQC with a legitimate next road index (idx == commitQCs.next, so waitForCommitQC(idx-1) passes) but an arbitrarily large EpochIndex. waitForAppQCEpoch then blocks until latestAppQC.EpochIndex() >= huge, which never happens, stalling this CommitQC stream handler indefinitely (until ctx cancel) before the QC is ever signature-checked.
Everywhere else the epoch is derived deterministically from the road (road / EpochLength), so trusting the embedded field here is both unnecessary and unsafe. Either derive the wait epoch from idx (e.g. types.EpochIndex(idx / epoch.EpochLength)), or move this wait after qc.Verify(ep) / the ep.RoadRange().Has(idx) drop check so the epoch is validated first. For legitimate traffic the behavior is unchanged (View.Verify already binds EpochIndex to the road range).
| q.pushBack(newBlockVotes()) | ||
| } | ||
| if _, ok := q.q[h.BlockNumber()].pushVote(inner.epoch, vote); ok { | ||
| duo := s.epochDuo.Load() |
There was a problem hiding this comment.
[suggestion] The committee used to verify the vote is snapshotted off-lock at line 561 (c := s.epochDuo.Load().Current.Committee()), but the vote is weighted under a second, independently re-loaded snapshot here (duo := s.epochDuo.Load() → pushVote(duo.Current, ...)). If an epoch boundary advances the duo between the two loads, a vote verified against the old committee is inserted/weighted under the new committee. Harmless today only because all placeholder epochs share the genesis committee, but it becomes a correctness gap once committees actually differ. Capture one duo/committee snapshot at the top and reuse it for both verification and pushVote (or re-check the epoch under the lock).
There was a problem hiding this comment.
Beyond the inline finding, this run also checked two other epoch-boundary races and ruled them out: WaitForDuo/AdvanceIfNeeded cannot spuriously skip epoch N+1 (SetupInitialDuo always seeds three consecutive epochs together, and sequential execution guarantees N+1 is registered before N+2 is seeded), and data.State.PushBlock's hard error on an EpochForRoad miss (vs. avail's ErrPruned wrapping) is bounded in practice by the blocksCacheSize backpressure that keeps block delivery within the live epoch window.
Extended reasoning...
This run re-examined two candidate races flagged by finder agents around the new epoch-seeding/registry code (sei-tendermint/internal/autobahn/epoch/registry.go, sei-tendermint/internal/autobahn/data/state.go) and confirmed both are not reachable given the invariants this PR establishes (epochs are always seeded three-at-a-time via SetupInitialDuo, and AdvanceIfNeeded only ever seeds N+2 once N+1 already exists from prior sequential execution). This is recorded for visibility only; it does not cover the new inline finding in this review (the unrelaxed epoch-equality check in FullProposal.Verify), which still needs a fix.
| return nil | ||
| } | ||
| // Wait for laneQCs. | ||
| laneQCsMap, err := s.avail.WaitForLaneQCs(ctx, vs.Epoch, vs.CommitQC) | ||
| laneQCsMap, ep, err := s.avail.WaitForLaneQCs(ctx, vs.CommitQC) | ||
| if err != nil { | ||
| return fmt.Errorf("s.avail.WaitForLaneQCs(): %w", err) | ||
| } | ||
| // The avail window may have advanced past the epoch we intend to | ||
| // propose in; skip and let the next view catch up. | ||
| if ep.EpochIndex() != vs.Epoch.EpochIndex() { | ||
| return nil | ||
| } | ||
| // Construct a full proposal. | ||
| fullProposal, err := types.NewProposal( | ||
| s.cfg.Key, |
There was a problem hiding this comment.
🔴 runPropose builds every proposal using s.avail.LastAppQC(), which can legitimately lag consensus's own epoch by exactly one right after a boundary (this PR's own invariant: AppQC and CommitQC are never more than one epoch apart). FullProposal.Verify (sei-tendermint/autobahn/types/proposal.go:493-496) still enforces strict equality between the embedded app's epoch and the proposal's epoch, right after the comment // TODO: relax to allow current_epoch-1 once epoch transitions are wired up — which is exactly what this PR does, but the check was never relaxed. This causes a leader's proposal to be rejected by every verifier at (or shortly after) each epoch boundary whenever the leader has fresh AppQC progress to report, forcing view timeouts and potentially livelock.
Extended reasoning...
The bug: FullProposal.Verify in sei-tendermint/autobahn/types/proposal.go (lines 491-496) verifies a freshly-embedded appQC's epoch against the proposal's own epoch with strict equality:
app, _ := m.proposal.Msg().App().Get()
// TODO: relax to allow current_epoch-1 once epoch transitions are wired up.
if got, want := app.EpochIndex(), m.proposal.Msg().EpochIndex(); got != want {
return fmt.Errorf("app epoch_index %d != proposal epoch_index %d", got, want)
}This code is untouched by the PR diff — confirmed by reading the current file, this exact TODO/check is still present verbatim. The TODO literally describes the exact situation this PR creates (multi-epoch handling, "EpochTrio"/EpochDuo), but the relaxation was never implemented.
Code path that triggers it: runPropose (sei-tendermint/internal/autobahn/consensus/state.go) calls s.avail.LastAppQC() and passes it straight into types.NewProposal, which via buildProposal embeds it as a fresh app whenever it represents newer progress than what the previous CommitQC already carried (NextOpt(app) > NextOpt(old)). The embedded AppProposal's epoch index is set at push time in avail.PushAppQC/data.PushAppHash to the epoch of the road whose CommitQC finalized that block — i.e. it structurally can never be ahead of, and is commonly exactly one epoch behind, consensus's own currently-proposing epoch right after a boundary. Consensus rotates its cached epoch synchronously the instant it observes a boundary-crossing CommitQC (consensus/inner.go pushCommitQC), while AppQC formation is asynchronous (it requires avail/data to observe the CommitQC, execute the block, and collect AppVotes) and therefore lags. The PR's own added gate in avail.PushCommitQC (waitForAppQCEpoch(ctx, qcEpoch-1)) explicitly permits AppQC to be exactly one epoch behind the admitted CommitQC — this is the PR's own stated invariant ("AppQC and CommitQC should never be more than one epoch apart"), and it is precisely the scenario the unrelaxed check in proposal.go rejects.
Why nothing else prevents it: runPropose's added epoch check (if ep.EpochIndex() != vs.Epoch.EpochIndex() { return nil }) only compares the epoch of the laneQCs (avail's Current epoch) against the view's epoch — it does not constrain the appQC's epoch at all. buildProposal only clears a candidate appQC when it is stale (not newer than the CommitQC-carried one) or from-the-future — never when it is merely from a past (but valid, one-behind) epoch. So a leader with genuinely fresh AppQC progress will embed it, sign the proposal, and have every peer reject it via FullProposal.Verify.
Impact: since PrepareVotes are gated on successful FullProposal.Verify, a rejected proposal never accumulates a PrepareQC, forcing a view timeout. This can repeat across consecutive views at a boundary (worst case, if every candidate leader has fresh-but-lagging AppQC progress to report) before a proposal happens to carry forward a non-fresh app instead, giving degraded liveness — forced timeouts — at essentially every epoch boundary (every EpochLength = 108,000 roads), directly undermining the multi-epoch liveness invariant this PR sets out to establish.
Step-by-step proof:
- Road
108,000 * (N+1) - 1is the last road of epoch N; its CommitQC is admitted into avail oncewaitForAppQCEpoch(N-1)is satisfied (AppQC only needs to be from epoch N-1 or later, per the PR's own gate). avail.PushCommitQCsees this CommitQC is the last road ofCurrent(epoch N), resolves and stores the nextEpochDuo(Currentbecomes epoch N+1).consensus.pushCommitQC(watchingavail.LastCommitQC()) observes the same boundary CommitQC and rotates its own cachedi.epochto N+1 synchronously —vs.Epochfor the next view is now N+1.- Suppose avail's
latestAppQCis still anchored to a road within epoch N (allowed — it only needed to reach epoch N-1 to unblock step 1, and per the invariant can lag the boundary CommitQC by up to one full epoch). - The leader for the first view of epoch N+1 calls
runPropose;s.avail.LastAppQC()returns this epoch-N AppQC. If it represents newer progress than what the justifying CommitQC already carried,buildProposalembeds it as fresh. - The leader signs and broadcasts this proposal (epoch N+1) carrying
app.EpochIndex() == N. - Every validator calls
FullProposal.Verify; it reaches theelsebranch (fresh appQC), computesgot = N,want = N+1, and returns"app epoch_index N != proposal epoch_index N+1". The proposal is rejected everywhere, no PrepareQC forms, and the view times out.
Fix: relax the check in proposal.go to accept app.EpochIndex()+1 == m.proposal.Msg().EpochIndex() in addition to equality (or <=), matching both the TODO comment and the "AppQC at most one epoch behind CommitQC" invariant the rest of this PR relies on.
…nner Make the sequenced-tip invariant explicit before Verify so we hard-error instead of re-resolving the epoch from the registry. Co-authored-by: Cursor <cursoragent@cursor.com>
There was a problem hiding this comment.
Large, well-tested refactor introducing EpochTrio/EpochDuo for multi-epoch handling across avail/data/consensus/registry. One blocking liveness issue: PushCommitQC waits on the CommitQC's unverified EpochIndex before range-checking or verifying the QC, letting a peer stall CommitQC ingestion with a forged epoch.
Findings: 1 blocking | 4 non-blocking | 1 posted inline
Blockers
- None at the file/PR level.
- 1 blocking issue(s) flagged inline on specific lines.
Non-blocking
PushAppQC(avail/state.go:432) now callswaitEpochForRoad(ctx, idx)on the unverifiedcommitQC.Proposal().Index()before any verification. Where the old code returned an error for an unknown epoch, a forged AppQC carrying a far-future road index now blocks the handler until the window advances (which it never will), so this shares the same wait-on-unverified-input DoS class as the PushCommitQC blocker. Consider bounding/verifying before waiting.- Second-opinion passes:
cursor-review.mdwas empty (Cursor produced no output) andREVIEW_GUIDELINES.mdwas empty, so no repo-specific standards were applied. Codex's single finding (the PushCommitQC epoch-wait) is confirmed and captured as the blocker below. - Placeholder epochs seeded by
SetupInitialDuo/AdvanceIfNeeded/makeEpochall reuse the genesis committee, timestamp, and firstBlock (acknowledged by TODOs). This is fine only while dynamic committees are unsupported; the manyTODO: verified snapshot / real N+2 committeecomments should be tracked so these placeholders are replaced before dynamic committee changes ship. makeEpochis documented as overwriting an existing entry, whileSetupInitialDuo/AdvanceIfNeededguard with existence checks. The test-onlymakeRegistryAtcallsmakeEpoch(s, startEpoch)unconditionally (harmlessly re-deriving genesis when startEpoch==0). Consider makingmakeEpochnon-clobbering, or renaming, to prevent a future caller from accidentally resetting a live epoch.
| // Epochs are numbered from 0 (unlike global block numbers). Epoch 0 has no | ||
| // prior AppQC to wait for; N+1 requires AppQC of epoch N. | ||
| if qcEpoch := qc.Proposal().EpochIndex(); qcEpoch > 0 { | ||
| if err := s.waitForAppQCEpoch(ctx, qcEpoch-1); err != nil { |
There was a problem hiding this comment.
[blocker] waitForAppQCEpoch(ctx, qcEpoch-1) is called on qc.Proposal().EpochIndex() before the QC is verified or its road is range-checked (verify/RoadRange().Has happen below at lines 338-345). A peer can send a CommitQC with idx == the next expected road (so waitForCommitQC(idx-1) returns immediately) but a forged, arbitrarily large EpochIndex; waitForAppQCEpoch then blocks forever waiting for an AppQC of an epoch that will never arrive, stalling this node's CommitQC ingestion (liveness DoS from a single byzantine peer).
Since admitted CommitQCs must lie in Current's range and are verified against Current, reorder so the range check + qc.Verify(ep) run first, then gate on the verified epoch (ep.EpochIndex()) rather than the untrusted qc.Proposal().EpochIndex(). This preserves the intended backpressure (avail tip at most one epoch ahead of the AppQC anchor) while removing the wait on attacker-controlled input.
There was a problem hiding this comment.
Beyond the inline finding, this run also examined two other candidates and ruled them out: avail.PushCommitQC's early waitForAppQCEpoch gate reads the CommitQC's own unverified EpochIndex() before cryptographic verification, but that value only controls how long the call waits — the epoch actually used for Verify is derived independently from the road index — so it can affect liveness for a malformed CommitQC on that peer stream but not admit anything unverified. avail.inner.advanceEpoch's vote reweight (laneVoteSet.add) short-circuits once a lane's QC is set, so reweighting on an epoch boundary cannot discard an already-quorum-reached LaneQC.
Extended reasoning...
This is an informational note, not a full review: the bug-hunting system's one new finding this run (data.State.PushQC resolving the epoch window before the needQC staleness check) is already posted as an inline comment. Beyond that, two additional candidates raised by finder agents were checked against the current code and ruled out for the reasons summarized above. This does not constitute a guarantee that the surrounding epoch-boundary logic is otherwise correct, and it does not change the standing recommendation that this PR needs human review, given the new confirmed bug and the multiple consensus-critical epoch-transition issues already discussed in this thread.
| func (s *State) PushQC(ctx context.Context, qc *types.FullCommitQC, blocks []*types.Block) error { | ||
| // Wait until QC is needed. | ||
| ep, ok := s.cfg.Registry.EpochByIndex(qc.QC().Proposal().EpochIndex()) | ||
| if !ok { | ||
| return fmt.Errorf("unknown epoch_index %d", qc.QC().Proposal().EpochIndex()) | ||
| // Out-of-window ⇒ below prune tip; reject. | ||
| ep, err := s.epochDuo.Load().EpochForRoad(qc.QC().Proposal().Index()) | ||
| if err != nil { | ||
| return err | ||
| } | ||
| gr := qc.QC().GlobalRange() | ||
| needQC, err := func() (bool, error) { |
There was a problem hiding this comment.
🔴 PushQC resolves the epoch window (EpochForRoad over the 2-epoch {Prev, Current} range) before checking whether the QC is actually needed, so a stale/duplicate QC redelivery that is 1+ epochs behind Current now hard-errors instead of silently no-oping via the needQC check just below. Reachable when one peer's clientStreamFullCommitQCs connection lags behind sibling connections feeding the same shared data.State during catch-up; the error tears down that peer's entire RunClient scope (including consensus vote exchange), not just the QC stream, until the dial loop reconnects. Fix: check staleness (or needQC) before requiring the epoch window, mirroring the sibling PushBlock's if n < inner.first { return nil } guard.
Extended reasoning...
data.State.PushQC (state.go) begins with:
ep, err := s.epochDuo.Load().EpochForRoad(qc.QC().Proposal().Index())
if err != nil {
return err
}
gr := qc.QC().GlobalRange()
needQC, err := func() (bool, error) { ... }()EpochForRoad only succeeds if the QC's road index falls inside the narrow {Prev, Current} sliding window (EpochDuo.all() in epoch_duo.go), a strict two-epoch span. Since epochDuo only ever advances, a QC whose road has fallen more than one epoch behind Current makes EpochForRoad fail — and PushQC returns that as a hard error before it ever reaches the needQC computation a few lines down. But needQC is exactly the mechanism that would otherwise make a stale/duplicate QC delivery a harmless no-op (inner.nextQC == gr.First would be false once nextQC has advanced past it, and the giga path passes nil blocks, so there'd be nothing else to do).
This is a genuine regression introduced by this PR's multi-epoch machinery. Before this PR, epoch lookup went through Registry.EpochByIndex against a registry that retains all epochs indefinitely — a stale QC's epoch would still be found, needQC would come back false, and the call would quietly no-op. The new EpochDuo window intentionally narrows acceptance to two epochs (to bound memory/lookup cost), but PushQC never re-added the pre-check that staleness alone should short-circuit before the window is required. The sibling function PushBlock, in the very same file, does exactly this correctly: if n < inner.first { return nil } runs before any epoch resolution, silently dropping already-pruned input. PushQC has no equivalent guard.
The path is reachable in production, not just in theory: a validator's gigaValidatorRouter.Run() dials every committee member in parallel, and each connection's giga.Service.RunClient spawns its own clientStreamFullCommitQCs goroutine that independently calls s.data.PushQC(ctx, qc, nil). Each such stream is seeded from NextBlock() captured once at connect time and then served sequentially by the peer (for i := req.NextBlock; ; i++ on the server side). If one peer connection lags — for example during a node's initial catch-up on a long-running chain, or after transient network degradation on just that link — while sibling connections race ahead and push the shared data.State's epochDuo forward, the lagging stream keeps redelivering QCs that are now more than one epoch stale relative to the shared window. Crucially, clientStreamFullCommitQCs runs inside the same scope.Run as clientConsensus, clientPing, clientStreamLaneVotes, etc. for that peer connection, so a hard error here tears down the entire peer session — including consensus vote exchange with that peer — not just the QC stream, until the outer per-peer dial loop reconnects with a fresh NextBlock() snapshot that happens to sidestep the issue (until it recurs on the next lagging link).
Step-by-step: (1) two peer connections A and B both feed the same shared data.State via independent clientStreamFullCommitQCs goroutines. (2) B's link is fast and keeps pushing new FullCommitQCs, advancing inner.nextQC and, at each boundary, epochDuo (via the WaitForDuo/lock logic in PushQC) until Current is far ahead. (3) A's link is slow/stalled and is still redelivering an old QC whose road index is now more than EpochLength (108,000 roads, ~12h at ~400ms/road) behind epochDuo.Current. (4) A's PushQC(qc) call hits s.epochDuo.Load().EpochForRoad(oldRoad), which returns an error because the road is outside {Prev, Current}. (5) PushQC returns that error immediately, never reaching the needQC check that would have recognized gr.First < inner.nextQC and silently no-op'd. (6) The error propagates out of clientStreamFullCommitQCs into A's RunClient scope, tearing down A's entire session — consensus votes, ping, everything — until the dial loop reconnects.
Fix: check whether the QC is stale (e.g. below the local prune tip / already applied) before requiring the epoch window to resolve, mirroring PushBlock's if n < inner.first { return nil } pattern. This is a low-risk, mechanical fix and does not change any accepted behavior for QCs that are actually needed.
….Epochs Carry an EpochDuo (Prev|Current) on ViewSpec and consensus inner so a fresh AppQC lagging the proposing epoch by one verifies against its own committee. Also gate avail PushCommitQC's AppQC wait on the verified epoch, not the unverified qc.Proposal().EpochIndex(). Co-authored-by: Cursor <cursoragent@cursor.com>
There was a problem hiding this comment.
Large, well-tested refactor of the autobahn consensus layer to a two-epoch sliding window (EpochDuo) with explicit epoch transitions across data/avail/consensus/registry/p2p. No confirmed correctness bugs; the main concerns are two epoch-boundary races (surfaced by Codex) where a committee is resolved off-lock and acted on under-lock — both appear recoverable/self-correcting but warrant the author's confirmation. Cursor produced no output.
Findings: 0 blocking | 6 non-blocking | 2 posted inline
Blockers
- None at the file/PR level.
Non-blocking
- Cursor's second-opinion review file (cursor-review.md) was empty — that pass produced no output.
- REVIEW_GUIDELINES.md was empty/missing, so no repo-specific standards were applied beyond AGENTS.md.
- PR title and description reference "EpochTrio" and "data.EpochTrio()", but the implementation is "EpochDuo" (Prev, Current; Next intentionally not held) exposed via data.EpochDuo(). Update the description to match the code to avoid confusing future readers.
- Context (not a defect): all placeholder epochs are seeded from the genesis committee (Registry.makeEpoch always uses genesis.Committee()), and EvmShard routing/committee derivation is still the genesis committee (explicit TODOs). The new epoch-boundary code paths are therefore not yet exercised against genuinely different committees; the multi-committee safety of these transitions remains untested until real committee derivation lands.
- 2 suggestion(s)/nit(s) flagged inline on specific lines.
| if !ok { | ||
| return fmt.Errorf("unknown epoch_index %d", qc.QC().Proposal().EpochIndex()) | ||
| // Out-of-window ⇒ below prune tip; reject. | ||
| ep, err := s.epochDuo.Load().EpochForRoad(qc.QC().Proposal().Index()) |
There was a problem hiding this comment.
[suggestion] PushQC now resolves the epoch via epochDuo.Load().EpochForRoad(idx) at the top and hard-returns the error for any road outside the current window. Previously EpochByIndex resolved ahead-of-window epochs from the registry and ordering was handled by the nextQC gating under the lock.
data.PushQC has two concurrent callers that share the same *data.State: the ordered avail Run loop (state.go:765) and the p2p clientStreamFullCommitQCs handler (giga/data.go:39). When the boundary CommitQC (last road of epoch N) is still being applied (it advances the window under-lock after an off-lock WaitForDuo), a concurrent stream delivering the first QC of epoch N+1 will hit EpochForRoad before the window advances, get an error, and tear down the p2p stream at each epoch boundary. It recovers via reconnect, so this is liveness/churn rather than a safety issue — but consider treating an ahead-of-window road as a WaitForDuo/wait (as PushCommitQC does) or a silent drop instead of a hard error. (Matches Codex's high-severity finding; I'd rate it lower given it's transient and once-per-epoch.)
| } | ||
| return vote.VerifySig(c) | ||
| }); err != nil { | ||
| c := s.epochDuo.Load().Current.Committee() |
There was a problem hiding this comment.
[suggestion] PushVote verifies against the committee loaded off-lock here (c := s.epochDuo.Load().Current.Committee()), then re-loads the duo under the inner lock (line 587) and weights the vote with duo.Current. Because advanceEpoch stores the new duo under that same inner lock, a boundary advance interleaving between verify and lock acquisition makes the verification committee differ from the counting committee. PushBlock has the analogous split (verify at line 510, insert under lock).
I don't see a path to a wrongly-formed QC here: pushVote re-gates weight against the committee it commits to (a signer absent from the new committee gets weight 0 and is kept only for reweight, and a new-committee-only signer would fail the old-committee verify and never reach the lock). But the verify-vs-count committee split is fragile and easy to break in future edits — recommend snapshotting a single duo and using it for both verification and weighting (or asserting the epoch is unchanged under the lock). (Matches Codex's medium-severity finding.)
Superseded: latest AI review found no blocking issues.
Co-authored-by: Cursor <cursoragent@cursor.com>
There was a problem hiding this comment.
A large, well-tested refactor that swaps global "latest epoch" lookups for a sliding EpochDuo{Prev, Current} window with road-index-keyed registry machinery. No confirmed correctness blockers — notably because all epochs currently share the genesis committee, so the Current-only verification concerns are latent rather than live — but there are notable design/documentation gaps worth resolving before real committee rotation lands.
Findings: 0 blocking | 8 non-blocking | 3 posted inline
Blockers
- None at the file/PR level.
Non-blocking
- PR title/description vs implementation mismatch: the title says "EpochTrio" and the description repeatedly describes
EpochTrio{Prev, Current, Next}and lane traffic verified "against Current+Next", but the code shipsEpochDuo{Prev, Current}with Next intentionally omitted (epoch_duo.go). This is Codex's P1. It is NOT a live correctness bug today because every epoch uses the genesis committee (makeEpoch), so no validator is excluded from Current; but once committees actually rotate, new-epoch-only producers cannot get blocks/lane-votes admitted until the boundary CommitQC advances Current. Please update the description to match the Duo design and confirm whether the Next window is genuinely unnecessary for committee changes. - Consensus
pushCommitQChard-errors (and fails the loop) at an epoch boundary if N+1 is not yet registered, whereas avail/dataPushCommitQC/PushQCblock onWaitForDuo. This asymmetry is documented as intentional (relying on execution seeding N+1 ahead of consensus), but it is fragile: any future change that lets execution lag consensus turns a transient race into a node-level failure. Consider at least a louder invariant assertion or a bounded wait. Registry.makeEpochsilently overwrites an existings.m[idx]and advanceshighestEpoch. All current callers (SetupInitialDuo, AdvanceIfNeeded) guard with an existence check first, so it's harmless while committees are identical, but the clobbering behavior is a foot-gun once epochs carry distinct committees — an accidental re-seed would replace a real committee with a genesis placeholder. Consider making it insert-only (or panic/return on conflict).- No test exercises a consensus epoch-boundary transition end-to-end (pushCommitQC crossing Current.Next into N+1). EpochLength=108_000 makes a real crossing impractical, and the boundary logic is only covered indirectly via advanceEpoch/registry unit tests plus the identical-committee assumption. Consider a focused unit test that drives consensus/avail across a synthetic boundary with distinct committees to lock in the transition before real rotation lands.
- Second-opinion coverage:
cursor-review.mdwas empty (Cursor produced no output);codex-review.mdwas present and its P1 is incorporated above. - 3 suggestion(s)/nit(s) flagged inline on specific lines.
|
|
||
| // EpochDuo is a sliding window of up to two consecutive epochs. | ||
| // Current is always set; Prev is absent only for epoch 0. | ||
| // |
There was a problem hiding this comment.
[suggestion] This is the crux of Codex's P1: the PR title/description describe an EpochTrio{Prev, Current, Next} where lane traffic verifies against Current+Next, but the implementation is a Duo that admits new-committee lane traffic only after the boundary CommitQC advances Current. That is fine today because every epoch shares the genesis committee (makeEpoch uses genesis.Committee()), so no validator is ever outside Current — but once committees actually rotate, a validator that is only in epoch N+1 will have its blocks/lane-votes rejected by PushBlock/PushVote (Current-only verify) for the entire duration of epoch N. Please confirm the Next window is truly unnecessary and reconcile the PR description.
| } | ||
| return p.VerifySig(c) | ||
| }); err != nil { | ||
| c := s.epochDuo.Load().Current.Committee() |
There was a problem hiding this comment.
[nit] PushBlock (and PushVote below) verify only against Current's committee. With today's identical-committee epochs this is correct, but it is the concrete spot where an upcoming-epoch-only producer would be rejected until the boundary advances the window. Worth a comment cross-referencing the EpochDuo/Next design decision so this isn't mistaken for a bug once committees differ.
| // Invariant: N+1 is registered before this CommitQC (setup / | ||
| // AdvanceIfNeeded). Hard-error if missing — do not WaitForDuo | ||
| // like avail/data do. | ||
| nextEp, err := s.registry.EpochAt(nextRoad) |
There was a problem hiding this comment.
[suggestion] This hard-errors if N+1 isn't registered at the boundary, unlike avail/data which WaitForDuo. The invariant (execution seeds N+1 before consensus needs it) holds under the current sequential model, but a hard error here means any future violation crashes the consensus loop rather than backpressuring. Given the logger.Error already flags it, consider whether a bounded wait would be safer, or at minimum document the exact ordering guarantee that makes EpochAt infallible here.
2 participants
Data flow
The system has three layers that each need epoch information, but for different reasons:
dataandavail— each caches its ownEpochTrio{Prev, Current, Next}(AtomicSendoninner,AtomicRecvonState). CommitQCs and blocks arrive in a sliding window that straddles epoch boundaries: a QC at the last road of epoch N must verify against epoch N's committee, while the next one needs epoch N+1's. Both layers advance their cached trio under theirinnerlock when a CommitQC crosses an epoch boundary. Lane votes/blocks verify against Current+Next so producers in the upcoming epoch can participate before AppQC (the prune tip) catches up; late AppQCs may still verify via Prev.consensus— caches a single*Epoch. Consensus processes one view at a time and always stays within a single epoch. It only needs one committee at a time; when a CommitQC crosses the boundary, it looks up the next epoch from the registry and replaces its cached pointer. Incoming votes are gated by the vote's ownView().EpochIndexagainst this cached epoch — a vote for another epoch is dropped rather than verified against the wrong committee. Dropped votes are not redelivered; lagging nodes recover via the timeout path (new view → fresh votes).EVM shard routing — the p2p router reads
data.EpochTrio()(returned by value fromAtomicRecv) fresh on each send and usesCurrent.Committee().EvmShard(sender)to route incoming EVM transactions to the correct validator shard.Registryand boundary seeding —AdvanceIfNeededis driven by block execution:executeBlockseeds from the executed block's CommitQC road (notFinalAppState, which can lag under AppQC carry-forward). A block executed in epoch N seeds epoch N+2 into the registry as a placeholder; epoch N+1 is seeded one epoch earlier by executing epoch N-1 blocks, or bySetupInitialTrioat startup. Live execution registers epochs forward without gaps ahead of the tip;SetupInitialTrio(from the data CommitQC WAL tip, plus consensus's persisted tip if it leads) seeds a window around restart and may leave unused gaps below that tip — those are neverWaitForEpochtargets.At an epoch boundary,
avail.PushCommitQCanddata.PushQCresolve the nextEpochTriobefore advancing — a plainTrioAtfast path, falling back toRegistry.WaitForEpoch(N+2)when a node's CommitQC stream has outrun its own block execution. Such a node stalls at the boundary (off any lock, so execution keeps seeding and wakes it) until N+2 is registered, rather than erroring. Consensus itself only ever needs N+1 — always seeded before N+2 by sequential execution / setup — so no consensus-side midpoint gate is required.