fix(archiver): authenticate every Inbox message recovery anchor on L1 - #25438
Draft
spalladino wants to merge 3 commits into
Draft
fix(archiver): authenticate every Inbox message recovery anchor on L1#25438spalladino wants to merge 3 commits into
spalladino wants to merge 3 commits into
Conversation
spalladino
added this pull request to stack #25417
September 9, 2026 21:01
spalladino
force-pushed
the
spl/fi2-f15-authenticate-recovery-anchor
branch
from
September 9, 2026 21:15
2471c06 to
628bd86
Compare
spalladino
removed this pull request from stack #25417
September 9, 2026 21:16
spalladino
added this pull request to stack #25441
September 9, 2026 21:16
spalladino
force-pushed
the
spl/fi2-f15-authenticate-recovery-anchor
branch
from
September 10, 2026 02:46
628bd86 to
2ffb542
Compare
spalladino
requested review from
IlyasRidhuan,
LeilaWang,
iAmMichaelConnor,
iakovenkos and
ledwards2225
as code owners
September 10, 2026 04:04
spalladino
force-pushed
the
spl/fi2-f15-authenticate-recovery-anchor
branch
from
September 10, 2026 04:04
2ffb542 to
9f09867
Compare
spalladino
force-pushed
the
spl/fi2-f15-authenticate-recovery-anchor
branch
from
September 10, 2026 05:12
9f09867 to
22feab1
Compare
spalladino
removed this pull request from stack #25441
September 10, 2026 11:49
spalladino
added this pull request to stack #25448
September 10, 2026 11:49
spalladino
removed this pull request from stack #25448
September 10, 2026 12:09
spalladino
changed the base branch from
spl/fi2-f14-l1-endpoint-check
to
spl/fi2-f16-rescue-tail-blob
September 10, 2026 12:10
spalladino
changed the base branch from
spl/fi2-f16-rescue-tail-blob
to
spl/fi2-f14-l1-endpoint-check
September 10, 2026 12:10
spalladino
added this pull request to stack #25449
September 10, 2026 12:11
spalladino
force-pushed
the
spl/fi2-f15-authenticate-recovery-anchor
branch
from
September 10, 2026 12:50
a212324 to
3e6e866
Compare
spalladino
removed this pull request from stack #25449
September 10, 2026 13:04
spalladino
changed the base branch from
spl/fi2-f14-l1-endpoint-check
to
spl/fi2-f16-rescue-tail-blob
September 10, 2026 13:05
spalladino
changed the base branch from
spl/fi2-f16-rescue-tail-blob
to
spl/fi2-f14-l1-endpoint-check
September 10, 2026 13:05
spalladino
added this pull request to stack #25451
September 10, 2026 13:05
spalladino
force-pushed
the
spl/fi2-f15-authenticate-recovery-anchor
branch
from
September 10, 2026 13:51
3e6e866 to
2937f3d
Compare
spalladino
removed this pull request from stack #25451
September 10, 2026 13:53
spalladino
changed the base branch from
spl/fi2-f14-l1-endpoint-check
to
spl/fi2-f16-rescue-tail-blob
September 10, 2026 13:54
spalladino
changed the base branch from
spl/fi2-f16-rescue-tail-blob
to
spl/fi2-f14-l1-endpoint-check
September 10, 2026 13:54
spalladino
added this pull request to stack #25452
September 10, 2026 13:54
spalladino
removed request for
IlyasRidhuan,
LeilaWang,
iAmMichaelConnor,
iakovenkos and
ledwards2225
September 10, 2026 14:22
spalladino
marked this pull request as draft
September 10, 2026 14:22
A stored message row keeps the L1 height it was first observed at, and recovery accepted any candidate recorded at or below the persisted finality marker as an anchor with no event lookup. That recorded height is never refreshed while the log agrees with the Inbox at the captured head, so a message re-mined to a higher block keeps its old, lower height — which can sit below the marker while the message itself is above it, unfinalized and replaceable. When such a message is later replaced, recovery keeps a prefix L1 no longer has: it rolls back from N to N, refetches, fails to chain the replacement onto the retained prefix, and restarts on the same false anchor. The node then makes no further progress on messages until it is restarted. Every anchor is now a message a bounded event lookup positively found on L1 at the same index and rolling hash. The per-pass lookup budget, the lookup window bounded above by the captured head, the deployment-block fallback and the single-transaction rollback are unchanged, and no new cache or persisted state is introduced. The finality marker is still written and advanced monotonically on authenticated syncs: what is removed is reliance on stale placement, not finality itself. The cost is that a recovery which used to stop early keeps walking backwards, and may prune proposed blocks whose messages sit below the marker; those messages return through ordinary forward ingestion.
…ad backwards addBlock triggers a sync it does not await, so the deployment-refill fixture left a pass in flight that had captured the pre-reorg head. Dropping the finalized-height shortcut made that pass do two event lookups and a rollback instead of returning at once, so it could now commit after the pass for the new, lower head and leave the old height as the synced one. The fixture failed about three times in ten; draining the pending sync before moving the head fixes it, 30 runs clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The helper this fixture calls drains the block-triggered sync itself, so the inline drain added here is a no-op second call.
spalladino
force-pushed
the
spl/fi2-f15-authenticate-recovery-anchor
branch
from
September 10, 2026 14:29
2937f3d to
cc6d63a
Compare
spalladino
removed this pull request from stack #25452
September 10, 2026 14:31
spalladino
changed the base branch from
spl/fi2-f14-l1-endpoint-check
to
spl/fi2-f16-rescue-tail-blob
September 10, 2026 14:31
spalladino
changed the base branch from
spl/fi2-f16-rescue-tail-blob
to
spl/fi2-f14-l1-endpoint-check
September 10, 2026 14:31
spalladino
added this pull request to stack #25453
September 10, 2026 14:32
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background
The archiver stores each L1-to-L2 Inbox message with its message index, cumulative rolling hash, and the L1 block
where the event was first observed. When its local message log disagrees with L1, recovery searches backward for a
recovery anchor: the newest stored message that L1 still contains at the same index and rolling hash. Recovery keeps
the prefix through that anchor, deletes the suffix, and resumes ingestion from around the anchor's L1 block. The
archiver also persists a finality marker from the last synchronization that successfully agreed with L1.
The problem
A message's stored L1 height does not change when the same event is re-mined elsewhere but the resulting Inbox
content remains identical. The old recovery path nevertheless treated a stored height at or below the finality
marker as proof that the message was final, without looking up its event.
The regression test demonstrates the resulting loop:
agrees with the local log, so synchronization has no reason to refetch the rows; they continue to say block 100.
their actual events are at block 130, above the marker and still replaceable.
accepts them without checking L1. It therefore retains both messages and effectively rolls the local count from
two back to two.
the retained two-message prefix. Recovery starts again and selects the same false anchor.
The node stops making progress on Inbox message synchronization. The old implementation repeats this cycle until a
restart causes a cold search rather than reusing the false anchor.
What this changes
A stored L1 height is now only a search hint. Every recovery anchor, including a candidate recorded at or below the
persisted finality marker, must be authenticated by a bounded L1 event lookup that finds the same message index and
rolling hash.
In the example, neither old message can be found unchanged on L1. Recovery walks past both candidates, selects the
zero-message position, rewinds to the Inbox deployment block, and lets ordinary forward ingestion store the two
replacement messages. The regression test confirms that synchronization reaches L1 head 140 and exits recovery.
The existing lookup budget still limits work per synchronization pass, and lookups remain bounded above by the
captured L1 head. Rollback, message deletion, cursor movement, and pruning of proposed blocks that depend on the
removed suffix remain one store transaction.
What this does not do
This does not update a stored message's original L1 height when an unchanged event is re-mined. It removes the
assumption that the old height proves the event's current placement. The authenticated, monotonically advancing
finality marker remains synchronization metadata, but it no longer bypasses anchor authentication.
Recovery may now perform more backward event lookups and may require multiple passes when it exhausts the existing
per-pass budget. Conservative rollback can prune proposed blocks whose consumed messages appeared to be below the
finality marker; forward ingestion restores the canonical messages, but this change does not restore those proposed
blocks. No cache or additional persisted recovery state is introduced.
The captured-head bound, deployment-block fallback, and recovery behavior for unreadable or replaced L1 views are
unchanged.
Testing
The regression test covers messages first observed at block 100, re-mined unchanged at block 130, a finality marker
at 120, replacement at block 130, fallback to an empty anchor, forward refill, and successful exit from recovery.
Existing recovery tests now require an event lookup even for candidates recorded at or below the finality marker,
and continue to cover bounded searches and finding an authenticated common prefix.
These are fake-chain archiver tests. The diff does not add a live-provider reorg test or a performance benchmark for
the additional backward lookups.