Skip to content

fix(archiver): authenticate every Inbox message recovery anchor on L1 - #25438

Draft
spalladino wants to merge 3 commits into
spl/fi2-f14-l1-endpoint-checkfrom
spl/fi2-f15-authenticate-recovery-anchor
Draft

fix(archiver): authenticate every Inbox message recovery anchor on L1#25438
spalladino wants to merge 3 commits into
spl/fi2-f14-l1-endpoint-checkfrom
spl/fi2-f15-authenticate-recovery-anchor

Conversation

@spalladino

Copy link
Copy Markdown
Contributor

Background

The archiver stores each L1-to-L2 Inbox message with its message index, cumulative rolling hash, and the L1 block
where the event was first observed. When its local message log disagrees with L1, recovery searches backward for a
recovery anchor: the newest stored message that L1 still contains at the same index and rolling hash. Recovery keeps
the prefix through that anchor, deletes the suffix, and resumes ingestion from around the anchor's L1 block. The
archiver also persists a finality marker from the last synchronization that successfully agreed with L1.

The problem

A message's stored L1 height does not change when the same event is re-mined elsewhere but the resulting Inbox
content remains identical. The old recovery path nevertheless treated a stored height at or below the finality
marker as proof that the message was final, without looking up its event.

The regression test demonstrates the resulting loop:

  1. Two messages are first observed at L1 block 100, so both stored rows record block 100.
  2. L1 re-mines the same messages at block 130 with the same indices, contents, and rolling hashes. The Inbox still
    agrees with the local log, so synchronization has no reason to refetch the rows; they continue to say block 100.
  3. The persisted finality marker advances to block 120. The stored rows now appear to be below finality even though
    their actual events are at block 130, above the marker and still replaceable.
  4. L1 replaces block 130 and inserts two different messages at the same indices.
  5. Recovery examines the old messages. Because their recorded block 100 is at or below marker 120, the old shortcut
    accepts them without checking L1. It therefore retains both messages and effectively rolls the local count from
    two back to two.
  6. Forward ingestion finds the replacements at block 130, but they begin at index zero and cannot be appended to
    the retained two-message prefix. Recovery starts again and selects the same false anchor.

The node stops making progress on Inbox message synchronization. The old implementation repeats this cycle until a
restart causes a cold search rather than reusing the false anchor.

What this changes

A stored L1 height is now only a search hint. Every recovery anchor, including a candidate recorded at or below the
persisted finality marker, must be authenticated by a bounded L1 event lookup that finds the same message index and
rolling hash.

In the example, neither old message can be found unchanged on L1. Recovery walks past both candidates, selects the
zero-message position, rewinds to the Inbox deployment block, and lets ordinary forward ingestion store the two
replacement messages. The regression test confirms that synchronization reaches L1 head 140 and exits recovery.

The existing lookup budget still limits work per synchronization pass, and lookups remain bounded above by the
captured L1 head. Rollback, message deletion, cursor movement, and pruning of proposed blocks that depend on the
removed suffix remain one store transaction.

What this does not do

This does not update a stored message's original L1 height when an unchanged event is re-mined. It removes the
assumption that the old height proves the event's current placement. The authenticated, monotonically advancing
finality marker remains synchronization metadata, but it no longer bypasses anchor authentication.

Recovery may now perform more backward event lookups and may require multiple passes when it exhausts the existing
per-pass budget. Conservative rollback can prune proposed blocks whose consumed messages appeared to be below the
finality marker; forward ingestion restores the canonical messages, but this change does not restore those proposed
blocks. No cache or additional persisted recovery state is introduced.

The captured-head bound, deployment-block fallback, and recovery behavior for unreadable or replaced L1 views are
unchanged.

Testing

The regression test covers messages first observed at block 100, re-mined unchanged at block 130, a finality marker
at 120, replacement at block 130, fallback to an empty anchor, forward refill, and successful exit from recovery.
Existing recovery tests now require an event lookup even for candidates recorded at or below the finality marker,
and continue to cover bounded searches and finding an authenticated common prefix.

These are fake-chain archiver tests. The diff does not add a live-provider reorg test or a performance benchmark for
the additional backward lookups.

@spalladino
spalladino added this pull request to stack #25417 September 9, 2026 21:01
@spalladino
spalladino force-pushed the spl/fi2-f15-authenticate-recovery-anchor branch from 2471c06 to 628bd86 Compare September 9, 2026 21:15
@spalladino
spalladino removed this pull request from stack #25417 September 9, 2026 21:16
@spalladino
spalladino added this pull request to stack #25441 September 9, 2026 21:16
@spalladino
spalladino force-pushed the spl/fi2-f15-authenticate-recovery-anchor branch from 628bd86 to 2ffb542 Compare September 10, 2026 02:46
@spalladino
spalladino force-pushed the spl/fi2-f15-authenticate-recovery-anchor branch from 2ffb542 to 9f09867 Compare September 10, 2026 04:04
@spalladino
spalladino force-pushed the spl/fi2-f15-authenticate-recovery-anchor branch from 9f09867 to 22feab1 Compare September 10, 2026 05:12
@spalladino
spalladino removed this pull request from stack #25441 September 10, 2026 11:49
@spalladino
spalladino added this pull request to stack #25448 September 10, 2026 11:49
@spalladino
spalladino removed this pull request from stack #25448 September 10, 2026 12:09
@spalladino
spalladino changed the base branch from spl/fi2-f14-l1-endpoint-check to spl/fi2-f16-rescue-tail-blob September 10, 2026 12:10
@spalladino
spalladino changed the base branch from spl/fi2-f16-rescue-tail-blob to spl/fi2-f14-l1-endpoint-check September 10, 2026 12:10
@spalladino
spalladino added this pull request to stack #25449 September 10, 2026 12:11
@spalladino
spalladino force-pushed the spl/fi2-f15-authenticate-recovery-anchor branch from a212324 to 3e6e866 Compare September 10, 2026 12:50
@spalladino
spalladino removed this pull request from stack #25449 September 10, 2026 13:04
@spalladino
spalladino changed the base branch from spl/fi2-f14-l1-endpoint-check to spl/fi2-f16-rescue-tail-blob September 10, 2026 13:05
@spalladino
spalladino changed the base branch from spl/fi2-f16-rescue-tail-blob to spl/fi2-f14-l1-endpoint-check September 10, 2026 13:05
@spalladino
spalladino added this pull request to stack #25451 September 10, 2026 13:05
@spalladino
spalladino force-pushed the spl/fi2-f15-authenticate-recovery-anchor branch from 3e6e866 to 2937f3d Compare September 10, 2026 13:51
@spalladino
spalladino removed this pull request from stack #25451 September 10, 2026 13:53
@spalladino
spalladino changed the base branch from spl/fi2-f14-l1-endpoint-check to spl/fi2-f16-rescue-tail-blob September 10, 2026 13:54
@spalladino
spalladino changed the base branch from spl/fi2-f16-rescue-tail-blob to spl/fi2-f14-l1-endpoint-check September 10, 2026 13:54
@spalladino
spalladino added this pull request to stack #25452 September 10, 2026 13:54
@spalladino
spalladino marked this pull request as draft September 10, 2026 14:22
spalladino and others added 3 commits September 10, 2026 11:28
A stored message row keeps the L1 height it was first observed at, and recovery accepted any candidate recorded
at or below the persisted finality marker as an anchor with no event lookup. That recorded height is never
refreshed while the log agrees with the Inbox at the captured head, so a message re-mined to a higher block
keeps its old, lower height — which can sit below the marker while the message itself is above it, unfinalized
and replaceable.

When such a message is later replaced, recovery keeps a prefix L1 no longer has: it rolls back from N to N,
refetches, fails to chain the replacement onto the retained prefix, and restarts on the same false anchor. The
node then makes no further progress on messages until it is restarted.

Every anchor is now a message a bounded event lookup positively found on L1 at the same index and rolling hash.
The per-pass lookup budget, the lookup window bounded above by the captured head, the deployment-block fallback
and the single-transaction rollback are unchanged, and no new cache or persisted state is introduced. The
finality marker is still written and advanced monotonically on authenticated syncs: what is removed is reliance
on stale placement, not finality itself.

The cost is that a recovery which used to stop early keeps walking backwards, and may prune proposed blocks
whose messages sit below the marker; those messages return through ordinary forward ingestion.
…ad backwards

addBlock triggers a sync it does not await, so the deployment-refill fixture left a pass in flight that
had captured the pre-reorg head. Dropping the finalized-height shortcut made that pass do two event
lookups and a rollback instead of returning at once, so it could now commit after the pass for the new,
lower head and leave the old height as the synced one. The fixture failed about three times in ten;
draining the pending sync before moving the head fixes it, 30 runs clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The helper this fixture calls drains the block-triggered sync itself, so the inline drain added here is a
no-op second call.
@spalladino
spalladino force-pushed the spl/fi2-f15-authenticate-recovery-anchor branch from 2937f3d to cc6d63a Compare September 10, 2026 14:29
@spalladino
spalladino removed this pull request from stack #25452 September 10, 2026 14:31
@spalladino
spalladino changed the base branch from spl/fi2-f14-l1-endpoint-check to spl/fi2-f16-rescue-tail-blob September 10, 2026 14:31
@spalladino
spalladino changed the base branch from spl/fi2-f16-rescue-tail-blob to spl/fi2-f14-l1-endpoint-check September 10, 2026 14:31
@spalladino
spalladino added this pull request to stack #25453 September 10, 2026 14:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant