From e95e15327482b5ad6edda9142f2ac398813145ce Mon Sep 17 00:00:00 2001 From: Santiago Palladino Date: Thu, 10 Sep 2026 00:32:54 -0300 Subject: [PATCH] feat(p2p): require the Inbox prefix reference on every block proposal The signed Inbox message-prefix reference is now a required field on a standalone BlockProposal and on the final block embedded in a CheckpointProposal, serialized without a presence flag ahead of the optional SignedTxs bundle and always included in the signing payload. The end-of-buffer fallback is gone: a proposal that omits or truncates the reference is malformed input rather than a valid zero-message proposal. The final block's reference is checked against the checkpoint header's inboxRollingHash whenever a final block is present, and a zero-new-message block re-states the prefix its parent ended at instead of leaving the field unset. Also links the three deferred backlog limitations from the code that still carries them. Co-Authored-By: Claude Opus 5 (1M context) --- .../docs/resources/migration_notes.md | 17 +++ .../operators/reference/changelog/v6.md | 6 + .../archiver/src/modules/l1_synchronizer.ts | 7 + .../aztec-node/node_public_calls_simulator.ts | 4 +- ...e2e_multi_validator_node_key_store.test.ts | 3 + .../sequencer/automine/automine_sequencer.ts | 6 + .../sequencer/checkpoint_proposal_job.test.ts | 6 +- .../src/sequencer/checkpoint_proposal_job.ts | 41 ++--- .../src/test/mock_checkpoint_builder.ts | 28 +++- .../sequencer-client/src/test/utils.ts | 9 +- .../stdlib/src/interfaces/validator.ts | 2 + .../stdlib/src/p2p/block_proposal.test.ts | 142 +++++++++--------- yarn-project/stdlib/src/p2p/block_proposal.ts | 65 ++++---- .../src/p2p/checkpoint_proposal.test.ts | 80 ++++++++-- .../stdlib/src/p2p/checkpoint_proposal.ts | 52 +++---- .../stdlib/src/p2p/wire_compat_fixtures.ts | 49 ++++-- yarn-project/stdlib/src/tests/mocks.ts | 9 +- yarn-project/validator-client/README.md | 5 +- .../src/duties/validation_service.test.ts | 4 + .../src/duties/validation_service.ts | 5 +- .../src/proposal_handler.test.ts | 25 +-- .../validator-client/src/proposal_handler.ts | 13 +- .../src/streaming_inbox_checks.test.ts | 15 +- .../src/streaming_inbox_checks.ts | 25 ++- .../src/validator.ha.integration.test.ts | 45 +++++- .../src/validator.integration.test.ts | 11 +- .../validator-client/src/validator.test.ts | 1 + .../validator-client/src/validator.ts | 2 +- 28 files changed, 406 insertions(+), 271 deletions(-) diff --git a/docs/docs-developers/docs/resources/migration_notes.md b/docs/docs-developers/docs/resources/migration_notes.md index fad58c5a826b..febd30e7a2cc 100644 --- a/docs/docs-developers/docs/resources/migration_notes.md +++ b/docs/docs-developers/docs/resources/migration_notes.md @@ -71,6 +71,23 @@ In `@aztec/ethereum/contracts`, the top-level `MessageSentLog.l1BlockTimestamp` **Impact**: Only node-internal and tooling consumers are affected; `getL1ToL2MessageIndex`, `getL1ToL2MessageMembershipWitness` and the Aztec.nr consumption functions are unchanged. Operators upgrading a node: see the node changelog for the archiver store reset. +### [Aztec Node] The Inbox prefix reference is a required proposal field, before the transaction bundle + +`BlockProposal.inboxPrefixRef` and `CheckpointProposal.lastBlock.inboxPrefixRef` are no longer optional, and the reference moves ahead of the optional `SignedTxs` bundle on the wire. Every proposed block carries one, including a block that consumes no new messages: such a block re-states the prefix its parent ended at, so the pair (the header's cumulative message count, the reference's rolling hash) is signed either way. The zero hash is a real value only for the empty prefix a chain starts from. + +The 32-byte reference is now written with no presence flag, immediately after the transaction hashes and before the `hasSignedTxs` flag, and it is always part of the signed payload. There is no end-of-buffer fallback: a proposal that omits or truncates the reference fails to decode instead of reading as a proposal that consumed nothing. A valid-reference proposal is four bytes smaller than before, because the presence flag is gone. The last block embedded in a `CheckpointProposal` uses the same layout, and its reference is checked against the checkpoint header's `inboxRollingHash` at construction and at decode. The embedded last block itself stays optional. + +This is a breaking peer-to-peer format change on top of the encoding change already noted above: a node on the previous format cannot decode a proposal from a node on this one, and vice versa. Every node that gossips, signs or validates proposals on a network has to run a matching version. + +**Migration:** + +```diff +- new BlockProposal(header, index, archive, txHashes, signature, signatureContext, signedTxs, inboxPrefixRef) ++ new BlockProposal(header, index, archive, txHashes, signature, signatureContext, inboxPrefixRef, signedTxs) +``` + +`BlockProposal.createProposalFromSigner` takes `inboxPrefixRef` after `signatureContext` and before the signer callbacks, and `Validator.createBlockProposal` takes it after `proposerAddress` and before `options`. `ValidationService.createBlockProposal` keeps its parameter order but the reference is required. Historical blocks replayed from L1 blobs are unaffected: the reference is peer-to-peer only and is not part of any published block. + ### [Aztec Node] Sequencer and validator streaming-Inbox internals For consumers wiring these packages directly. `Sequencer` takes an `InboxContract` constructor argument (after `rollupContract`) and `AutomineSequencerDeps` gains `inboxContract`; completion uses it to resolve a checkpoint's final message position to a live bucket. `NodePublicCallsSimulator` deps drop `l1Client` and `useAutomineSequencer`. `SequencerPublisher.validateCheckpointHeader` is replaced by `validateCheckpointHeaderAndInbox(header, { expectedTotal, expectedParentCheckpointNumber }, simulationOverridesPlan?)`, which returns the `bucketHint` to pass to `propose`. `@aztec/sequencer-client` no longer exports `InboxBucketConfirmationTracker`, `InboxBucketEligibility`, `L1BlockReader`, `immediateEligibility`, `ConsumedBucketCursor`, `InboxBucketSelection`, `InboxBucketSource`, `SelectInboxBucketInput` or `selectInboxBucketForBlock`; it exports `InboxConsumptionCaps`, `PROTOCOL_INBOX_CONSUMPTION_CAPS`, `InboxEndpointResolver`, `StreamingMessageSource`, `selectOrdinaryMessageEnd`, `selectSafeLocalEnd`, `getOrdinaryCeiling`, `mustQueryEndpoint`, `getEndpointUpperBound` and `resolveEndpoint` instead. These helpers are stateless: nothing is retained between blocks. `selectOrdinaryMessageEnd` returns the greedy end of the locally observed messages under the per-block and checkpoint caps; `getOrdinaryCeiling` is the threshold one bucket's worth of messages below the checkpoint cap, and `selectSafeLocalEnd` holds the greedy end down to it. `mustQueryEndpoint` decides, from that prospective end, whether the block has to resolve a live L1 bucket end at all, and `resolveEndpoint` performs the single Inbox lookup (bounded by `getEndpointUpperBound`) and authenticates the resolved end against the local message log in one snapshot. diff --git a/docs/docs-operate/operators/reference/changelog/v6.md b/docs/docs-operate/operators/reference/changelog/v6.md index e1407301e32a..f2ff05eb9e4a 100644 --- a/docs/docs-operate/operators/reference/changelog/v6.md +++ b/docs/docs-operate/operators/reference/changelog/v6.md @@ -16,6 +16,12 @@ The archiver now stores L1-to-L2 messages as a plain ordered log (compact index, The archiver is the only store the streaming Inbox changes; world state follows the archiver through the block stream, and the L1 transaction state store is unaffected. Other v6 work bumps other stores on its own schedule: `PXE_DATA_SCHEMA_VERSION` moves from 13 to 14 in this release for unrelated reasons, so a PXE reset in v6 is not caused by the streaming Inbox. +### Block proposals must carry the Inbox prefix reference + +Every proposed block now carries a required 32-byte reference to the Inbox message prefix it consumed through, including a block that consumes no new messages, which re-states the prefix its parent ended at. The reference is serialized without a presence flag, ahead of the optional transaction bundle, and is always part of the signed payload; a proposal that omits or truncates it is rejected as malformed rather than read as a block that consumed nothing. + +**Migration**: this is a breaking peer-to-peer format change. A node on the previous format cannot decode a proposal from a node on this one, and vice versa, so upgrade the peer set together. There is no supported mode in which both encodings circulate at once. Nothing in the L1 contracts, the published block format or any node configuration changes, and a node that syncs historical blocks from L1 is unaffected. + ### `maxBlocksPerCheckpoint` floor lowered from 7 to 4 The network consensus configuration requires `maxBlocksPerCheckpoint` to be at least `MIN_BLOCKS_FOR_INBOX_CATCHUP`, so a checkpoint can always clear a cap-sized backlog of mandatory L1-to-L2 messages. Blocks now consume messages by count rather than by whole L1 bucket, so the floor is `ceil(1024 / 256) = 4` instead of 7. Existing configurations remain valid; networks that were forced to shorten their block duration to satisfy the old floor may relax it. diff --git a/yarn-project/archiver/src/modules/l1_synchronizer.ts b/yarn-project/archiver/src/modules/l1_synchronizer.ts index 59cb2852d991..e43a765e02f6 100644 --- a/yarn-project/archiver/src/modules/l1_synchronizer.ts +++ b/yarn-project/archiver/src/modules/l1_synchronizer.ts @@ -680,6 +680,13 @@ export class ArchiverL1Synchronizer implements Traceable { // We must only set this block number based on actually retrieved logs. // TODO(#8621): Tackle this properly when we handle L1 Re-orgs. // await this.stores.blocks.setSynchedL1BlockNumber(currentL1BlockNumber); + // + // Known limitation: this short-circuit only compares the pending archive root, so a reconciliation that + // replaced the checkpoint at the same height leaves work outstanding that this pass skips, and the refetch + // waits for L1 to produce a block that moves the root again. A "checkpoint refetch still pending" flag + // carried out of reconciliation past this optimization would let the next pass fetch at the same head. It is + // general checkpoint-event syncing rather than an Inbox consumption rule, and is deferred: + // https://linear.app/aztec-labs/issue/A-1985 this.log.debug(`No checkpoints to retrieve from ${blocksSynchedTo + 1n} to ${currentL1BlockNumber}`); return { rollupStatus, fetchCheckpoints: false, provenArchive }; } diff --git a/yarn-project/aztec-node/src/aztec-node/node_public_calls_simulator.ts b/yarn-project/aztec-node/src/aztec-node/node_public_calls_simulator.ts index 0cd41b8ffa87..7f9af79decf7 100644 --- a/yarn-project/aztec-node/src/aztec-node/node_public_calls_simulator.ts +++ b/yarn-project/aztec-node/src/aztec-node/node_public_calls_simulator.ts @@ -253,7 +253,9 @@ export class NodePublicCallsSimulator { * 400 messages observed and live buckets ending at 200 and 400, this appends 256 while a final block lands on * 200: a public call consuming message index 220 simulates successfully and then fails when it runs for real. * Callers that need certainty check inclusion at an L2 tip that already exists, with `isL1ToL2MessageReady` from - * `@aztec/aztec.js/messaging`. + * `@aztec/aztec.js/messaging`. Closing that gap would mean this node running the sequencer's live endpoint + * selection, including its Inbox reads, on every simulation; it is deferred: + * https://linear.app/aztec-labs/issue/A-1982 * * Any failure, such as messages not synced yet or a torn archiver snapshot, leaves the fork at the tip state, * which is what the transaction sees if the next block consumes nothing. diff --git a/yarn-project/end-to-end/src/composed/web3signer/e2e_multi_validator_node_key_store.test.ts b/yarn-project/end-to-end/src/composed/web3signer/e2e_multi_validator_node_key_store.test.ts index 87cfd0da01a0..4bc33f0ae965 100644 --- a/yarn-project/end-to-end/src/composed/web3signer/e2e_multi_validator_node_key_store.test.ts +++ b/yarn-project/end-to-end/src/composed/web3signer/e2e_multi_validator_node_key_store.test.ts @@ -16,6 +16,7 @@ import { type EthPrivateKey, KeystoreManager, loadKeystores, mergeKeystores } fr import { StatefulTestContractArtifact } from '@aztec/noir-test-contracts.js/StatefulTest'; import type { Sequencer, SequencerClient, SequencerPublisherFactory } from '@aztec/sequencer-client'; import type { TestSequencer, TestSequencerClient } from '@aztec/sequencer-client/test'; +import type { InboxMessagePrefixRef } from '@aztec/stdlib/messaging'; import type { BlockProposalOptions } from '@aztec/stdlib/p2p'; import type { BlockHeader, Tx } from '@aztec/stdlib/tx'; import { NodeKeystoreAdapter, ValidatorClient } from '@aztec/validator-client'; @@ -379,6 +380,7 @@ describe('e2e_multi_validator_node', () => { archive: Fr, txs: Tx[], proposerAddress: EthAddress | undefined, + inboxPrefixRef: InboxMessagePrefixRef, options: BlockProposalOptions, ) => { if (proposerAddress) { @@ -399,6 +401,7 @@ describe('e2e_multi_validator_node', () => { archive, txs, proposerAddress, + inboxPrefixRef, options, ); }; diff --git a/yarn-project/sequencer-client/src/sequencer/automine/automine_sequencer.ts b/yarn-project/sequencer-client/src/sequencer/automine/automine_sequencer.ts index 65db37ad5695..16f9811de5bf 100644 --- a/yarn-project/sequencer-client/src/sequencer/automine/automine_sequencer.ts +++ b/yarn-project/sequencer-client/src/sequencer/automine/automine_sequencer.ts @@ -481,6 +481,12 @@ export class AutomineSequencer { // resolve the live L1 bucket end at or below it with one Inbox call, and authenticate the range to it against // the local log. Landing on that boundary can leave observed messages behind, so the block may consume fewer // than the caps allow. The parent total is the fork's L1-to-L2 leaf count (compact indexing). + // + // Known limitation: a single block can insert at most the per-block cap, so once more messages than that have + // aged past L1's censorship deadline, no single-block checkpoint can consume all the ones `propose` demands and + // automine stops making progress. An ordinary sequencer spreads the backlog over several blocks of one slot; + // automine has no second block to spread it over. Working around it by warping the L1 timestamp so the deadline + // moves is deferred: https://linear.app/aztec-labs/issue/A-1981 const parentTotalMsgCount = (await fork.getTreeInfo(MerkleTreeId.L1_TO_L2_MESSAGE_TREE)).size; const cursor = await this.deps.l1ToL2MessageSource.getMessagePosition(parentTotalMsgCount); if (cursor === undefined) { diff --git a/yarn-project/sequencer-client/src/sequencer/checkpoint_proposal_job.test.ts b/yarn-project/sequencer-client/src/sequencer/checkpoint_proposal_job.test.ts index 07b0ff6470f7..f9d5cb0fcc95 100644 --- a/yarn-project/sequencer-client/src/sequencer/checkpoint_proposal_job.test.ts +++ b/yarn-project/sequencer-client/src/sequencer/checkpoint_proposal_job.test.ts @@ -307,7 +307,7 @@ describe('CheckpointProposalJob', () => { validatorClient = mock(); validatorClient.collectAttestations.mockImplementation(() => Promise.resolve([])); validatorClient.createBlockProposal.mockImplementation( - async (blockHeader, _checkpointNumber, indexWithinCheckpoint, archiveRoot, txs) => { + async (blockHeader, _checkpointNumber, indexWithinCheckpoint, archiveRoot, txs, _proposer, inboxPrefixRef) => { const txHashes = await Promise.all((txs ?? []).map((tx: Tx) => tx.getTxHash())); return new BlockProposal( blockHeader, @@ -316,6 +316,7 @@ describe('CheckpointProposalJob', () => { txHashes, mockedSig, signatureContext, + inboxPrefixRef, ); }, ); @@ -342,6 +343,7 @@ describe('CheckpointProposalJob', () => { indexWithinCheckpoint: lastBlockInfo.indexWithinCheckpoint, txHashes, signature: mockedSig, + inboxPrefixRef: lastBlockInfo.inboxPrefixRef, // Note: signedTxs omitted since publishTxsWithProposals is false in tests }, ); @@ -1606,7 +1608,7 @@ describe('CheckpointProposalJob', () => { }; const bundleLengths = () => checkpointBuilder.buildBlockCalls.map(call => call.opts.l1ToL2Messages?.length); const signedPrefixes = () => - validatorClient.createBlockProposal.mock.calls.map(call => call[7]?.inboxRollingHash.toString()); + validatorClient.createBlockProposal.mock.calls.map(call => call[6].inboxRollingHash.toString()); const prefixAt = (count: number) => streamingInbox.positionAt(BigInt(count)).rollingHash.toString(); const preflightTotals = () => publisher.validateCheckpointHeaderAndInbox.mock.calls.map(call => call[1].expectedTotal); diff --git a/yarn-project/sequencer-client/src/sequencer/checkpoint_proposal_job.ts b/yarn-project/sequencer-client/src/sequencer/checkpoint_proposal_job.ts index e894e8515c00..78dfa1f2982b 100644 --- a/yarn-project/sequencer-client/src/sequencer/checkpoint_proposal_job.ts +++ b/yarn-project/sequencer-client/src/sequencer/checkpoint_proposal_job.ts @@ -1204,7 +1204,7 @@ export class CheckpointProposalJob implements Traceable { checkpointBuilder: CheckpointBuilder, timestamp: bigint, blockProposalOptions: BlockProposalOptions, - streamingState?: StreamingCheckpointState, + streamingState: StreamingCheckpointState, ): Promise { const blocksInCheckpoint: L2Block[] = []; const txHashesAlreadyIncluded = new Set(); @@ -1250,17 +1250,15 @@ export class CheckpointProposalJob implements Traceable { // includes the block that reaches the per-checkpoint block cap, not just the timetable's last sub-slot. const maxBlocks = Math.min(this.config.maxBlocksPerCheckpoint, this.timetable.getMaxBlocksPerCheckpoint()); const isCheckpointFinalBlock = timingInfo.isLastBlock || blocksBuilt + 1 >= maxBlocks; - const selection = streamingState - ? await this.selectStreamingBundle(streamingState, { - isFinalBlock: isCheckpointFinalBlock, - buildDeadline: timingInfo.deadline, - }) - : undefined; + const selection = await this.selectStreamingBundle(streamingState, { + isFinalBlock: isCheckpointFinalBlock, + buildDeadline: timingInfo.deadline, + }); // No checkpoint ending on the messages this block could consume can be published: stop before signing // anything more and give up the slot. - if (selection?.kind === 'abort') { - this.reportStreamingAbort(streamingState!, selection.reason, { + if (selection.kind === 'abort') { + this.reportStreamingAbort(streamingState, selection.reason, { blocksBuilt, blockNumber, ...selection.context, @@ -1268,7 +1266,7 @@ export class CheckpointProposalJob implements Traceable { return { aborted: true }; } - const streamingBundle = selection?.range.messages; + const streamingBundle = selection.range.messages; const buildResult = await this.buildSingleBlock(checkpointBuilder, { // Create all blocks with the same timestamp @@ -1327,11 +1325,8 @@ export class CheckpointProposalJob implements Traceable { // Streaming Inbox: the block built successfully, so advance the cursor to the prefix it consumed through and // sign that prefix as this block's reference. A block that consumed nothing re-signs the cursor's prefix. - let blockPrefixRef: InboxMessagePrefixRef | undefined = undefined; - if (streamingState && selection) { - streamingState.cursor = selection.range.end; - blockPrefixRef = InboxMessagePrefixRef.fromPosition(streamingState.cursor); - } + streamingState.cursor = selection.range.end; + const blockPrefixRef = InboxMessagePrefixRef.fromPosition(streamingState.cursor); // Sign the block proposal. This will throw if HA signing fails. const proposal = await this.createBlockProposal( @@ -1378,7 +1373,6 @@ export class CheckpointProposalJob implements Traceable { // block rather than lose the slot; this is the only place the loop overrides the timetable. A cursor still at the // checkpoint start needs nothing: the parent checkpoint already ended on a live bucket end. if ( - streamingState && ranOutOfSubslots && blocksInCheckpoint.length > 0 && streamingState.cursor.totalMessageCount > streamingState.checkpointStartTotalMsgCount @@ -1524,7 +1518,7 @@ export class CheckpointProposalJob implements Traceable { block: L2Block, usedTxs: Tx[], blockProposalOptions: BlockProposalOptions, - inboxPrefixRef?: InboxMessagePrefixRef, + inboxPrefixRef: InboxMessagePrefixRef, ): Promise { if (this.config.fishermanMode) { this.log.info(`Skipping block proposal for block ${block.number} in fisherman mode`); @@ -1537,8 +1531,8 @@ export class CheckpointProposalJob implements Traceable { block.archive.root, usedTxs, this.proposer, - blockProposalOptions, inboxPrefixRef, + blockProposalOptions, ); } @@ -2320,10 +2314,7 @@ export class CheckpointProposalJob implements Traceable { * and fee analysis only, and pushing them to the archiver causes spurious reorg cascades * whenever the real proposer's block arrives from L1. */ - private async syncProposedBlockToArchiver( - block: L2Block, - inboxPrefixRef: InboxMessagePrefixRef | undefined, - ): Promise { + private async syncProposedBlockToArchiver(block: L2Block, inboxPrefixRef: InboxMessagePrefixRef): Promise { if (this.config.skipPushProposedBlocksToArchiver || this.config.fishermanMode) { this.log.warn(`Skipping push of proposed block ${block.number} to archiver`, { blockNumber: block.number, @@ -2332,11 +2323,7 @@ export class CheckpointProposalJob implements Traceable { return; } // The archiver re-validates this reference against its own messages inside the insert transaction, which is what - // stops a block built before an L1 reorg from landing after the reorg pruned the chain it belongs to. Streaming - // block building always produces one, so a missing reference here is a wiring bug, not a compatibility case. - if (inboxPrefixRef === undefined) { - throw new Error(`Streaming inbox: proposed block ${block.number} has no signed Inbox prefix reference`); - } + // stops a block built before an L1 reorg from landing after the reorg pruned the chain it belongs to. this.log.debug(`Syncing proposed block ${block.number} to archiver`, { blockNumber: block.number, slot: block.header.globalVariables.slotNumber, diff --git a/yarn-project/sequencer-client/src/test/mock_checkpoint_builder.ts b/yarn-project/sequencer-client/src/test/mock_checkpoint_builder.ts index 87370112724e..bca5a827897d 100644 --- a/yarn-project/sequencer-client/src/test/mock_checkpoint_builder.ts +++ b/yarn-project/sequencer-client/src/test/mock_checkpoint_builder.ts @@ -11,6 +11,7 @@ import type { ICheckpointsBuilder, MerkleTreeWriteOperations, } from '@aztec/stdlib/interfaces/server'; +import { accumulateInboxRollingHash } from '@aztec/stdlib/messaging'; import { CheckpointHeader } from '@aztec/stdlib/rollup'; import { makeAppendOnlyTreeSnapshot } from '@aztec/stdlib/testing'; import type { CheckpointGlobalVariables, Tx } from '@aztec/stdlib/tx'; @@ -43,10 +44,26 @@ export class MockCheckpointBuilder implements ICheckpointBlockBuilder { /** Set to an error to make buildBlock throw on next call */ public errorOnBuild: Error | undefined = undefined; + /** + * Rolling hash over every Inbox message consumed through the blocks built so far, starting from the parent + * checkpoint's. The real builder derives the checkpoint header's `inboxRollingHash` this way, and the last block's + * signed prefix reference has to equal it, so the mock tracks it rather than leaving the header's hash at zero. + */ + private inboxRollingHash: Fr; + constructor( private readonly constants: CheckpointGlobalVariables, private readonly checkpointNumber: CheckpointNumber, - ) {} + private previousInboxRollingHash: Fr = Fr.ZERO, + ) { + this.inboxRollingHash = previousInboxRollingHash; + } + + /** Sets the parent checkpoint's rolling hash this checkpoint accumulates from, and rewinds to it. */ + setPreviousInboxRollingHash(previousInboxRollingHash: Fr): void { + this.previousInboxRollingHash = previousInboxRollingHash; + this.inboxRollingHash = previousInboxRollingHash; + } /** Seed the builder with blocks to return on successive buildBlock calls */ seedBlocks(blocks: L2Block[], usedTxsPerBlock?: Tx[][]): this { @@ -83,6 +100,8 @@ export class MockCheckpointBuilder implements ICheckpointBlockBuilder { throw this.errorOnBuild; } + this.inboxRollingHash = accumulateInboxRollingHash(this.inboxRollingHash, opts.l1ToL2Messages ?? []); + let block: L2Block; let usedTxs: Tx[]; @@ -163,6 +182,7 @@ export class MockCheckpointBuilder implements ICheckpointBlockBuilder { const checkpointHeader = CheckpointHeader.empty({ lastArchiveRoot: firstBlock.header.lastArchive.root, blockHeadersHash: Fr.random(), + inboxRollingHash: this.inboxRollingHash, slotNumber: gv.slotNumber, timestamp: gv.timestamp, coinbase: gv.coinbase, @@ -183,6 +203,7 @@ export class MockCheckpointBuilder implements ICheckpointBlockBuilder { resetCheckpointState(): void { this.builtBlocks = []; this.blockIndex = 0; + this.inboxRollingHash = this.previousInboxRollingHash; this.consumedTxHashes.clear(); this.completeCheckpointCalled = false; this.getCheckpointCalled = false; @@ -269,7 +290,7 @@ export class MockCheckpointsBuilder implements ICheckpointsBuilder { constants: CheckpointGlobalVariables, feeAssetPriceModifier: bigint, previousCheckpointOutHashes: Fr[], - _previousInboxRollingHash: Fr, + previousInboxRollingHash: Fr, _fork: MerkleTreeWriteOperations, _bindings?: LoggerBindings, ): Promise { @@ -282,9 +303,10 @@ export class MockCheckpointsBuilder implements ICheckpointsBuilder { if (!this.checkpointBuilder) { // Auto-create a builder if none was set - this.checkpointBuilder = new MockCheckpointBuilder(constants, checkpointNumber); + this.checkpointBuilder = new MockCheckpointBuilder(constants, checkpointNumber, previousInboxRollingHash); } else { this.checkpointBuilder.resetCheckpointState(); + this.checkpointBuilder.setPreviousInboxRollingHash(previousInboxRollingHash); } return Promise.resolve(this.checkpointBuilder); diff --git a/yarn-project/sequencer-client/src/test/utils.ts b/yarn-project/sequencer-client/src/test/utils.ts index 880f6680d8d2..1f6d0f2f4b69 100644 --- a/yarn-project/sequencer-client/src/test/utils.ts +++ b/yarn-project/sequencer-client/src/test/utils.ts @@ -12,6 +12,7 @@ import { DEFAULT_BLOCK_DURATION_MS } from '@aztec/stdlib/config'; import type { L1RollupConstants } from '@aztec/stdlib/epoch-helpers'; import { type InboxMessagePosition, + InboxMessagePrefixRef, type InboxMessageRange, type L1ToL2MessageSource, accumulateInboxRollingHash, @@ -145,7 +146,11 @@ function createCheckpointHeaderFromBlock(block: L2Block): CheckpointHeader { /** * Creates a block proposal from a block and signature */ -export function createBlockProposal(block: L2Block, signature: Signature): BlockProposal { +export function createBlockProposal( + block: L2Block, + signature: Signature, + inboxPrefixRef: InboxMessagePrefixRef = InboxMessagePrefixRef.random(), +): BlockProposal { const txHashes = block.body.txEffects.map(tx => tx.txHash); return new BlockProposal( block.header, @@ -154,6 +159,7 @@ export function createBlockProposal(block: L2Block, signature: Signature): Block txHashes, signature, TEST_COORDINATION_SIGNATURE_CONTEXT, + inboxPrefixRef, ); } @@ -179,6 +185,7 @@ export function createCheckpointProposal( indexWithinCheckpoint: block.indexWithinCheckpoint, txHashes, signature: blockSignature ?? checkpointSignature, // Use checkpoint signature as block signature if not provided + inboxPrefixRef: new InboxMessagePrefixRef(checkpointHeader.inboxRollingHash), }, ); } diff --git a/yarn-project/stdlib/src/interfaces/validator.ts b/yarn-project/stdlib/src/interfaces/validator.ts index 784940c07140..972d971faecf 100644 --- a/yarn-project/stdlib/src/interfaces/validator.ts +++ b/yarn-project/stdlib/src/interfaces/validator.ts @@ -5,6 +5,7 @@ import type { EthAddress } from '@aztec/foundation/eth-address'; import type { Signature } from '@aztec/foundation/eth-signature'; import { schemas, zodFor } from '@aztec/foundation/schemas'; import type { SequencerConfig, SlasherConfig } from '@aztec/stdlib/interfaces/server'; +import type { InboxMessagePrefixRef } from '@aztec/stdlib/messaging'; import type { BlockProposal, BlockProposalOptions, @@ -158,6 +159,7 @@ export interface Validator { archive: Fr, txs: Tx[], proposerAddress: EthAddress | undefined, + inboxPrefixRef: InboxMessagePrefixRef, options: BlockProposalOptions, ): Promise; diff --git a/yarn-project/stdlib/src/p2p/block_proposal.test.ts b/yarn-project/stdlib/src/p2p/block_proposal.test.ts index 88aaa25b22ae..f794b3359b8a 100644 --- a/yarn-project/stdlib/src/p2p/block_proposal.test.ts +++ b/yarn-project/stdlib/src/p2p/block_proposal.test.ts @@ -13,13 +13,16 @@ import { TxHash } from '../tx/tx_hash.js'; import { BlockProposal } from './block_proposal.js'; import { EMPTY_COORDINATION_SIGNATURE_CONTEXT } from './signature_utils.js'; import { SignedTxs } from './signed_txs.js'; -import { LEGACY_BLOCK_PROPOSAL_HEX, LEGACY_BLOCK_PROPOSAL_PAYLOAD_HEX } from './wire_compat_fixtures.js'; +import { + BLOCK_PROPOSAL_HEX, + BLOCK_PROPOSAL_PAYLOAD_HEX, + PRE_REQUIRED_PREFIX_BLOCK_PROPOSAL_HEX, +} from './wire_compat_fixtures.js'; /** - * Deterministic legacy-shaped proposal (no signedTxs, no inboxPrefixRef) matching the golden fixtures in - * wire_compat_fixtures.ts. Constructed identically to how those bytes were captured on the pre-change code. + * Deterministic proposal (no signedTxs) matching the golden fixtures in wire_compat_fixtures.ts. */ -const makeLegacyFixtureProposal = () => +const makeFixtureProposal = () => new BlockProposal( BlockHeader.empty(), IndexWithinCheckpoint(3), @@ -27,6 +30,7 @@ const makeLegacyFixtureProposal = () => [TxHash.fromField(new Fr(7n)), TxHash.fromField(new Fr(8n))], Signature.empty(), EMPTY_COORDINATION_SIGNATURE_CONTEXT, + new InboxMessagePrefixRef(new Fr(42n)), ); describe('Block Proposal serialization / deserialization', () => { @@ -108,6 +112,7 @@ describe('Block Proposal serialization / deserialization', () => { proposal.txHashes, proposal.signature, proposal.signatureContext, + proposal.inboxPrefixRef, foreignSignedTxs, ); @@ -115,62 +120,83 @@ describe('Block Proposal serialization / deserialization', () => { }); describe('inbox prefix reference', () => { - it('round-trips with a inbox prefix reference set', async () => { - const inboxPrefixRef = InboxMessagePrefixRef.random(); - const proposal = await makeBlockProposal({ inboxPrefixRef }); + it.each([ + ['no signed txs', () => makeBlockProposal({ inboxPrefixRef: InboxMessagePrefixRef.random() })], + [ + 'signed txs', + async () => + makeBlockProposal({ + txs: await Promise.all([Tx.random(), Tx.random()]), + inboxPrefixRef: InboxMessagePrefixRef.random(), + }), + ], + ['the empty genesis prefix', () => makeBlockProposal({ inboxPrefixRef: InboxMessagePrefixRef.empty() })], + ])('round-trips and reports its own size with %s', async (_name, build) => { + const proposal = await build(); const deserialized = BlockProposal.fromBuffer(proposal.toBuffer()); - expect(deserialized.inboxPrefixRef).toBeDefined(); - expect(deserialized.inboxPrefixRef!.equals(inboxPrefixRef)).toBe(true); - expect(deserialized.getSize()).toEqual(proposal.getSize()); + expect(deserialized.inboxPrefixRef.equals(proposal.inboxPrefixRef)).toBe(true); expect(deserialized).toEqual(proposal); + expect(proposal.getSize()).toEqual(proposal.toBuffer().length); + expect(deserialized.getSize()).toEqual(proposal.getSize()); }); - it('round-trips with a inbox prefix reference set alongside signed txs', async () => { - const inboxPrefixRef = InboxMessagePrefixRef.random(); - const txs = await Promise.all([Tx.random(), Tx.random()]); - const proposal = await makeBlockProposal({ txs, inboxPrefixRef }); - - const deserialized = BlockProposal.fromBuffer(proposal.toBuffer()); + it('serializes to the pinned wire bytes', () => { + const proposal = makeFixtureProposal(); + expect(bufferToHex(proposal.toBuffer())).toEqual(BLOCK_PROPOSAL_HEX); + expect(bufferToHex(proposal.getPayloadToSign())).toEqual(BLOCK_PROPOSAL_PAYLOAD_HEX); + }); - expect(deserialized.inboxPrefixRef!.equals(inboxPrefixRef)).toBe(true); - expect(deserialized.txs?.length).toEqual(txs.length); - expect(deserialized).toEqual(proposal); + it('places the reference between the tx hashes and the signed-txs flag', async () => { + const inboxPrefixRef = InboxMessagePrefixRef.random(); + const proposal = await makeBlockProposal({ inboxPrefixRef }); + const buffer = proposal.toBuffer(); + + // The reference occupies the 32 bytes ending four bytes (the hasSignedTxs flag) before the end of a proposal + // that carries no bundle, so nothing but the flag separates it from the end of the buffer. + const flagOffset = buffer.length - 4; + const refOffset = flagOffset - InboxMessagePrefixRef.SIZE; + expect(buffer.subarray(refOffset, flagOffset)).toEqual(inboxPrefixRef.toBuffer()); + expect(buffer.readUInt32BE(flagOffset)).toEqual(0); + // Immediately before the reference are the last tx hash's bytes, so no other field was inserted between them. + const lastTxHash = proposal.txHashes[proposal.txHashes.length - 1]; + expect(buffer.subarray(refOffset - TxHash.SIZE, refOffset)).toEqual(lastTxHash.toBuffer()); }); - it('serializes byte-identically to the legacy format when unset', () => { - const proposal = makeLegacyFixtureProposal(); - expect(proposal.inboxPrefixRef).toBeUndefined(); - expect(bufferToHex(proposal.toBuffer())).toEqual(LEGACY_BLOCK_PROPOSAL_HEX); - expect(bufferToHex(proposal.getPayloadToSign())).toEqual(LEGACY_BLOCK_PROPOSAL_PAYLOAD_HEX); + it('rejects a buffer written without the required reference', () => { + expect(() => BlockProposal.fromBuffer(hexToBuffer(PRE_REQUIRED_PREFIX_BLOCK_PROPOSAL_HEX))).toThrow( + /beyond buffer length/, + ); }); - it('decodes a legacy buffer (no tail) as having no inbox prefix reference', () => { - const deserialized = BlockProposal.fromBuffer(hexToBuffer(LEGACY_BLOCK_PROPOSAL_HEX)); - expect(deserialized.inboxPrefixRef).toBeUndefined(); - // Re-encoding a legacy buffer yields the same legacy bytes: no phantom tail is introduced. - expect(bufferToHex(deserialized.toBuffer())).toEqual(LEGACY_BLOCK_PROPOSAL_HEX); + it('rejects a buffer whose reference is truncated', () => { + const complete = makeFixtureProposal().toBuffer(); + // Drop one byte of the reference: the flag and the reference together are four bytes longer than what is left. + const truncated = complete.subarray(0, complete.length - 5); + expect(() => BlockProposal.fromBuffer(truncated)).toThrow(/beyond buffer length/); }); - it('appends the inbox prefix reference only when set, changing the signed payload', async () => { - const inboxPrefixRef = InboxMessagePrefixRef.random(); - const withRef = await makeBlockProposal({ inboxPrefixRef }); - const withoutRef = await makeBlockProposal({ - blockHeader: withRef.blockHeader, - indexWithinCheckpoint: withRef.indexWithinCheckpoint, - archiveRoot: withRef.archiveRoot, - txHashes: withRef.txHashes, - }); - - const withRefPayload = withRef.getPayloadToSign(); - const withoutRefPayload = withoutRef.getPayloadToSign(); - - // The set payload extends the unset payload by exactly the reference bytes (appended tail, no marker). - expect(withRefPayload.length).toEqual(withoutRefPayload.length + InboxMessagePrefixRef.SIZE); - expect(withRefPayload.subarray(0, withoutRefPayload.length)).toEqual(withoutRefPayload); - // The payload hashes differ, so the attestation pool treats set-vs-unset as distinct payloads. - expect(withRef.getPayloadHash().toString()).not.toEqual(withoutRef.getPayloadHash().toString()); + it('signs a different payload for a different reference', async () => { + const shared = { + blockHeader: BlockHeader.empty(), + indexWithinCheckpoint: IndexWithinCheckpoint(1), + archiveRoot: new Fr(5n), + txHashes: [TxHash.fromField(new Fr(7n))], + }; + const first = await makeBlockProposal({ ...shared, inboxPrefixRef: new InboxMessagePrefixRef(new Fr(11n)) }); + const second = await makeBlockProposal({ ...shared, inboxPrefixRef: new InboxMessagePrefixRef(new Fr(12n)) }); + + const firstPayload = first.getPayloadToSign(); + const secondPayload = second.getPayloadToSign(); + + // The reference is the payload's tail, so the payloads agree up to it and differ only over its bytes. + expect(firstPayload.length).toEqual(secondPayload.length); + const prefixLength = firstPayload.length - InboxMessagePrefixRef.SIZE; + expect(firstPayload.subarray(0, prefixLength)).toEqual(secondPayload.subarray(0, prefixLength)); + expect(firstPayload.subarray(prefixLength)).toEqual(first.inboxPrefixRef.toBuffer()); + // Distinct payload hashes keep the attestation pool from treating the two as the same signed payload. + expect(first.getPayloadHash().toString()).not.toEqual(second.getPayloadHash().toString()); }); it('covers the inbox prefix reference under the proposal signature', async () => { @@ -196,29 +222,11 @@ describe('Block Proposal serialization / deserialization', () => { proposal.txHashes, proposal.signature, proposal.signatureContext, - proposal.signedTxs, new InboxMessagePrefixRef(new Fr(8n)), - ); - expect(tampered.getSender()).not.toEqual(signer.address); - }); - - it('breaks sender recovery when a inbox prefix reference is injected into an unsigned proposal', async () => { - const signer = Secp256k1Signer.random(); - const proposal = await makeBlockProposal({ signer }); - expect(proposal.getSender()).toEqual(signer.address); - - // A relay injecting a reference the proposer never signed over is rejected by signature recovery. - const injected = new BlockProposal( - proposal.blockHeader, - proposal.indexWithinCheckpoint, - proposal.archiveRoot, - proposal.txHashes, - proposal.signature, - proposal.signatureContext, proposal.signedTxs, - InboxMessagePrefixRef.random(), ); - expect(injected.getSender()).not.toEqual(signer.address); + expect(tampered.getSender()).not.toEqual(signer.address); + expect(tampered.getPayloadHash().toString()).not.toEqual(proposal.getPayloadHash().toString()); }); }); }); diff --git a/yarn-project/stdlib/src/p2p/block_proposal.ts b/yarn-project/stdlib/src/p2p/block_proposal.ts index 9170a0e59971..50f7aa2e7841 100644 --- a/yarn-project/stdlib/src/p2p/block_proposal.ts +++ b/yarn-project/stdlib/src/p2p/block_proposal.ts @@ -83,17 +83,21 @@ export class BlockProposal extends Gossipable implements Signable { /** The signing domain (chainId + rollupAddress) the signature is bound to */ public readonly signatureContext: CoordinationSignatureContext, - /** The signed transactions in the block (optional, for DA guarantees) */ - public readonly signedTxs?: SignedTxs, - /** * The signed Inbox message-prefix reference this block proposes to have consumed through: the rolling hash over * the first `blockHeader.state.l1ToL2MessageTree.nextAvailableLeafIndex` messages. Validators confirm the hash at * that count against their own Inbox view and read the consumed bundle from the resulting count range, rather * than trusting a proposer-supplied message list. The position may be interior to an L1 bucket. Covered by the * proposal signature (part of `getPayloadToSign`), so the count and the hash are signed as a pair. + * + * Every proposed block carries one, including a block that consumes no new messages: such a block re-states the + * prefix its parent ended at, so the pair (count, hash) is authenticated either way. The zero hash is a real + * value only for the empty global prefix a chain starts from. */ - public readonly inboxPrefixRef?: InboxMessagePrefixRef, + public readonly inboxPrefixRef: InboxMessagePrefixRef, + + /** The signed transactions in the block (optional, for DA guarantees) */ + public readonly signedTxs?: SignedTxs, ) { super(); } @@ -131,9 +135,9 @@ export class BlockProposal extends Gossipable implements Signable { /** * Get the payload to sign for this block proposal. - * The signature is over: blockHeader + indexWithinCheckpoint + archiveRoot + txHashes, plus the Inbox prefix - * reference when set. Appending only when set binds the reference to the signature so a relay cannot strip or - * inject it without breaking recovery. + * The signature is over: blockHeader + indexWithinCheckpoint + archiveRoot + txHashes + the Inbox prefix reference. + * The reference is always part of the payload, so a relay can neither strip nor substitute it without breaking + * signer recovery. */ getPayloadToSign(): Buffer { return serializeToBuffer([ @@ -142,7 +146,7 @@ export class BlockProposal extends Gossipable implements Signable { this.archiveRoot, this.txHashes.length, this.txHashes, - ...(this.inboxPrefixRef ? [this.inboxPrefixRef] : []), + this.inboxPrefixRef, ]); } @@ -169,9 +173,9 @@ export class BlockProposal extends Gossipable implements Signable { txHashes: TxHash[], txs: Tx[] | undefined, signatureContext: CoordinationSignatureContext, + inboxPrefixRef: InboxMessagePrefixRef, proposalSigner: (typedData: TypedDataDefinition, context: SigningContext) => Promise, txsSigner?: (typedData: TypedDataDefinition, context: SigningContext) => Promise, - inboxPrefixRef?: InboxMessagePrefixRef, ): Promise { // Create a temporary proposal to get the payload to sign const tempProposal = new BlockProposal( @@ -181,7 +185,6 @@ export class BlockProposal extends Gossipable implements Signable { txHashes, Signature.empty(), signatureContext, - undefined, inboxPrefixRef, ); @@ -216,8 +219,8 @@ export class BlockProposal extends Gossipable implements Signable { txHashes, sig, signatureContext, - signedTxs, inboxPrefixRef, + signedTxs, ); } @@ -263,6 +266,9 @@ export class BlockProposal extends Gossipable implements Signable { serializeCoordinationSignatureContext(this.signatureContext), this.txHashes.length, this.txHashes, + // The Inbox prefix reference sits ahead of the optional transaction bundle and carries no presence flag: a + // buffer that ends before its 32 bytes is malformed, not a proposal that consumed no messages. + this.inboxPrefixRef, ]; if (this.signedTxs) { buffer.push(1); // hasSignedTxs = true @@ -270,12 +276,6 @@ export class BlockProposal extends Gossipable implements Signable { } else { buffer.push(0); // hasSignedTxs = false } - // Optional Inbox prefix-reference tail. Appended only when set, so a proposal without a reference - // serializes without the tail and a decoder that reaches EOF reads it as unset. - if (this.inboxPrefixRef) { - buffer.push(1); // hasInboxPrefixRef = true - buffer.push(this.inboxPrefixRef.toBuffer()); - } return serializeToBuffer(buffer); } @@ -292,23 +292,12 @@ export class BlockProposal extends Gossipable implements Signable { throw new Error(`txHashes count ${txHashCount} exceeds maximum ${MAX_TXS_PER_BLOCK}`); } const txHashes = reader.readArray(txHashCount, TxHash); + const inboxPrefixRef = reader.readObject(InboxMessagePrefixRef); let signedTxs: SignedTxs | undefined; - if (!reader.isEmpty()) { - const hasSignedTxs = reader.readNumber(); - if (hasSignedTxs) { - signedTxs = SignedTxs.fromBuffer(reader); - } - } - - // Optional Inbox prefix-reference tail. A buffer that ends after the signedTxs flag decodes as - // "no reference", so proposals written without the tail round-trip cleanly. - let inboxPrefixRef: InboxMessagePrefixRef | undefined; - if (!reader.isEmpty()) { - const hasInboxPrefixRef = reader.readNumber(); - if (hasInboxPrefixRef) { - inboxPrefixRef = InboxMessagePrefixRef.fromBuffer(reader); - } + const hasSignedTxs = reader.readNumber(); + if (hasSignedTxs) { + signedTxs = SignedTxs.fromBuffer(reader); } return new BlockProposal( @@ -318,8 +307,8 @@ export class BlockProposal extends Gossipable implements Signable { txHashes, signature, signatureContext, - signedTxs, inboxPrefixRef, + signedTxs, ); } @@ -333,9 +322,9 @@ export class BlockProposal extends Gossipable implements Signable { 20 /* rollupAddress */ + 4 /* txHashes.length */ + this.txHashes.length * TxHash.SIZE + + this.inboxPrefixRef.getSize() + 4 /* hasSignedTxs flag */ + - (this.signedTxs ? this.signedTxs.getSize() : 0) + - (this.inboxPrefixRef ? 4 /* hasInboxPrefixRef flag */ + this.inboxPrefixRef.getSize() : 0) + (this.signedTxs ? this.signedTxs.getSize() : 0) ); } @@ -347,6 +336,7 @@ export class BlockProposal extends Gossipable implements Signable { [], Signature.empty(), EMPTY_COORDINATION_SIGNATURE_CONTEXT, + InboxMessagePrefixRef.empty(), ); } @@ -358,6 +348,7 @@ export class BlockProposal extends Gossipable implements Signable { [TxHash.random(), TxHash.random()], Signature.random(), EMPTY_COORDINATION_SIGNATURE_CONTEXT, + InboxMessagePrefixRef.random(), ); } @@ -370,7 +361,7 @@ export class BlockProposal extends Gossipable implements Signable { txHashes: this.txHashes.map(h => h.toString()), chainId: this.signatureContext.chainId, rollupAddress: this.signatureContext.rollupAddress.toString(), - inboxPrefixRef: this.inboxPrefixRef?.toInspect(), + inboxPrefixRef: this.inboxPrefixRef.toInspect(), }; } @@ -396,8 +387,8 @@ export class BlockProposal extends Gossipable implements Signable { this.txHashes, this.signature, this.signatureContext, - undefined, this.inboxPrefixRef, + undefined, ); } } diff --git a/yarn-project/stdlib/src/p2p/checkpoint_proposal.test.ts b/yarn-project/stdlib/src/p2p/checkpoint_proposal.test.ts index 24f9e223d9f1..d019c73121bd 100644 --- a/yarn-project/stdlib/src/p2p/checkpoint_proposal.test.ts +++ b/yarn-project/stdlib/src/p2p/checkpoint_proposal.test.ts @@ -10,15 +10,17 @@ import { CheckpointHeader } from '../rollup/checkpoint_header.js'; import { makeCheckpointProposal, mockTx } from '../tests/mocks.js'; import { BlockHeader } from '../tx/block_header.js'; import { TxHash } from '../tx/tx_hash.js'; +import { BlockProposal } from './block_proposal.js'; import { CheckpointProposal } from './checkpoint_proposal.js'; import { EMPTY_COORDINATION_SIGNATURE_CONTEXT } from './signature_utils.js'; -import { LEGACY_CHECKPOINT_PROPOSAL_HEX } from './wire_compat_fixtures.js'; +import { CHECKPOINT_PROPOSAL_HEX, PRE_REQUIRED_PREFIX_CHECKPOINT_PROPOSAL_HEX } from './wire_compat_fixtures.js'; /** - * Deterministic legacy-shaped checkpoint proposal (lastBlock without signedTxs or inboxPrefixRef) matching the golden - * fixture in wire_compat_fixtures.ts. Constructed identically to how those bytes were captured on the pre-change code. + * Deterministic checkpoint proposal (lastBlock without signedTxs) matching the golden fixture in + * wire_compat_fixtures.ts. An empty checkpoint header's rolling hash is zero, so the last block's reference is the + * empty prefix rather than an arbitrary value. */ -const makeLegacyFixtureCheckpointProposal = () => +const makeFixtureCheckpointProposal = () => new CheckpointProposal( CheckpointHeader.empty(), new Fr(123n), @@ -30,6 +32,7 @@ const makeLegacyFixtureCheckpointProposal = () => indexWithinCheckpoint: IndexWithinCheckpoint(4), txHashes: [TxHash.fromField(new Fr(7n))], signature: Signature.empty(), + inboxPrefixRef: InboxMessagePrefixRef.empty(), }, ); @@ -82,17 +85,37 @@ describe('CheckpointProposal serialization / deserialization', () => { expect(deserialized.toBuffer()).toEqual(proposal.toBuffer()); }); - it('serializes byte-identically to the legacy format when unset', () => { - const proposal = makeLegacyFixtureCheckpointProposal(); - expect(proposal.lastBlock?.inboxPrefixRef).toBeUndefined(); - expect(bufferToHex(proposal.toBuffer())).toEqual(LEGACY_CHECKPOINT_PROPOSAL_HEX); + it('serializes to the pinned wire bytes', () => { + const proposal = makeFixtureCheckpointProposal(); + expect(bufferToHex(proposal.toBuffer())).toEqual(CHECKPOINT_PROPOSAL_HEX); }); - it('decodes a legacy buffer (no tail) as having no inbox prefix reference', () => { - const deserialized = CheckpointProposal.fromBuffer(hexToBuffer(LEGACY_CHECKPOINT_PROPOSAL_HEX)); - expect(deserialized.lastBlock).toBeDefined(); - expect(deserialized.lastBlock?.inboxPrefixRef).toBeUndefined(); - expect(bufferToHex(deserialized.toBuffer())).toEqual(LEGACY_CHECKPOINT_PROPOSAL_HEX); + it('places the last block reference between its tx hashes and its signed-txs flag', async () => { + const checkpointHeader = CheckpointHeader.random(); + const inboxPrefixRef = new InboxMessagePrefixRef(checkpointHeader.inboxRollingHash); + const proposal = await makeCheckpointProposal({ + checkpointHeader, + lastBlock: { txHashes: [TxHash.fromField(new Fr(7n))], inboxPrefixRef }, + }); + const buffer = proposal.toBuffer(); + + const flagOffset = buffer.length - 4; + const refOffset = flagOffset - InboxMessagePrefixRef.SIZE; + expect(buffer.subarray(refOffset, flagOffset)).toEqual(inboxPrefixRef.toBuffer()); + expect(buffer.readUInt32BE(flagOffset)).toEqual(0); + expect(buffer.subarray(refOffset - TxHash.SIZE, refOffset)).toEqual(TxHash.fromField(new Fr(7n)).toBuffer()); + }); + + it('rejects a buffer written without the required reference', () => { + expect(() => CheckpointProposal.fromBuffer(hexToBuffer(PRE_REQUIRED_PREFIX_CHECKPOINT_PROPOSAL_HEX))).toThrow( + /beyond buffer length/, + ); + }); + + it('rejects a buffer whose last block reference is truncated', () => { + const complete = makeFixtureCheckpointProposal().toBuffer(); + const truncated = complete.subarray(0, complete.length - 5); + expect(() => CheckpointProposal.fromBuffer(truncated)).toThrow(/beyond buffer length/); }); it('carries the inbox prefix reference through getBlockProposal, covered by the block signature', async () => { @@ -102,11 +125,40 @@ describe('CheckpointProposal serialization / deserialization', () => { const proposal = await makeCheckpointProposal({ signer, checkpointHeader, lastBlock: { inboxPrefixRef } }); const blockProposal = proposal.getBlockProposal(); - expect(blockProposal?.inboxPrefixRef?.equals(inboxPrefixRef)).toBe(true); + expect(blockProposal?.inboxPrefixRef.equals(inboxPrefixRef)).toBe(true); expect(blockProposal?.getSender()).toEqual(signer.address); expect(proposal.getSender()).toEqual(signer.address); }); + it('recovers the same signed block identity from the embedded block as from a standalone one', async () => { + const signer = Secp256k1Signer.random(); + const checkpointHeader = CheckpointHeader.random(); + const inboxPrefixRef = new InboxMessagePrefixRef(checkpointHeader.inboxRollingHash); + const proposal = await makeCheckpointProposal({ signer, checkpointHeader, lastBlock: { inboxPrefixRef } }); + + const embedded = proposal.getBlockProposal()!; + const roundTripped = CheckpointProposal.fromBuffer(proposal.toBuffer()).getBlockProposal()!; + const standalone = BlockProposal.fromBuffer(embedded.toBuffer()); + + expect(roundTripped.getPayloadHash().toString()).toEqual(embedded.getPayloadHash().toString()); + expect(standalone.getPayloadHash().toString()).toEqual(embedded.getPayloadHash().toString()); + expect(roundTripped.getSender()).toEqual(signer.address); + expect(standalone.toBuffer()).toEqual(roundTripped.toBuffer()); + }); + + it('rejects a well-formed buffer whose last block reference disagrees with the checkpoint header', async () => { + const checkpointHeader = CheckpointHeader.random(); + const inboxPrefixRef = new InboxMessagePrefixRef(checkpointHeader.inboxRollingHash); + const proposal = await makeCheckpointProposal({ checkpointHeader, lastBlock: { inboxPrefixRef } }); + + // Overwrite the reference in place, leaving every length and every other field intact. + const tampered = Buffer.from(proposal.toBuffer()); + const refOffset = tampered.length - 4 - InboxMessagePrefixRef.SIZE; + new InboxMessagePrefixRef(new Fr(0xdeadn)).toBuffer().copy(tampered, refOffset); + + expect(() => CheckpointProposal.fromBuffer(tampered)).toThrow(/inboxPrefixRef rolling hash/); + }); + it('accepts a last-block reference whose rolling hash matches the checkpoint header', () => { const checkpointHeader = CheckpointHeader.random({ inboxRollingHash: new Fr(0x1234n) }); expect( diff --git a/yarn-project/stdlib/src/p2p/checkpoint_proposal.ts b/yarn-project/stdlib/src/p2p/checkpoint_proposal.ts index 8ca8d697c0a6..9eae4a8c0a15 100644 --- a/yarn-project/stdlib/src/p2p/checkpoint_proposal.ts +++ b/yarn-project/stdlib/src/p2p/checkpoint_proposal.ts @@ -71,12 +71,12 @@ export type CheckpointLastBlock = Omit & { /** The signed transactions in the last block (optional, for DA guarantees) */ signedTxs?: SignedTxs; /** - * The signed Inbox message-prefix reference the last block proposes to have consumed through. When set it must - * equal the checkpoint header's `inboxRollingHash` (enforced at construction), which makes the last block's position - * the checkpoint's own: unlike an intermediate block's, that position must resolve to a live L1 bucket at - * publication, because L1 reads the header's hash out of the bucket the `propose` hint names. + * The signed Inbox message-prefix reference the last block proposes to have consumed through. It must equal the + * checkpoint header's `inboxRollingHash` (enforced at construction), which makes the last block's position the + * checkpoint's own: unlike an intermediate block's, that position must resolve to a live L1 bucket at publication, + * because L1 reads the header's hash out of the bucket the `propose` hint names. */ - inboxPrefixRef?: InboxMessagePrefixRef; + inboxPrefixRef: InboxMessagePrefixRef; }; /** @@ -114,11 +114,8 @@ export class CheckpointProposal extends Gossipable implements Signable { // Check that last block properties match those of the checkpoint. The last block's Inbox prefix reference // commits to the same rolling hash as the checkpoint header, so the checkpoint's consumed position is exactly - // its last block's. Only enforced when the reference is set. - if ( - lastBlock?.inboxPrefixRef && - !lastBlock.inboxPrefixRef.inboxRollingHash.equals(checkpointHeader.inboxRollingHash) - ) { + // its last block's. + if (lastBlock && !lastBlock.inboxPrefixRef.inboxRollingHash.equals(checkpointHeader.inboxRollingHash)) { throw new Error( `CheckpointProposal lastBlock inboxPrefixRef rolling hash ${lastBlock.inboxPrefixRef.inboxRollingHash} does not match checkpoint inboxRollingHash ${checkpointHeader.inboxRollingHash}`, ); @@ -160,8 +157,8 @@ export class CheckpointProposal extends Gossipable implements Signable { this.lastBlock.txHashes, this.lastBlock.signature, this.signatureContext, - this.lastBlock.signedTxs, this.lastBlock.inboxPrefixRef, + this.lastBlock.signedTxs, ); } @@ -294,18 +291,14 @@ export class CheckpointProposal extends Gossipable implements Signable { buffer.push(this.lastBlock.signature); buffer.push(this.lastBlock.txHashes.length); buffer.push(this.lastBlock.txHashes); + // Same layout as a standalone block proposal: the required reference precedes the optional transaction bundle. + buffer.push(this.lastBlock.inboxPrefixRef); if (this.lastBlock.signedTxs) { buffer.push(1); // hasSignedTxs = true buffer.push(this.lastBlock.signedTxs.toBuffer()); } else { buffer.push(0); // hasSignedTxs = false } - // Optional Inbox prefix-reference tail. Appended only when set, so a proposal without a reference - // serializes without the tail and a decoder that reaches EOF reads it as unset. - if (this.lastBlock.inboxPrefixRef) { - buffer.push(1); // hasInboxPrefixRef = true - buffer.push(this.lastBlock.inboxPrefixRef.toBuffer()); - } } else { buffer.push(0); // hasLastBlock = false } @@ -333,23 +326,12 @@ export class CheckpointProposal extends Gossipable implements Signable { throw new Error(`txHashes count ${txHashCount} exceeds maximum ${MAX_TXS_PER_BLOCK}`); } const txHashes = reader.readArray(txHashCount, TxHash); + const inboxPrefixRef = reader.readObject(InboxMessagePrefixRef); let signedTxs: SignedTxs | undefined; - if (!reader.isEmpty()) { - const hasSignedTxs = reader.readNumber(); - if (hasSignedTxs) { - signedTxs = SignedTxs.fromBuffer(reader); - } - } - - // Optional Inbox prefix-reference tail. A buffer that ends after the signedTxs flag decodes as - // "no reference", so proposals written without the tail round-trip cleanly. - let inboxPrefixRef: InboxMessagePrefixRef | undefined; - if (!reader.isEmpty()) { - const hasInboxPrefixRef = reader.readNumber(); - if (hasInboxPrefixRef) { - inboxPrefixRef = InboxMessagePrefixRef.fromBuffer(reader); - } + const hasSignedTxs = reader.readNumber(); + if (hasSignedTxs) { + signedTxs = SignedTxs.fromBuffer(reader); } return new CheckpointProposal(checkpointHeader, archive, feeAssetPriceModifier, signature, signatureContext, { @@ -384,8 +366,9 @@ export class CheckpointProposal extends Gossipable implements Signable { } static random(): CheckpointProposal { + const checkpointHeader = CheckpointHeader.random(); return new CheckpointProposal( - CheckpointHeader.random(), + checkpointHeader, Fr.random(), 0n, Signature.random(), @@ -395,6 +378,7 @@ export class CheckpointProposal extends Gossipable implements Signable { indexWithinCheckpoint: IndexWithinCheckpoint(Math.floor(Math.random() * 5)), txHashes: [TxHash.random(), TxHash.random()], signature: Signature.random(), + inboxPrefixRef: new InboxMessagePrefixRef(checkpointHeader.inboxRollingHash), }, ); } @@ -413,7 +397,7 @@ export class CheckpointProposal extends Gossipable implements Signable { indexWithinCheckpoint: this.lastBlock.indexWithinCheckpoint, txHashes: this.lastBlock.txHashes.map(h => h.toString()), signature: this.lastBlock.signature.toString(), - inboxPrefixRef: this.lastBlock.inboxPrefixRef?.toInspect(), + inboxPrefixRef: this.lastBlock.inboxPrefixRef.toInspect(), } : undefined, }; diff --git a/yarn-project/stdlib/src/p2p/wire_compat_fixtures.ts b/yarn-project/stdlib/src/p2p/wire_compat_fixtures.ts index 0c884c56e573..08ef789d9fd4 100644 --- a/yarn-project/stdlib/src/p2p/wire_compat_fixtures.ts +++ b/yarn-project/stdlib/src/p2p/wire_compat_fixtures.ts @@ -1,23 +1,40 @@ -// Golden wire fixtures that pin the exact bytes a peer produces and consumes, so the optional bucket-reference tail -// added to proposals stays wire compatible: an unset proposal must serialize to these bytes, and decoding these bytes -// must yield no Inbox prefix reference. The block-proposal fixtures were refreshed for the removal of the legacy `inHash` -// field; the checkpoint-proposal fixture was refreshed for the checkpoint-header format change. +// Golden wire fixtures that pin the exact bytes a peer produces and consumes for proposals carrying the required +// Inbox message-prefix reference, plus the pre-change bytes that must now be rejected. The reference sits after the +// transaction hashes and before the optional transaction bundle, with no presence flag, so a buffer written by a peer +// on the older optional-tail encoding is 32 bytes short of a valid proposal and fails to decode. // -// Both fixtures come from a deterministic proposal built with: +// Every fixture comes from a deterministic proposal built with: // BlockHeader.empty(), IndexWithinCheckpoint(3), archiveRoot=Fr(99), -// txHashes=[TxHash.fromField(Fr(7)), TxHash.fromField(Fr(8))], Signature.empty(), EMPTY_COORDINATION_SIGNATURE_CONTEXT +// txHashes=[TxHash.fromField(Fr(7)), TxHash.fromField(Fr(8))], Signature.empty(), +// EMPTY_COORDINATION_SIGNATURE_CONTEXT, inboxPrefixRef=InboxMessagePrefixRef(Fr(42)) // (checkpoint: CheckpointHeader.empty(), archive=Fr(123), feeAssetPriceModifier=0, empty signature/context, and a -// lastBlock with BlockHeader.empty(), IndexWithinCheckpoint(4), txHashes=[TxHash.fromField(Fr(7))], empty signature). -// Regenerate the checkpoint fixture only when the checkpoint serialization format changes on purpose. +// lastBlock with BlockHeader.empty(), IndexWithinCheckpoint(4), txHashes=[TxHash.fromField(Fr(7))], empty signature +// and the zero prefix reference the empty checkpoint header's rolling hash requires). +// Regenerate only when a serialization format changes on purpose. -/** Legacy `BlockProposal.toBuffer()` bytes (no signedTxs, no Inbox prefix reference). */ -export const LEGACY_BLOCK_PROPOSAL_HEX = +/** `BlockProposal.toBuffer()` bytes: required prefix reference, no signedTxs. */ +export const BLOCK_PROPOSAL_HEX = + '0x000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000003000000000000000000000000000000000000000000000000000000000000006300000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000000000000000070000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000000000000002a00000000'; + +/** `BlockProposal.getPayloadToSign()` bytes: the reference is always part of the signed payload. */ +export const BLOCK_PROPOSAL_PAYLOAD_HEX = + '0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000300000000000000000000000000000000000000000000000000000000000000630000000200000000000000000000000000000000000000000000000000000000000000070000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000000000000002a'; + +/** `CheckpointProposal.toBuffer()` bytes: lastBlock with the required prefix reference and no signedTxs. */ +export const CHECKPOINT_PROPOSAL_HEX = + '0x000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000007b00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000040000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000007000000000000000000000000000000000000000000000000000000000000000000000000'; + +/** + * `BlockProposal.toBuffer()` bytes as written before the reference became a required field: the same proposal without + * the 32-byte reference, ending on the `hasSignedTxs` flag. Decoding must reject it rather than read the flag as a + * reference. + */ +export const PRE_REQUIRED_PREFIX_BLOCK_PROPOSAL_HEX = '0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000300000000000000000000000000000000000000000000000000000000000000630000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000020000000000000000000000000000000000000000000000000000000000000007000000000000000000000000000000000000000000000000000000000000000800000000'; -/** Legacy `CheckpointProposal.toBuffer()` bytes (lastBlock without signedTxs or Inbox prefix reference). */ -export const LEGACY_CHECKPOINT_PROPOSAL_HEX = +/** + * `CheckpointProposal.toBuffer()` bytes as written before the reference became a required field, for the same + * proposal. Decoding must reject it. + */ +export const PRE_REQUIRED_PREFIX_CHECKPOINT_PROPOSAL_HEX = '0x000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000007b0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000004000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000700000000'; - -/** Legacy `BlockProposal.getPayloadToSign()` bytes (no Inbox prefix reference). */ -export const LEGACY_BLOCK_PROPOSAL_PAYLOAD_HEX = - '0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000300000000000000000000000000000000000000000000000000000000000000630000000200000000000000000000000000000000000000000000000000000000000000070000000000000000000000000000000000000000000000000000000000000008'; diff --git a/yarn-project/stdlib/src/tests/mocks.ts b/yarn-project/stdlib/src/tests/mocks.ts index 6c1171aca27b..3209203db02a 100644 --- a/yarn-project/stdlib/src/tests/mocks.ts +++ b/yarn-project/stdlib/src/tests/mocks.ts @@ -599,7 +599,7 @@ export const makeBlockProposal = (options?: MakeBlockProposalOptions): Promise Promise.resolve(signTypedData(signer, typedData)), (typedData, _context) => Promise.resolve(signTypedData(signer, typedData)), - inboxPrefixRef, ); }; @@ -632,7 +632,10 @@ export const makeCheckpointProposal = async (options?: MakeCheckpointProposalOpt txs: options.lastBlock.txs, signer, signatureContext, - inboxPrefixRef: options.lastBlock.inboxPrefixRef, + // The checkpoint constructor requires the last block's reference to equal the header's rolling hash, so the + // default follows the header rather than being random like a standalone proposal's. + inboxPrefixRef: + options.lastBlock.inboxPrefixRef ?? new InboxMessagePrefixRef(checkpointHeader.inboxRollingHash), }) : undefined; diff --git a/yarn-project/validator-client/README.md b/yarn-project/validator-client/README.md index db1e3ad1d18a..e18ccc45761e 100644 --- a/yarn-project/validator-client/README.md +++ b/yarn-project/validator-client/README.md @@ -21,10 +21,10 @@ A `BlockProposal` is broadcast by the proposer for each block **except the last BlockProposal { blockHeader // Per-block header with global variables indexWithinCheckpoint // 0, 1, 2, ... position within checkpoint - inboxPrefixRef // Rolling hash of the Inbox message prefix the block consumed through (with the header's leaf count) archive // Archive root after this block txHashes // Transaction hashes in order signature // Proposer's signature + inboxPrefixRef // Rolling hash of the Inbox message prefix the block consumed through (with the header's leaf count) signedTxs? // Optional full transactions for DA } ``` @@ -45,11 +45,14 @@ CheckpointProposal { indexWithinCheckpoint txHashes signature + inboxPrefixRef // Must equal the checkpointHeader's inboxRollingHash signedTxs? } } ``` +`inboxPrefixRef` is a required field on every proposed block, including one that consumes no new messages: such a block re-states the prefix its parent ended at. It is serialized without a presence flag, immediately before the optional transaction bundle, and is always part of the signed payload, so a proposal that omits or truncates it fails to decode rather than reading as a block that consumed nothing. + The `checkpointHeader` contains aggregated data: `blockHeadersHash` (hash of all block headers), `contentCommitment` (blobsHash, inboxRollingHash, outHash), and shared global variables. ### Checkpoint Attestations diff --git a/yarn-project/validator-client/src/duties/validation_service.test.ts b/yarn-project/validator-client/src/duties/validation_service.test.ts index b631709247e5..e92dec7564bc 100644 --- a/yarn-project/validator-client/src/duties/validation_service.test.ts +++ b/yarn-project/validator-client/src/duties/validation_service.test.ts @@ -3,6 +3,7 @@ import { CheckpointNumber, IndexWithinCheckpoint } from '@aztec/foundation/brand import { Buffer32 } from '@aztec/foundation/buffer'; import { Fr } from '@aztec/foundation/curves/bn254'; import { EthAddress } from '@aztec/foundation/eth-address'; +import { InboxMessagePrefixRef } from '@aztec/stdlib/messaging'; import { TEST_COORDINATION_SIGNATURE_CONTEXT, makeBlockHeader, @@ -44,6 +45,7 @@ describe('ValidationService', () => { txs, addresses[0], { publishFullTxs: true }, + InboxMessagePrefixRef.random(), ); expect(proposal.getSender()).toEqual(store.getAddress(0)); expect(proposal.txs).toBeDefined(); @@ -64,6 +66,7 @@ describe('ValidationService', () => { txs, addresses[0], { publishFullTxs: false }, + InboxMessagePrefixRef.random(), ); expect(proposal.getSender()).toEqual(addresses[0]); expect(proposal.txs).toBeUndefined(); @@ -95,6 +98,7 @@ describe('ValidationService', () => { txs, addresses[0], { publishFullTxs: true }, + new InboxMessagePrefixRef(checkpointHeader.inboxRollingHash), ); // Create a spy keystore to capture signing contexts diff --git a/yarn-project/validator-client/src/duties/validation_service.ts b/yarn-project/validator-client/src/duties/validation_service.ts index 5dad33bd60a4..85fa71ff0e90 100644 --- a/yarn-project/validator-client/src/duties/validation_service.ts +++ b/yarn-project/validator-client/src/duties/validation_service.ts @@ -39,6 +39,7 @@ export class ValidationService { * @param txs - Ordered list of transactions (Tx[]) * @param proposerAttesterAddress - The address of the proposer/attester, or undefined * @param options - Block proposal options (including broadcastInvalidBlockProposal for testing) + * @param inboxPrefixRef - Rolling hash of the Inbox message prefix the block consumed through * * @returns A block proposal signing the above information * @throws DutyAlreadySignedError if HA signer indicates duty already signed by another node @@ -52,7 +53,7 @@ export class ValidationService { txs: Tx[], proposerAttesterAddress: EthAddress | undefined, options: BlockProposalOptions, - inboxPrefixRef?: InboxMessagePrefixRef, + inboxPrefixRef: InboxMessagePrefixRef, ): Promise { // For testing: change the new archive to trigger state_mismatch validation failure if (options.broadcastInvalidBlockProposal) { @@ -79,9 +80,9 @@ export class ValidationService { txs.map(tx => tx.getTxHash()), options.publishFullTxs ? txs : undefined, this.signatureContext, + inboxPrefixRef, payloadSigner, txsSigner, - inboxPrefixRef, ); } diff --git a/yarn-project/validator-client/src/proposal_handler.test.ts b/yarn-project/validator-client/src/proposal_handler.test.ts index b10d77b318a4..c76a648bdce5 100644 --- a/yarn-project/validator-client/src/proposal_handler.test.ts +++ b/yarn-project/validator-client/src/proposal_handler.test.ts @@ -1811,10 +1811,7 @@ describe('ProposalHandler checkpoint validation', () => { } /** Genesis-parent block proposal at slot 1 consuming two messages, with the handler wired to reach the checks. */ - async function setupStreamingProposal( - inboxPrefixRef: InboxMessagePrefixRef | undefined, - options: { nowMs?: number } = {}, - ) { + async function setupStreamingProposal(inboxPrefixRef: InboxMessagePrefixRef, options: { nowMs?: number } = {}) { const blockHeader = makeBlockHeader(1, { slotNumber: SlotNumber(1) }); blockHeader.state.l1ToL2MessageTree.nextAvailableLeafIndex = 2; const proposal = ValidatedBlockProposal( @@ -1877,16 +1874,6 @@ describe('ProposalHandler checkpoint validation', () => { expect(reexecuteSpy).not.toHaveBeenCalled(); }); - it('rejects without collecting txs when the proposal carries no prefix reference', async () => { - const { proposal, blockHandler, txProvider } = await setupStreamingProposal(undefined); - mockLocalView(prefixHash); - - const result = await blockHandler.handleBlockProposal(proposal, {} as any, true); - - expect(result).toEqual(rejection('inbox_prefix_unavailable')); - expect(txProvider.getTxsForBlockProposal).not.toHaveBeenCalled(); - }); - it('re-executes with the bundle read by count when the checks pass, and inserts it with the signed reference', async () => { const { proposal, blockHandler, txProvider } = await setupStreamingProposal(signedRef); mockLocalView(prefixHash); @@ -1995,16 +1982,6 @@ describe('ProposalHandler checkpoint validation', () => { expect(blockSource.syncImmediate).not.toHaveBeenCalled(); }); - it('rejects immediately without syncing when the proposal carries no prefix reference', async () => { - const { proposal, blockHandler } = await setupStreamingProposal(undefined, { nowMs: BEFORE_DEADLINE_MS }); - mockLocalView(prefixHash); - - const result = await blockHandler.handleBlockProposal(proposal, {} as any, true); - - expect(result).toEqual(rejection('inbox_prefix_unavailable')); - expect(blockSource.syncImmediate).not.toHaveBeenCalled(); - }); - // This node may be the stale side of an L1 reorg holding a present-but-noncanonical hash at the signed count, // so a mismatch is retried exactly like a missing prefix and stays nonpunitive when it persists. it('rejects a mismatch that survives the deadline as a mismatch, after retrying', async () => { diff --git a/yarn-project/validator-client/src/proposal_handler.ts b/yarn-project/validator-client/src/proposal_handler.ts index 2f4d1f6ceeb4..b304ae4e4fe6 100644 --- a/yarn-project/validator-client/src/proposal_handler.ts +++ b/yarn-project/validator-client/src/proposal_handler.ts @@ -754,7 +754,7 @@ export class ProposalHandler { if (!streamingMetadata.accepted) { this.log.warn(`Streaming Inbox block acceptance check failed, skipping processing`, { reason: streamingMetadata.reason, - inboxPrefixRef: proposal.inboxPrefixRef?.toInspect(), + inboxPrefixRef: proposal.inboxPrefixRef.toInspect(), ...proposalInfo, }); return { isValid: false, blockNumber, reason: streamingMetadata.reason }; @@ -788,7 +788,7 @@ export class ProposalHandler { if (!bundle.accepted) { this.log.warn(`Streaming Inbox bundle read failed, skipping processing`, { reason: bundle.reason, - inboxPrefixRef: proposal.inboxPrefixRef?.toInspect(), + inboxPrefixRef: proposal.inboxPrefixRef.toInspect(), ...proposalInfo, }); return { isValid: false, blockNumber, reason: bundle.reason }; @@ -1236,11 +1236,11 @@ export class ProposalHandler { }); return 'inbox_prefix_unavailable'; } - if (!current.rollingHash.equals(proposal.inboxPrefixRef!.inboxRollingHash)) { + if (!current.rollingHash.equals(proposal.inboxPrefixRef.inboxRollingHash)) { this.log.warn(`Re-execution mismatch while the local Inbox prefix changed, not attributing it`, { ...proposalInfo, endTotalMsgCount: streamingMetadata.endTotalMsgCount, - signed: proposal.inboxPrefixRef!.inboxRollingHash.toString(), + signed: proposal.inboxPrefixRef.inboxRollingHash.toString(), local: current.rollingHash.toString(), }); return 'inbox_prefix_mismatch'; @@ -1330,13 +1330,12 @@ export class ProposalHandler { proposalInfo: LogData, ): Promise { const first = await this.checkStreamingBlockMetadata(proposal, blockNumber, parentBlock); - const inboxPrefixRef = proposal.inboxPrefixRef; - if (first.accepted || inboxPrefixRef === undefined || !isRetryableStreamingBlockCheckReason(first.reason)) { + if (first.accepted || !isRetryableStreamingBlockCheckReason(first.reason)) { return first; } const slotNumber = proposal.slotNumber; - const inboxRollingHash = inboxPrefixRef.inboxRollingHash.toString(); + const inboxRollingHash = proposal.inboxPrefixRef.inboxRollingHash.toString(); this.log.info(`Referenced Inbox prefix ${inboxRollingHash} unconfirmed locally, awaiting archiver sync`, { reason: first.reason, inboxRollingHash, diff --git a/yarn-project/validator-client/src/streaming_inbox_checks.test.ts b/yarn-project/validator-client/src/streaming_inbox_checks.test.ts index 6da435e675cf..4399449baf2a 100644 --- a/yarn-project/validator-client/src/streaming_inbox_checks.test.ts +++ b/yarn-project/validator-client/src/streaming_inbox_checks.test.ts @@ -69,7 +69,7 @@ class FakeInboxView implements StreamingInboxMessageSource { function baseInput(overrides: Partial): StreamingBlockCheckInput { return { messageSource: new FakeInboxView(), - inboxPrefixRef: undefined, + inboxPrefixRef: InboxMessagePrefixRef.empty(), endTotalMsgCount: 0n, parentTotalMsgCount: 0n, checkpointStartTotalMsgCount: 0n, @@ -80,14 +80,7 @@ function baseInput(overrides: Partial): StreamingBlock } describe('checkStreamingBlockProposal', () => { - describe('check 1: reference present', () => { - it('rejects a proposal with no prefix reference as unavailable', async () => { - const result = await checkStreamingBlockProposal(baseInput({ inboxPrefixRef: undefined })); - expect(result).toEqual({ accepted: false, reason: 'inbox_prefix_unavailable' }); - }); - }); - - describe('check 2: consumption moves forward', () => { + describe('check 1: consumption moves forward', () => { it('rejects an end count behind the parent block', async () => { const view = new FakeInboxView(); const end = view.append(3); @@ -103,7 +96,7 @@ describe('checkStreamingBlockProposal', () => { }); }); - describe('check 3: caps', () => { + describe('check 2: caps', () => { it('rejects a bundle over the per-block cap', async () => { const view = new FakeInboxView(); const end = view.append(PER_BLOCK_CAP + 1); @@ -133,7 +126,7 @@ describe('checkStreamingBlockProposal', () => { }); }); - describe('check 4: prefix hash at the signed count', () => { + describe('check 3: prefix hash at the signed count', () => { it('rejects as unavailable when the local view has not synced the end count', async () => { const view = new FakeInboxView(); view.append(2); diff --git a/yarn-project/validator-client/src/streaming_inbox_checks.ts b/yarn-project/validator-client/src/streaming_inbox_checks.ts index f7d98f9f69e9..f0e8ddda15c7 100644 --- a/yarn-project/validator-client/src/streaming_inbox_checks.ts +++ b/yarn-project/validator-client/src/streaming_inbox_checks.ts @@ -38,7 +38,7 @@ export type StreamingBlockMetadataCheckInput = { * The proposal's signed Inbox prefix reference: the rolling hash of the message prefix the block consumed through, * interpreted together with {@link endTotalMsgCount}. It need not name an L1 bucket boundary. */ - inboxPrefixRef: InboxMessagePrefixRef | undefined; + inboxPrefixRef: InboxMessagePrefixRef; /** Cumulative Inbox message count consumed through this block, from its signed header's L1-to-L2 leaf count. */ endTotalMsgCount: bigint; /** Cumulative Inbox message count consumed through the parent block (its L1-to-L2 tree leaf count; 0 at genesis). */ @@ -105,11 +105,13 @@ export type StreamingBlockCheckResult = * it, a matching hash at the end count proves the proposer consumed exactly the message prefix this node holds, so * the bundle between the parent's count and this one is determined by content alone. The checks, in order: * - * 1. **Reference present**: a streaming proposal must carry a prefix reference to authenticate against. - * 2. **Moves forward**: the end count is at least the parent block's, so consumption never rewinds. Equal counts + * 1. **Moves forward**: the end count is at least the parent block's, so consumption never rewinds. Equal counts * mean the block consumes nothing (empty bundle), which still has its prefix hash checked. - * 3. **Caps**: the per-block message count and the running per-checkpoint total fit their respective caps. - * 4. **Prefix matches**: the canonical prefix hash at the end count exists locally and equals the signed reference. + * 2. **Caps**: the per-block message count and the running per-checkpoint total fit their respective caps. + * 3. **Prefix matches**: the canonical prefix hash at the end count exists locally and equals the signed reference. + * + * There is no "reference present" check: the wire format makes the reference a required field, so a proposal that + * omits it fails to decode and never reaches here. * * There is deliberately no requirement that the end count sit on an L1 bucket boundary and no check on how recently * the messages arrived: blocks consume whatever message prefix the proposer's archiver had observed, and only the @@ -136,27 +138,22 @@ export async function checkStreamingBlockProposalMetadata( perCheckpointCap, } = input; - // Check 1: a streaming proposal must carry a prefix reference to authenticate its consumed range against. - if (inboxPrefixRef === undefined) { - return { accepted: false, reason: 'inbox_prefix_unavailable' }; - } - - // Check 2: consumption moves forward relative to the parent block. + // Check 1: consumption moves forward relative to the parent block. if (endTotalMsgCount < parentTotalMsgCount) { return { accepted: false, reason: 'consumption_moves_backwards' }; } - // Check 3a: the per-block message count fits the per-block cap. + // Check 2a: the per-block message count fits the per-block cap. if (endTotalMsgCount - parentTotalMsgCount > BigInt(perBlockCap)) { return { accepted: false, reason: 'bundle_over_block_cap' }; } - // Check 3b: the running per-checkpoint total fits the per-checkpoint cap. + // Check 2b: the running per-checkpoint total fits the per-checkpoint cap. if (endTotalMsgCount - checkpointStartTotalMsgCount > BigInt(perCheckpointCap)) { return { accepted: false, reason: 'checkpoint_over_msg_cap' }; } - // Check 4: the canonical prefix at the signed end count hashes to the signed reference. An empty block is checked + // Check 3: the canonical prefix at the signed end count hashes to the signed reference. An empty block is checked // here too: its unchanged count must still name the prefix the proposer signed, so an empty range is never a // licence to skip the hash. const canonical = await messageSource.getMessagePosition(endTotalMsgCount); diff --git a/yarn-project/validator-client/src/validator.ha.integration.test.ts b/yarn-project/validator-client/src/validator.ha.integration.test.ts index 832ec55ba2fa..17d580b2d855 100644 --- a/yarn-project/validator-client/src/validator.ha.integration.test.ts +++ b/yarn-project/validator-client/src/validator.ha.integration.test.ts @@ -18,7 +18,7 @@ import { AztecAddress } from '@aztec/stdlib/aztec-address'; import type { L2BlockSink, L2BlockSource } from '@aztec/stdlib/block'; import { CheckpointReexecutionTracker } from '@aztec/stdlib/checkpoint'; import type { SlasherConfig, ValidatorClientFullConfig, WorldStateSynchronizer } from '@aztec/stdlib/interfaces/server'; -import type { L1ToL2MessageSource } from '@aztec/stdlib/messaging'; +import { InboxMessagePrefixRef, type L1ToL2MessageSource } from '@aztec/stdlib/messaging'; import { TEST_COORDINATION_SIGNATURE_CONTEXT, makeBlockHeader, @@ -30,7 +30,7 @@ import { ConsensusTimetable } from '@aztec/stdlib/timetable'; import { TxHash } from '@aztec/stdlib/tx'; import { type TelemetryClient, getTelemetryClient } from '@aztec/telemetry-client'; import { INSERT_SCHEMA_VERSION, SCHEMA_SETUP, SCHEMA_VERSION } from '@aztec/validator-ha-signer/db'; -import { DutyAlreadySignedError } from '@aztec/validator-ha-signer/errors'; +import { DutyAlreadySignedError, SlashingProtectionError } from '@aztec/validator-ha-signer/errors'; import { createHASigner } from '@aztec/validator-ha-signer/factory'; import { Pool } from '@aztec/validator-ha-signer/test'; import type { ValidatorHASigner } from '@aztec/validator-ha-signer/validator-ha-signer'; @@ -308,6 +308,9 @@ describe('ValidatorClient HA Integration', () => { const archive = Fr.random(); const txs = await Promise.all([1, 2, 3].map(() => mockTx())); const proposerAddress = EthAddress.fromString(validatorAccounts[0].address); + // Shared across the validators: a per-validator reference would make each payload different, which the HA + // signer reports as a slashing-protection conflict rather than the duplicate duty this test covers. + const inboxPrefixRef = InboxMessagePrefixRef.random(); // All 5 validators try to create a block proposal for the same slot simultaneously const results = await Promise.allSettled( @@ -319,6 +322,7 @@ describe('ValidatorClient HA Integration', () => { archive, txs, proposerAddress, + inboxPrefixRef, { publishFullTxs: false, }, @@ -342,6 +346,42 @@ describe('ValidatorClient HA Integration', () => { expect(successfulResult?.value?.getSender()).toEqual(proposerAddress); }); + it('should refuse to sign a second prefix reference for the same block duty', async () => { + // After an L1 reorg a peer can rebuild the same block against a different message prefix. The reference is part + // of the signed payload, so that second attempt is a conflicting signature for a duty already signed, not a + // duplicate of it. + const blockHeader = makeBlockHeader(1); + const indexWithinCheckpoint = IndexWithinCheckpoint(0); + const archive = Fr.random(); + const txs = await Promise.all([1, 2, 3].map(() => mockTx())); + const proposerAddress = EthAddress.fromString(validatorAccounts[0].address); + + const first = await validators[0].createBlockProposal( + blockHeader, + CheckpointNumber(1), + indexWithinCheckpoint, + archive, + txs, + proposerAddress, + new InboxMessagePrefixRef(new Fr(1n)), + { publishFullTxs: false }, + ); + expect(first.getSender()).toEqual(proposerAddress); + + await expect( + validators[1].createBlockProposal( + blockHeader, + CheckpointNumber(1), + indexWithinCheckpoint, + archive, + txs, + proposerAddress, + new InboxMessagePrefixRef(new Fr(2n)), + { publishFullTxs: false }, + ), + ).rejects.toThrow(SlashingProtectionError); + }); + it('should allow different validators to create proposals for different slots', async () => { const proposerAddress = EthAddress.fromString(validatorAccounts[0].address); const txs = await Promise.all([1, 2, 3].map(() => mockTx())); @@ -358,6 +398,7 @@ describe('ValidatorClient HA Integration', () => { archive, txs, proposerAddress, + InboxMessagePrefixRef.random(), { publishFullTxs: false }, ); }), diff --git a/yarn-project/validator-client/src/validator.integration.test.ts b/yarn-project/validator-client/src/validator.integration.test.ts index 084f6fc5814a..95ed4a013daa 100644 --- a/yarn-project/validator-client/src/validator.integration.test.ts +++ b/yarn-project/validator-client/src/validator.integration.test.ts @@ -290,11 +290,14 @@ describe('ValidatorClient Integration', () => { // Attach the signed reference to the message prefix this block consumed through (the rolling hash at its // cumulative L1-to-L2 leaf count), mirroring the sequencer's block-building loop which carries one on every - // proposal. Without it the validator's streaming acceptance check rejects the proposal as - // `inbox_prefix_unavailable`. A block that consumed nothing references the parent's prefix. + // proposal. A block that consumed nothing references the parent's prefix. A prefix the proposer's own archiver + // cannot serve means the test fixture is out of sync, so fail loudly rather than signing a placeholder. const blockTotal = BigInt(block.header.state.l1ToL2MessageTree.nextAvailableLeafIndex); const position = await proposer.archiver.getMessagePosition(blockTotal); - const inboxPrefixRef = position ? InboxMessagePrefixRef.fromPosition(position) : undefined; + if (position === undefined) { + throw new Error(`No Inbox message position at cumulative total ${blockTotal} for block ${blockNumber}`); + } + const inboxPrefixRef = InboxMessagePrefixRef.fromPosition(position); const proposal = await proposer.validator.createBlockProposal( block.header, @@ -303,8 +306,8 @@ describe('ValidatorClient Integration', () => { block.archive.root, usedTxs, proposerSigner.address, - {}, inboxPrefixRef, + {}, ); logger.warn(`Built block proposal for block ${blockNumber}`, { ...block.toBlockInfo() }); diff --git a/yarn-project/validator-client/src/validator.test.ts b/yarn-project/validator-client/src/validator.test.ts index 1c74de0d6244..1b0d2f12db89 100644 --- a/yarn-project/validator-client/src/validator.test.ts +++ b/yarn-project/validator-client/src/validator.test.ts @@ -269,6 +269,7 @@ describe('ValidatorClient', () => { archive, txs, EthAddress.fromString(validatorAccounts[0].address), + InboxMessagePrefixRef.random(), { publishFullTxs: false }, ); diff --git a/yarn-project/validator-client/src/validator.ts b/yarn-project/validator-client/src/validator.ts index 2642856aea66..12b872f18a94 100644 --- a/yarn-project/validator-client/src/validator.ts +++ b/yarn-project/validator-client/src/validator.ts @@ -1019,8 +1019,8 @@ export class ValidatorClient extends (EventEmitter as new () => WatcherEmitter) archive: Fr, txs: Tx[], proposerAddress: EthAddress | undefined, + inboxPrefixRef: InboxMessagePrefixRef, options: BlockProposalOptions = {}, - inboxPrefixRef?: InboxMessagePrefixRef, ): Promise { // Validate that we're not creating a proposal for an older or equal position if (this.lastProposedBlock) {