Skip to content

fix(relay): certify artifact sync visibility durably - #8933

Draft
dr0uleau wants to merge 22 commits into
mainfrom
nexus/harden-artifact-sync-with-durable-certification-c4f079a9
Draft

dr0uleau wants to merge 22 commits into
mainfrom
nexus/harden-artifact-sync-with-durable-certification-c4f079a9

Conversation

@dr0uleau

@dr0uleau dr0uleau commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

ELIE: Durable artifact certification

Summary

A late database commit can leave an artifact change outside the 120-second overlap. Each relevant writer records a pending operation in its own transaction. Recovery proves visibility within the overlap or requests the existing policy_changed reset. It completes the reset and certification atomically. Recovery does not require a client retry.

The wire format stays at version 1. Existing clients use the same reset flow. The pending insert shares the existing mutation statement, so it adds no SQL round trip. Recovery processes up to 1,000 records across communities in one transaction. A background election lets one worker process each batch. Confirmed commits reuse connections. Cancellation and uncertain results close them. Serving writes take no election lock or shared counter.

Migration 0081 is an additive expand step. It adds one table with a primary key and an index for recovery. It adds no trigger, function, foreign key, or backfill. The desired-state manifest, deletion inventory, purge list, and catalog parity checks include the table. Old serving writers ignore it. Between the migration and the fleet upgrade, old relays refuse whole-community deletion because their table list omits the new table. Normal reads and writes continue. The existing SQLx VersionMissing restriction still applies to rollback with startup migrations enabled.

Full coverage requires all serving writers to use the new code. This includes artifact changes and redaction, grants and removals, channel creation and access settings, DMs, and admin removals. Drain old writers and use the documented reset procedure after the fleet upgrade. This PR does not deploy a relay.

Related issue

No duplicate found. This hardens incremental artifact reads. Local merge checks find no conflicts with migration startup or artifact CLI commands at their checked heads.

Testing

The real relay process test covers a 500-record backlog, restart recovery, signed HTTP, existing checkpoints, and ordinary reads.

Performance

On Blox with loopback PostgreSQL 16 and debug builds, recovery drains 10,000 records across 100 communities in a median 49.5 ms. The first draft takes 14.16 seconds. Warm recovery opens no new connections; the first draft opens 100. Single-writer artifact creation takes a median 0.961 ms, compared with 0.918 ms on the base. These short runs do not establish production throughput or the cost of WAL and vacuum.

Generated with Codex

Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
@dr0uleau
dr0uleau requested review from a team, gilgtm and kazemek21 November 2, 2026 17:53
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant