Repository navigation
Conversation
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ELIE: Durable artifact certification
Summary
A late database commit can leave an artifact change outside the 120-second overlap. Each relevant writer records a pending operation in its own transaction. Recovery proves visibility within the overlap or requests the existing
policy_changedreset. It completes the reset and certification atomically. Recovery does not require a client retry.The wire format stays at version 1. Existing clients use the same reset flow. The pending insert shares the existing mutation statement, so it adds no SQL round trip. Recovery processes up to 1,000 records across communities in one transaction. A background election lets one worker process each batch. Confirmed commits reuse connections. Cancellation and uncertain results close them. Serving writes take no election lock or shared counter.
Migration
0081is an additive expand step. It adds one table with a primary key and an index for recovery. It adds no trigger, function, foreign key, or backfill. The desired-state manifest, deletion inventory, purge list, and catalog parity checks include the table. Old serving writers ignore it. Between the migration and the fleet upgrade, old relays refuse whole-community deletion because their table list omits the new table. Normal reads and writes continue. The existing SQLxVersionMissingrestriction still applies to rollback with startup migrations enabled.Full coverage requires all serving writers to use the new code. This includes artifact changes and redaction, grants and removals, channel creation and access settings, DMs, and admin removals. Drain old writers and use the documented reset procedure after the fleet upgrade. This PR does not deploy a relay.
Related issue
No duplicate found. This hardens incremental artifact reads. Local merge checks find no conflicts with migration startup or artifact CLI commands at their checked heads.
Testing
The real relay process test covers a 500-record backlog, restart recovery, signed HTTP, existing checkpoints, and ordinary reads.
Performance
On Blox with loopback PostgreSQL 16 and debug builds, recovery drains 10,000 records across 100 communities in a median 49.5 ms. The first draft takes 14.16 seconds. Warm recovery opens no new connections; the first draft opens 100. Single-writer artifact creation takes a median 0.961 ms, compared with 0.918 ms on the base. These short runs do not establish production throughput or the cost of WAL and vacuum.
Generated with Codex