Skip to content

fix: keep the earliest row per source id in state history mirrors - #1532

Merged
kvlonge merged 12 commits into
mainfrom
fix/deterministic-history-mirror
Nov 17, 2026
Merged

kvlonge merged 12 commits into
mainfrom
fix/deterministic-history-mirror

Conversation

@kvlonge

@kvlonge kvlonge commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

Why

When a state table holds two rows with the same source id, such as one check result written twice, the server's mirror kept whichever arrived first. A full reread could keep a different row than an incremental scan, so the history shown depended on page order, and one integration test failed intermittently.

Mirrors now keep the same row whatever the order. A server that kept a different row corrects itself at the location's next full reread: at location start, and on the first sync at least 5 minutes after the previous one.

Changes

  • A conflict on a source id keeps the earliest observed_at, then the smallest record text in byte order. SQLite and PostgreSQL may differ on an exact tie; each is deterministic and a store migration rebuilds the mirror.
  • Estimated query costs keep the latest row, as an explicit per-kind rule: two builds refreshing one run's estimate at once can write the same id, and the warehouse treats the latest row as current.
  • The Snowflake-like connection keys warning prints once per process, not twice per command.
  • The server docs describe the rule and the self-correction.
  • A dispatcher health test waits for delivery instead of a fixed interval.

Verification

  • New Rust test: two rows per source id in both orders, on SQLite and PostgreSQL, for audit results and costs, plus a same-time tie; asserts the kept record, time and typed columns, and that a replay reports no change. Fails on main and without the per-kind rule.
  • New unit and E2E tests for one warning per process; both failed first.
  • Dispatcher test under CPU load: old 26/40 failures, new 0/40.
  • Growth: +4 code lines in sqlbuild-python (66,756 to 66,760); cap set exactly.
  • make check-ci on the final head.

DuckShop: audit_result_history.json already carries 105 duplicate source ids at a later time; the cost case is built from resource_cost_history.json's record.

…ages arrive in

A state history mirror kept the first row it received for a source id.
When one id had rows at different times, a rebuild that read a later
page first kept a different row than a clean scan. The earliest row
now wins, then the smallest record text, on SQLite and PostgreSQL.

Also print the Snowflake-like connection keys warning once per process.
A refreshed cost estimate reuses its observation id and the warehouse
treats the latest row as current, so its mirror keeps the latest row.
Every other history kind keeps the earliest.
…cher

The test waited for the first periodic summary and then expected the event
to be delivered, but a summary can be published while the event is still
queued. Wait for delivery and for a summary that reports it, then count
deliveries after shutdown drains the queue.
A refresh writes a new attempt with a new id; the same id comes from two
builds refreshing one run's estimate at once.
@chio-labs-delivery
chio-labs-delivery Bot added this pull request to the merge queue Nov 17, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Nov 17, 2026
@chio-labs-delivery
chio-labs-delivery Bot added this pull request to the merge queue Nov 17, 2026
@kvlonge
kvlonge removed this pull request from the merge queue due to a manual request Nov 17, 2026
@kvlonge
kvlonge added this pull request to the merge queue Nov 17, 2026
Merged via the queue into main with commit 4708af5 Nov 17, 2026
34 checks passed
@kvlonge
kvlonge deleted the fix/deterministic-history-mirror branch November 17, 2026 05:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant