Skip to content

refactor(location): drop old-server fallbacks and duplicate sync state - #1587

Merged
kvlonge merged 5 commits into
mainfrom
refactor/sync-tidy
Nov 18, 2026
Merged

kvlonge merged 5 commits into
mainfrom
refactor/sync-tidy

Conversation

@kvlonge

@kvlonge kvlonge commented Nov 18, 2026

Copy link
Copy Markdown
Contributor

Why

The location sync carried fallbacks for older servers and state that duplicated other state. There are no external users yet and the server upgrades first, so the fallbacks go. The connections page also counted 14 mirror tables on every load to show mirrored totals.

Changes

  • The sync-due reply carries the state history versions; the separate retention endpoint and its client call are removed.
  • Preview copies always carry a version; the unversioned read and its retry are removed.
  • busy comes from the read failure, the page time is a constant, and the reader-start read failure is folded into the read error.
  • Resuming a halted sync resets its failure count however it resumes.
  • An explicit safety-sync interval no longer overrides the rule that warehouses whose history reads wake compute skip it.
  • Mirrored totals: the server counts a table's mirror when a read delivers its last page and returns the count. The location reports it in sync health, a table skipped as unchanged repeats the newest count of its kind, and the connections page reads it from health. The /mirrored-totals endpoint is removed.
  • Regenerated TypeScript client and server docs.

Upgrade note: a location older than this server cannot decode the new sync-due reply, so upgrade locations with the server. Health events of the removed reader-start kind written in the last 7 days no longer decode.

Verification

  • Failing-first tests: the halted-resume reset, the safety-interval rule, and a skipped table's mirrored count, with a two-location case.
  • sqlbuild-cli location metadata, sync schedule and changed-build tests.
  • Orchestration maintenance, warehouse metadata, preview copy and location health tests on SQLite and PostgreSQL.
  • UI svelte-check and vitest.
  • make check-ci on the final head.

- Sync-due replies carry the server's state-history limit; the retention
  endpoint and its client call are gone.
- Preview copies are always versioned: no `versioned` flag, no unversioned
  retry, and `version` is always present.
- A stopped sync resumed by a run, the timed sync or a request backs off
  again from a fresh failure count.
- A busy warehouse file is read from the failure itself, the page timeout is
  a constant, and a reader that did not start reports a read error.
- The timed sync never wakes billed compute, whether or not the interval was
  set.
- Mirrored record counts come from the sync: the server counts a table after
  its read's last page and the location reports it, replacing the
  per-page-load count endpoint.
A table the warehouse reports unchanged is skipped without delivery, so
it reported no mirrored count and its total dropped out of the
connections page once older reports aged out. The recorded change token
now keeps the count from the read that recorded it, and the skipped read
reports it again. Regenerate the server docs reference.
…kind

The server counts a kind's mirror across every state location, so a read
of the same kind in another location replaces the count a skipped table
repeats. Without this, a skipped table could report an older total than
the read before it.
@kvlonge
kvlonge enabled auto-merge November 18, 2026 21:44
# Conflicts:
#	crates/sqlbuild-cli/src/native_commands/models.rs
#	crates/sqlbuild-cli/src/native_commands/tests/test_types.rs
#	crates/sqlbuild-orchestration/src/warehouse_metadata/_helpers/ingestion.rs
@kvlonge
kvlonge added this pull request to the merge queue Nov 18, 2026
Merged via the queue into main with commit 069c955 Nov 18, 2026
35 checks passed
@kvlonge
kvlonge deleted the refactor/sync-tidy branch November 18, 2026 23:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant