Skip to content

feat(server): migrate the control store with numbered migrations - #1563

Merged
kvlonge merged 4 commits into
mainfrom
feat/control-store-migrations
Nov 18, 2026
Merged

kvlonge merged 4 commits into
mainfrom
feat/control-store-migrations

Conversation

@kvlonge

@kvlonge kvlonge commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Why

The control store had no upgrade path. A server refused any store whose schema checksum differed, so every schema.sql change forced users to recreate their store, losing people, roles, tokens, triggers and run history.

Changes

  • sqb_schema_version records a version. A new store is created from schema.sql and stamped with the current version. Older stores apply the numbered migrations after their version in one transaction under the store lock. On SQLite that transaction runs with foreign keys off, so a migration can rebuild a referenced table, and checks them before committing.
  • Version 1 is today's schema; unversioned stores at it are upgraded in place, whether a Linux, macOS or Windows release wrote them. The checksum covers indexes.sql and ignores line endings, so Windows builds agree with the others.
  • The server refuses, unchanged, a store from a newer release, one older than version 1, and one at the current version whose schema differs, saying how to fix each.
  • sqb server migrate applies pending migrations and exits; --dry-run lists them and changes nothing.
  • Migrations live in store_migrations/, one per version with SQLite and PostgreSQL SQL.
  • Server docs describe upgrading, the dry run, refusals and rollback by backup.

Verification

  • Contract tests on SQLite and PostgreSQL:
    • The baseline plus every migration must match a new store, and a populated store keeps its rows.
    • The current version's checksum is pinned, also for CRLF SQL (failed before the fix).
    • An sqb_runs rebuild, refusals, a migration that breaks a reference, and the dry run.
  • Fail-first: equivalence after a schema.sql edit without a migration; the rebuild with SQLite foreign keys on.
  • A CLI test covers sqb server migrate [--dry-run], including a dry run without a SQLite store, which must create nothing (failed before the guard).
  • Orchestration crate tests and CLI command tests pass with PostgreSQL.
  • make check-ci passes on this head.

The control store records a schema version next to its checksum. A
server creates a new store at the current version and upgrades an older
one by applying the numbered migrations after it, in one transaction
under the store lock. On SQLite, the transaction runs with foreign keys
off so a migration can rebuild a referenced table, then checks them
before committing. Stores from releases before versioning that match
the version 1 baseline are upgraded in place.

The server refuses a store written by a newer release, a store older
than the baseline, and a store at the current version whose schema
differs, each with the fix. `sqb server migrate` applies the pending
migrations and exits; `--dry-run` lists them without changing the store.

Contract tests on SQLite and PostgreSQL check that the baseline plus
every migration matches a new store, that a populated store keeps its
rows, that a migration can rebuild sqb_runs, the refusals, and the dry
run.
`sqb server migrate --dry-run` opened the store to plan the migration,
and opening a SQLite store creates its file and parent directories. A
dry run on a missing file now reports that there is no store and that a
migration would create one, without creating anything.
@kvlonge
kvlonge disabled auto-merge November 17, 2026 23:17
@chio-labs-delivery
chio-labs-delivery Bot added this pull request to the merge queue Nov 17, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Nov 18, 2026
Windows checkouts turn schema.sql and indexes.sql into CRLF, and the
binaries embed them as checked out. A Windows build therefore recorded
a different checksum than a Linux or macOS build for the same schema:
it refused the baseline store from a Windows release as older than
version 1, and Windows and Linux servers sharing a PostgreSQL store
would disagree about whether its schema matches.

The checksum is now taken over LF text, and the baseline accepts both
checksums released stores recorded: from LF and from CRLF SQL. The SQL
files are also checked out with LF everywhere.
@kvlonge
kvlonge disabled auto-merge November 18, 2026 01:54
@kvlonge
kvlonge enabled auto-merge November 18, 2026 01:55
@kvlonge
kvlonge added this pull request to the merge queue Nov 18, 2026
Merged via the queue into main with commit 3f83a7f Nov 18, 2026
34 checks passed
@kvlonge
kvlonge deleted the feat/control-store-migrations branch November 18, 2026 02:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant