Repository navigation
fix(server): commit SQLite control-store writes durably without a checkpoint on every close - #1481
Merged
Merged
Conversation
…ckpoint on every close Each control-store transaction opens its own connection, and closing the last one checkpointed the WAL and synced the database file on every commit, which stalled writes and scheduler ticks on slow disks. Connections now keep the WAL when they close, writes commit with synchronous=FULL, platform maintenance truncates the WAL every 30 seconds so constant readers cannot grow it without bound, and a commit that leaves a WAL over 64 MiB resets it at once. The docs say to back SQLite up with .backup or with the -wal file, never the database file alone.
A renewal that found the lease expired between trigger evaluations or in the launcher's poll ended the scheduler loop, which stopped the whole server. The loop now retries, and the next cycle rejoins the election. The tick retention test writes its backlog in one transaction, and its background lease holder reports why it stopped renewing.
WIP: review fixes. Hold the store's setup connection for its lifetime so transactions that never overlap don't rebuild the WAL index and restart checkpointing; stop the fenced-loop test once it rejoins; neutral wording for a fenced cycle; bound the crash test from below only.
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
Nov 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The SQLite control store opens a connection per transaction. Closing the last one checkpointed the WAL and synced the database on every commit, so a slow disk stalled writes and scheduler ticks. On a slow CI runner a renewal ran past the lease, the cycle returned
Fenced, and the scheduler loop stopped the server. Commits also relied on the default sync level.Changes
synchronous=FULL..backup, or stop the server first; never copy the database file alone.Verification
make check-cipass. The review findings were each re-checked after fixing.