Repository navigation
feat(location): sync warehouse history after builds, while idle and around a busy DuckDB file - #1449
Merged
Merged
Conversation
…nutes A [control] server and location in sqlbuild_project.toml, or SQLBUILD_CONTROL_* variables, let a signed-in sqb build ask that location to sync the target it built. The request is best-effort, bounded to five seconds and never changes the exit code. A sync request while one is pending for the target joins it instead of queueing another. sqb location serve also syncs an idle location at least every sync_at_least_every (default 30m, 0 or off disables it), as one more trigger into the same outstanding sync.
A metadata read that finds the DuckDB file locked by another process now reports a busy sync instead of a failed read. The location backs off one poll interval without shrinking its page, logs one line when the wait starts, and the console shows the table as waiting rather than failing.
…ync on its own thread A failed read after a busy reply no longer stays on the busy path, the post-build request runs entirely within its bound (keyring, DNS and server), its errors never change the build's result, and the console shows a waiting table in the warning tone.
…h-free Satisfies the repository's structure and test rules: the build sync's phases live in a helper module, the binding imports it directly, environment reads match their errors, and the safety and lock tests use case data instead of branches.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The console's warehouse history went stale: a person's
sqb builddid not tell the server, an idle location never synced, a build holding the DuckDB file made syncs fail, and audit result history grew without bound.Changes
_sqlbuild_audit_resultsto the newest rows per audit binding on every adapter.sqb build, the CLI asks the server to sync the configured location. The location comes from a new[control]section orSQLBUILD_CONTROL_*variables. Bounded to 5 seconds, it prints one line after the build summary and never changes the build's result.sqb location servesyncs at least every 30 minutes while idle (--sync-at-least-every, environment variable or[control]; minimum 1 minute,offdisables). Any sync restarts the interval.Verification
test_person_sql.py); a served DuckDB location reports busy while a write lock is held and syncs once it is released (test_warehouse_metadata.py).test_janitor.py).offand precedence.make check-cipassed on the final head.DuckShop: adds the held-lock stage to
incremental_warehouse_history.jsonand the pruning timeline with its boundary tie asdirect_state_pruning.json.