Repository navigation
perf(location): read sync pages back to back and check git HEAD at most every 2 s - #1454
Merged
Merged
Conversation
…most every 2 s A synchronization sent one 100-row page per 250 ms watch tick, so a 10,000-row reconcile spent most of its time waiting for the next tick. While a sync has a page due, the loop now skips the tick; idle work polling is unchanged. Only the watch tick asks git for HEAD, at most every 2 s; the work and metadata fences compare project files only. The SQLite consumption lifecycle case now inserts recorded_at as text, the way the writer does.
The DuckDB overlap case took its write lock while the restored sync was still reading, so the busy report landed on whichever table that sync had reached. With pages read back to back that is no longer source_freshness. The case now waits for the sync to finish and expects the next sync's first table to report the lock.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A served location sent one 100-row warehouse-metadata page per 250 ms watch tick, so a sync of 10,000 fingerprints spent most of its time waiting for the next tick. Each idle tick also spawned
git rev-parsethree times (once per fence), which is wasted work on an idle server.Changes
recorded_atas ISO text, the way the store writer does, and the consumption lifecycle test covers SQLite as well as DuckDB.DuckShop: no new edge case. The change is to how often a served location polls and pages, not to project shapes, data values or a build lifecycle sequence.
Verification
fensu checkis clean;make check-cipasses.