Repository navigation
fix!: atomic keyed writes, indexed control store and matching loader locations - #1478
Merged
Merged
Conversation
… and expand loader location templates
Debug passed the DuckDB connection file path as the target database, while models build in the file's own catalog. Debug now drops it for namespaceless adapters, as diff already does.
A managed source's loader writes to the source's own schema or the target's loader_schema, so a target schema of preserve no longer stops it. The check now applies when the selected target's own schema is preserve and the loader's destination names no schema, the same way for compile, build and load.
sqb load filled a managed source's missing schema from the target's schema, while build uses the target's loader_schema first, so the two commands wrote the same source to different schemas. Load now resolves it the same way, and its missing-schema error names a managed source as one.
…preserve With loader_schema or schema set to preserve and no schema on the source, build stops with P001 while load wrote into a schema literally named preserve. Load now raises the same error.
sqb load read the target's database, schema and loader_schema as
written when it placed a managed source, so a value such as
${raw_schema} failed with a parser error while build expanded it. Load
now expands them with the project and command-line vars, as build does.
…k to Expanding the whole target up front failed load on unused templates that build ignores, and accepted run context templates that build refuses. Load now expands just the fallback value, with build's own options.
…og slow cycles The loop slept for the next due time's distance measured before the cycle's last commit, so a slow commit or WAL checkpoint pushed every wake back by its own length. It now sleeps until that deadline. Each cycle records per-phase timings, the worker supervisor logs cycles slower than 500 ms with them, and the latency test bounds how late the scheduler wakes while printing store time separately.
Named-target checks reuse the compile attachment's resolved_target instead of a copy, the named-target schema binding declares its keyword arguments directly, and both load-selection bindings read their request through one helper.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Unindexed control-store foreign keys held the SQLite write lock through scans. Databricks and Snowflake keyed writes were not atomic.
sqb loadplaced managed sources differently from build, and a loader under a target schema ofpreservewrote into a schema namedpreserve. A slow store commit delayed every scheduler wake.Changes
delete_insertis one MERGE; Snowflake runs its staging CTAS before the transaction.sqb loadputs a managed source where build does: its own location, else the target'sloader_schema, thenschema. It expands only the target value it falls back to, with--vars, and refuses${CTX:...}as build does.preservestops an intermediate loader with no destination schema (S103) and a managed source with no schema (P001).sqb debugon DuckDB are corrected.BREAKING CHANGE: loaders that wrote into a schema named
preservenow fail with S103 or P001 until they name a schema.sqb loadnow writes managed sources toloader_schema, like build.Verification
make check-cipasses. DuckShop is unchanged: these are target settings and command placement, not project shapes.