Repository navigation
Commit 53d8395
authored
fix!: atomic keyed writes, indexed control store and matching loader locations (#1478)
* fix(server): index every control-store foreign key
* fix: write keyed delete_insert atomically, add store indexes in place and expand loader location templates
* style: shorten the Databricks keyed MERGE docstring
* fix: stop a loader from writing to a schema named preserve
* fix: say a connection schema counts only for a project's only top-level connection
* fix: show the S104 hint's connection schema with command-line vars applied
* fix(debug): report no database for DuckDB targets
Debug passed the DuckDB connection file path as the target database,
while models build in the file's own catalog. Debug now drops it for
namespaceless adapters, as diff already does.
* fix: check only intermediate loaders under a preserved target schema
A managed source's loader writes to the source's own schema or the
target's loader_schema, so a target schema of preserve no longer stops
it. The check now applies when the selected target's own schema is
preserve and the loader's destination names no schema, the same way for
compile, build and load.
* refactor: read the loader node flag without a Python call
* fix(load): load a managed source into the target's loader_schema
sqb load filled a managed source's missing schema from the target's
schema, while build uses the target's loader_schema first, so the two
commands wrote the same source to different schemas. Load now resolves
it the same way, and its missing-schema error names a managed source as
one.
* refactor(load): pick a managed source's schema without comparing field names
* fix(load): stop a managed source that would load into a schema named preserve
With loader_schema or schema set to preserve and no schema on the
source, build stops with P001 while load wrote into a schema literally
named preserve. Load now raises the same error.
* fix(load): expand templated target locations for managed sources
sqb load read the target's database, schema and loader_schema as
written when it placed a managed source, so a value such as
${raw_schema} failed with a parser error while build expanded it. Load
now expands them with the project and command-line vars, as build does.
* fix(load): expand only the target location a managed source falls back to
Expanding the whole target up front failed load on unused templates that
build ignores, and accepted run context templates that build refuses.
Load now expands just the fallback value, with build's own options.
* fix(load): read an expanded target location as text, as build does
* docs(deploy): run the server and a location as Podman Quadlet units
* fix(scheduler): wake on a due time however long the store took, and log slow cycles
The loop slept for the next due time's distance measured before the cycle's
last commit, so a slow commit or WAL checkpoint pushed every wake back by its
own length. It now sleeps until that deadline. Each cycle records per-phase
timings, the worker supervisor logs cycles slower than 500 ms with them, and
the latency test bounds how late the scheduler wakes while printing store time
separately.
* refactor: share target lookup and load-selection request reading
Named-target checks reuse the compile attachment's resolved_target instead of
a copy, the named-target schema binding declares its keyword arguments
directly, and both load-selection bindings read their request through one
helper.1 parent 1b98482 commit 53d8395
50 files changed
Lines changed: 1370 additions & 199 deletions
File tree
- crates
- sqlbuild-analysis/src/assembly/target_checks
- main
- tests
- sqlbuild-cli/src/workers
- _helpers
- sqlbuild-orchestration
- src
- scheduler
- _helpers
- main
- store
- _helpers
- main
- tests
- scheduler/ticks
- store
- schema
- sqlbuild-python/src/bindings/_helpers
- compile_attachment
- compile_driver
- debug_command
- diff_report
- load
- deploy
- src/sqlbuild
- .agents/skills/sqlbuild/references/docs/concepts/python-nodes
- adapters
- databricks
- classes
- snowflake/classes
- tests
- e2e/src/sqlbuild/cli/commands/main
- compile
- debug
- goldens/compiler/failure
- unit/src/sqlbuild
- adapters
- databricks
- snowflake
- cli/commands/_helpers/load_selection
- website/src/content/docs/docs/concepts/python-nodes
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
| 16 | + | |
| 17 | + | |
16 | 18 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
35 | 35 | | |
36 | 36 | | |
37 | 37 | | |
38 | | - | |
| 38 | + | |
| 39 | + | |
39 | 40 | | |
40 | 41 | | |
41 | 42 | | |
42 | 43 | | |
43 | 44 | | |
44 | 45 | | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
45 | 51 | | |
46 | 52 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
46 | 46 | | |
47 | 47 | | |
48 | 48 | | |
49 | | - | |
| 49 | + | |
50 | 50 | | |
51 | 51 | | |
52 | 52 | | |
53 | 53 | | |
54 | 54 | | |
55 | 55 | | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
56 | 61 | | |
57 | 62 | | |
58 | 63 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
16 | 16 | | |
17 | 17 | | |
18 | 18 | | |
19 | | - | |
| 19 | + | |
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
| |||
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
| 29 | + | |
29 | 30 | | |
30 | 31 | | |
31 | 32 | | |
| |||
149 | 150 | | |
150 | 151 | | |
151 | 152 | | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
152 | 169 | | |
153 | 170 | | |
154 | 171 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | | - | |
| 1 | + | |
2 | 2 | | |
| 3 | + | |
3 | 4 | | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | | - | |
| 3 | + | |
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
| |||
42 | 42 | | |
43 | 43 | | |
44 | 44 | | |
45 | | - | |
| 45 | + | |
46 | 46 | | |
47 | 47 | | |
48 | 48 | | |
49 | | - | |
| 49 | + | |
50 | 50 | | |
51 | 51 | | |
| 52 | + | |
52 | 53 | | |
53 | 54 | | |
54 | 55 | | |
55 | 56 | | |
56 | 57 | | |
57 | 58 | | |
58 | 59 | | |
59 | | - | |
| 60 | + | |
60 | 61 | | |
61 | 62 | | |
62 | 63 | | |
63 | 64 | | |
64 | | - | |
| 65 | + | |
| 66 | + | |
65 | 67 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | | - | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
4 | 6 | | |
5 | 7 | | |
6 | 8 | | |
| |||
9 | 11 | | |
10 | 12 | | |
11 | 13 | | |
12 | | - | |
| 14 | + | |
13 | 15 | | |
14 | 16 | | |
15 | 17 | | |
16 | 18 | | |
17 | 19 | | |
18 | 20 | | |
19 | 21 | | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
20 | 31 | | |
21 | 32 | | |
22 | 33 | | |
| |||
26 | 37 | | |
27 | 38 | | |
28 | 39 | | |
| 40 | + | |
29 | 41 | | |
30 | 42 | | |
31 | 43 | | |
32 | 44 | | |
33 | 45 | | |
34 | 46 | | |
| 47 | + | |
| 48 | + | |
35 | 49 | | |
36 | 50 | | |
37 | 51 | | |
38 | 52 | | |
| 53 | + | |
39 | 54 | | |
| 55 | + | |
| 56 | + | |
40 | 57 | | |
41 | 58 | | |
42 | 59 | | |
43 | 60 | | |
| 61 | + | |
| 62 | + | |
44 | 63 | | |
45 | 64 | | |
46 | | - | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
47 | 69 | | |
48 | 70 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
| 6 | + | |
6 | 7 | | |
7 | 8 | | |
8 | 9 | | |
| |||
24 | 25 | | |
25 | 26 | | |
26 | 27 | | |
| 28 | + | |
27 | 29 | | |
28 | 30 | | |
29 | 31 | | |
30 | | - | |
| 32 | + | |
31 | 33 | | |
32 | 34 | | |
33 | 35 | | |
34 | | - | |
35 | | - | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
36 | 39 | | |
37 | 40 | | |
38 | | - | |
| 41 | + | |
39 | 42 | | |
40 | 43 | | |
41 | 44 | | |
42 | 45 | | |
43 | | - | |
| 46 | + | |
44 | 47 | | |
45 | 48 | | |
46 | 49 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
40 | 40 | | |
41 | 41 | | |
42 | 42 | | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
43 | 63 | | |
44 | 64 | | |
45 | 65 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | | - | |
| 3 | + | |
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
21 | 24 | | |
22 | 25 | | |
23 | 26 | | |
0 commit comments