Repository navigation
Commit 0fc54fa
authored
fix!: keep Python node results in one location per target (#1551)
* fix: keep Python node results in one location per target
Loaders, tasks, assets and checks now write their results and loader
fingerprints to one location for every target: the target's schema, the
target's loader_schema when the schema is preserve, or the adapter's
default schema when the target schema is unset or resolved per model.
Scheduled loaders and build-time checks used the adapter default before,
so adapters without one, such as Snowflake, skipped those writes.
Compile stops with S105 when a preserve target without a loader_schema
has Python nodes.
For one release, a node whose result is missing from the new location is
read from the adapter's default location instead.
* fix: read results left in a preserve schema and cover every runtime
Under a preserve target, also read a node's result from the schema named
preserve that earlier releases wrote to, after the adapter default. The
location for `sqb load`, build-time checks and `sqb check` now comes from
the target settings, so each runtime reads and writes the same place.
* fix: resolve sqb load's result location with CLI vars and share loader result writes
sqb load now takes its node-result location from the CLI-var-aware loader
defaults, as build does, so a schema set with --vars is read from the same
place both commands write. Build and ingress loaders write their results
through one load operation.
* test: add the preserved-schema refusal to the compiler failure corpus
* test: give the preserved-target load cases a loader schema
A preserve target with a loader now needs loader_schema (S105), so the
managed-source load cases set one. The source still lands in its own
schema.
* fix!: require a loader schema for preserve targets with Python nodes
BREAKING CHANGE: a preserve target with Python loaders, tasks, assets or
checks now needs loader_schema, which is where their results are kept.
Without it, compile fails with S105, which says to set
targets.<name>.loader_schema.1 parent f5d8664 commit 0fc54fa
49 files changed
Lines changed: 936 additions & 167 deletions
File tree
- benchmarks/duckshop/edge_cases/loaders
- crates
- sqlbuild-analysis/src/assembly/target_checks
- main
- sqlbuild-python-base/src/warehouse_access/main
- sqlbuild-python-compile/src/compile_driver
- _helpers/setup
- main/setup
- sqlbuild-python/src/bindings/_helpers
- build_command
- check
- load
- scripts/compiler_differential/_helpers/analysis_corpus
- src/sqlbuild
- .agents/skills/sqlbuild/references/docs/concepts
- python-nodes
- executor
- build
- _helpers
- load
- _helpers
- main
- node_results
- classes
- main
- python_nodes
- _helpers
- classes
- tests
- e2e/src/sqlbuild/cli/commands/main
- build
- compile
- goldens/compiler/failure
- integration/src/sqlbuild/executor/node_results/main
- unit/src/sqlbuild/executor
- build/_helpers
- python_nodes/_helpers
- website/src/content/docs/docs/concepts
- python-nodes
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
151 | 151 | | |
152 | 152 | | |
153 | 153 | | |
| 154 | + | |
154 | 155 | | |
155 | 156 | | |
156 | 157 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
| 13 | + | |
| 14 | + | |
13 | 15 | | |
14 | 16 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
| 18 | + | |
| 19 | + | |
18 | 20 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
48 | 48 | | |
49 | 49 | | |
50 | 50 | | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
51 | 56 | | |
52 | 57 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
1 | 2 | | |
2 | 3 | | |
3 | 4 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
| 2 | + | |
| 3 | + | |
2 | 4 | | |
3 | 5 | | |
4 | 6 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
0 commit comments