Skip to content

fix!: keep Python node results in one location per target - #1551

Merged
kvlonge merged 7 commits into
mainfrom
fix/preserve-node-results
Nov 17, 2026
Merged

kvlonge merged 7 commits into
mainfrom
fix/preserve-node-results

Conversation

@kvlonge

@kvlonge kvlonge commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Why

Python node results landed in different places by path. Under schema = "preserve" they went to a schema named preserve. Scheduled loaders (sqb build, sqb load) and build-time checks used the adapter's default schema while tasks read the target's, so ctx.result_of could miss. On Snowflake, which has no default schema, scheduled loaders saved no results or fingerprints.

Changes

Loader, task, asset and check results, and loader fingerprints, go to one place per target:

Target schema Location
A name The target's schema and database
preserve The target's loader_schema
Unset or ${CTX:...} The adapter's default schema

A preserve database uses the adapter's default database.

  • New S105: compile stops when a preserve target with loaders, tasks, assets or checks has no loader_schema.
  • Old locations are read for one release, per node: the adapter's default location and, under preserve, the preserve schema. Writes go only to the new location.
  • Build-time checks and sqb check reads qualify unqualified names into this location, as build tasks do.
  • The plan may show an existing loader as (new) until its next build.
  • Docs: result location; when loader_schema is needed.
  • DuckShop: loaders/loader_results, a task reading two loaders.

Verification

  • e2e, preserve schema, preserve database and --vars schema: two builds keep loader, task and check results in one schema; sqb load reads a loader result there.
  • e2e: S105 refusal; preserved-target load cases with a loader_schema.
  • e2e: results in main, then a mart build of one loader: the task reads both. Fails without fallback.
  • Integration: per-node fallback. Unit: no-default-schema adapter.
  • Targeted suites, fensu, check-ci.

BREAKING CHANGE: a preserve target with Python loaders, tasks, assets or checks now needs loader_schema. Without it, compile fails with S105, which says to set targets.<name>.loader_schema.

Loaders, tasks, assets and checks now write their results and loader
fingerprints to one location for every target: the target's schema, the
target's loader_schema when the schema is preserve, or the adapter's
default schema when the target schema is unset or resolved per model.
Scheduled loaders and build-time checks used the adapter default before,
so adapters without one, such as Snowflake, skipped those writes.

Compile stops with S105 when a preserve target without a loader_schema
has Python nodes.

For one release, a node whose result is missing from the new location is
read from the adapter's default location instead.
Under a preserve target, also read a node's result from the schema named
preserve that earlier releases wrote to, after the adapter default. The
location for `sqb load`, build-time checks and `sqb check` now comes from
the target settings, so each runtime reads and writes the same place.
…r result writes

sqb load now takes its node-result location from the CLI-var-aware loader
defaults, as build does, so a schema set with --vars is read from the same
place both commands write. Build and ingress loaders write their results
through one load operation.
@kvlonge
kvlonge disabled auto-merge November 17, 2026 13:48
@kvlonge
kvlonge enabled auto-merge November 17, 2026 13:55
@kvlonge
kvlonge added this pull request to the merge queue Nov 17, 2026
@kvlonge
kvlonge removed this pull request from the merge queue due to a manual request Nov 17, 2026
@kvlonge
kvlonge added this pull request to the merge queue Nov 17, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Nov 17, 2026
A preserve target with a loader now needs loader_schema (S105), so the
managed-source load cases set one. The source still lands in its own
schema.
@kvlonge kvlonge changed the title fix: keep Python node results in one location per target fix!: keep Python node results in one location per target Nov 17, 2026
BREAKING CHANGE: a preserve target with Python loaders, tasks, assets or
checks now needs loader_schema, which is where their results are kept.
Without it, compile fails with S105, which says to set
targets.<name>.loader_schema.
@chio-labs-delivery
chio-labs-delivery Bot added this pull request to the merge queue Nov 17, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Nov 17, 2026
@kvlonge
kvlonge enabled auto-merge November 17, 2026 16:23
@kvlonge
kvlonge disabled auto-merge November 17, 2026 16:25
@kvlonge
kvlonge added this pull request to the merge queue Nov 17, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Nov 17, 2026
@kvlonge
kvlonge added this pull request to the merge queue Nov 17, 2026
Merged via the queue into main with commit 0fc54fa Nov 17, 2026
34 checks passed
@kvlonge
kvlonge deleted the fix/preserve-node-results branch November 17, 2026 18:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant