Skip to content

fix!: turn silent fallbacks into clear errors - #1473

Merged
kvlonge merged 12 commits into
mainfrom
fix/silent-fallbacks
Nov 14, 2026
Merged

kvlonge merged 12 commits into
mainfrom
fix/silent-fallbacks

Conversation

@kvlonge

@kvlonge kvlonge commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Why

Five paths fell back silently to something weaker than the docs promise: keyed cursor writes duplicated rows, loader decimals lost precision, keep_in_use archived without usage, Databricks freshness switched every source to DDL time, and locations fell back to the connection.

Changes

  • delete_insert with a cursor and unique_key deletes rows in the window or whose key is in the delta, then inserts. Snowflake stages outside its transaction, BigQuery uses one MERGE, Databricks DELETE then INSERT.
  • Loader Decimal values become DECIMAL, written exactly; BigQuery widens to BIGNUMERIC.
  • keep_in_use stops with C503 when usage is missing or partial.
  • Databricks freshness marks only failing sources unknown; connection errors raise.
  • Locations, shared by models, loaders, janitor, cost, debug and diff: a lone top-level [connection] is the default target; with named connections or targets, the target then [defaults], never the connection. S101/S103/S104 and sqb load name an ignored connection value and the line to add.

BREAKING CHANGE: tools no longer read a named connection's location; with [defaults] set and no target value, loader tables move there. An inline [connection] now places models on Snowflake, BigQuery and Databricks, where 0.223.0 gave S101. With named connections or targets, a connection schema no longer places models: such projects fail with S104 or S103 naming it. Undeclared loader columns with unsupported, mixed or widening decimal values fail. keep_in_use stops with C503 when usage is incomplete.

Verification

  • Each fix has a test that fails on main, including inline schema, inline database, ignored connection locations, and models and loaders agreeing.
  • Targeted unit, integration and DuckDB E2E tests and make check-ci pass.
  • DuckShop gained a keyed cursor edge model. Location rules are project configuration, not a model shape, so DuckShop is unchanged for them.
  • Warehouse SQL is unit-tested only.

Cursor delete_insert with a unique_key also replaces rows whose key is in the delta.
Python loaders type Decimal values as DECIMAL and reject unrecognised types.
keep_in_use stops when usage can't be read in full.
Databricks freshness observes each source on its own.
Project-level target locations resolve like models' locations.

Not yet reviewed; make check-ci not yet run.
@kvlonge kvlonge added the full-verify Run the full Verify matrix on this pull request label Nov 14, 2026
@kvlonge
kvlonge enabled auto-merge November 14, 2026 06:11
@kvlonge
kvlonge disabled auto-merge November 14, 2026 08:21
…a check

A managed source's loader writes to the source's own schema or the
target's loader_schema, so an unused connection schema no longer
reports it as an intermediate loader without a schema.

Two older tests follow the connection location rules: debug reports no
database for a lone named connection, and the Snowflake early-failure
case sets no database anywhere, since a lone top-level connection now
supplies one.
@kvlonge
kvlonge enabled auto-merge November 14, 2026 08:50
@kvlonge
kvlonge disabled auto-merge November 14, 2026 09:07
@kvlonge
kvlonge added this pull request to the merge queue Nov 14, 2026
Merged via the queue into main with commit b4b19c8 Nov 14, 2026
44 checks passed
@kvlonge
kvlonge deleted the fix/silent-fallbacks branch November 14, 2026 10:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full-verify Run the full Verify matrix on this pull request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant