Skip to content

Commit b625100

Browse files
authored
fix(scenario): honour promotion mode and run scenarios concurrently (#984)
* fix(scenario): honour table promotion mode and enforced contracts Scenario models were always materialized with immediate table promotion, so any table model with an enforced contract failed with K011 and declared type enforcement failed with R004 even when the project configured staged promotion. Scenarios now resolve the same promotion mode as build (project table_promotion_mode, else the adapter default) and use staged promotion whenever a model's contract is enforced, so the runtime contract check runs and the scenario proceeds to its expectations. Cleanup also drops the staging relation a failed staged promotion keeps for inspection. * perf(scenario): run scenarios concurrently with fewer statements Scenario runs issued a blind DROP IF EXISTS for every planned relation both before and after each scenario, ensured the destination schema before every fixture and model, and ran strictly one scenario at a time. - Cleanup reads the catalog once per database and drops only the scenario-owned relations that exist, using the catalogued relation type, so a retained relation whose materialization changed is still replaced. When the catalog cannot be read, cleanup falls back to the previous blind drops. - Scenario destination schemas are ensured once per invocation. - An interrupt or unexpected error inside a scenario still drops that scenario's relations before propagating (unless --retain). - Independent scenarios run concurrently up to --concurrency or the project concurrency setting, each on its own connection; results are rendered in selection order. Scenario artifacts are already namespaced per scenario. - Text output reports each scenario's wall time and the total; --json adds scenarios[].duration_ms and summary.duration_ms. Nested model progress rows are labelled with their scenario so concurrent rows stay unambiguous. * fix(scenario): allow __empty_fixture() in scenario fixtures Scenario planning already expands SELECT * FROM __empty_fixture() into the relation's declared, typed columns, as unit tests do, but the SQL Rules engine only exempted the marker in TEST files, so scenarios using it failed SQBRSQL021. Scenario files now receive the same exemption. * fix(scenario): fit auxiliary relation names within the identifier limit Staging, type-enforcement, delta and snapshot-delta relations appended a suffix to the target name without fitting the result. On Postgres (63-char identifiers) a long model name truncated the staging name to the target name, so staged promotion failed with R007, and slightly shorter names leaked a truncated staging table that cleanup could not find. Every auxiliary name now goes through fit_auxiliary_relation_name, which keeps the suffix and replaces the overflow with a stable hash. Build, load and scenario cleanup all use the same fitted names. * fix(scenario): drop targets whose catalog listing is not authoritative Adapters now declare whether list_relations covers a scope. Cleanup lists only authoritative scopes, matches schema and name case-insensitively, and falls back to DROP IF EXISTS for every other target, including the type-enforcement relation. * fix(scenario): stop concurrent scenarios cleanly on interrupt and match build K011 The first Ctrl-C prints one notice, cancels queued scenarios and stops running ones at their next step boundary so they clean up before the connections close; repeat presses are absorbed. Enforced contracts with immediate promotion now fail with K011 exactly as sqb build does. * fix(contracts): report K011 promotion conflict before execution An enforced contract on a table or non-microbatch incremental model under immediate table promotion is now a compile diagnostic, so compile, plan, build and scenario test (including --local) fail before any connection or statement. The help names the file and current value, shows the exact staged TOML, and offers removing the enforced contract. Build and plan failures now carry the help when every error shares it. The run-time guard stays as a defensive check for paths the compile check does not cover. * fix(scenario): cancel in-flight statements on Ctrl-C and fit names by bytes Running every scenario through a worker pool and absorbing repeat interrupts made long statements uninterruptible, even at concurrency 1. Concurrency 1, --local replay and capture now run on the main thread again, so Ctrl-C reaches the driver as before. A SIGINT handler records the stop and cancels in-flight statements through the new adapter interrupt_connection (DuckDB interrupt, Postgres cancel_safe); without it a DuckDB query interrupted only by the signal kept running in the background and blocked cleanup. Concurrent runs cancel queued scenarios, cancel or close in-flight worker connections, wait for teardown, and sweep started scenarios on a valid or fresh connection. A second Ctrl-C prints one notice and exits at once; daemon workers never block exit, and the next run of the same scenarios drops what was left behind. Identifier fitting now measures UTF-8 bytes and cuts on character boundaries, so non-ASCII names fit Postgres's 63-byte limit.
1 parent 06fa96d commit b625100

86 files changed

Lines changed: 3527 additions & 332 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎Makefile‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -121,7 +121,11 @@ E2E_DUCKDB_CLI_DATA_PATHS := \
121121
tests/e2e/src/sqlbuild/cli/commands/main/scenario/test_authored_check_sql.py \
122122
tests/e2e/src/sqlbuild/cli/commands/main/scenario/test_scenario.py \
123123
tests/e2e/src/sqlbuild/cli/commands/main/scenario/test_capture.py \
124-
tests/e2e/src/sqlbuild/cli/commands/main/scenario/test_namespace.py
124+
tests/e2e/src/sqlbuild/cli/commands/main/scenario/test_namespace.py \
125+
tests/e2e/src/sqlbuild/cli/commands/main/scenario/test_promotion.py \
126+
tests/e2e/src/sqlbuild/cli/commands/main/scenario/test_concurrency.py \
127+
tests/e2e/src/sqlbuild/cli/commands/main/scenario/test_interrupts.py \
128+
tests/e2e/src/sqlbuild/cli/commands/main/scenario/test_empty_fixture.py
125129

126130
E2E_DUCKDB_CLI_PATHS := \
127131
tests/e2e/scripts/cli_preview \

‎fensu.toml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,12 @@ path = "src/sqlbuild/compiler/planner/_helpers/warehouse/source_columns.py"
9999
symbols = ["gather_source_inspection"]
100100
reason = "Each type-enforced expression source is its own query whose output columns can only be read by running it; table sources are listed in one batch per database."
101101

102+
[[targets.python.rule_exceptions]]
103+
rule = "XSB051"
104+
path = "src/sqlbuild/executor/scenario/_helpers/lifecycle/cleanup.py"
105+
symbols = ["existing_scenario_cleanup_targets"]
106+
reason = "Scenario cleanup lists every scenario-owned relation in one batch per database instead of dropping each planned relation blindly."
107+
102108
[[targets.python.rule_exceptions]]
103109
rule = "XSB051"
104110
path = "src/sqlbuild/executor/migrations/_helpers/columns.py"

‎src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/scenario.md‎

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@ sqb scenario test [flags]
3232
| `--scenario-namespace NAMESPACE` | Isolate warehouse artifacts for this run; overrides environment and config |
3333
| `--local` | Run locally against DuckDB using captured JSONL snapshots |
3434
| `--strict` | Treat missing/stale local snapshots as errors instead of skips |
35+
| `--concurrency N` | Run up to `N` scenarios at once (default: the project `concurrency` setting) |
3536
| `--sync-snapshots` | Capture missing/stale snapshots before local run (requires `--local`) |
3637
| `--refresh` | Recapture all selected snapshots before local run (requires `--local`) |
3738
| `--force` | Bypass snapshot capture safety limits |
@@ -93,17 +94,20 @@ sqb scenario test --local --strict
9394
Remote scenarios report per-scenario PASS/FAIL with nested check rows:
9495

9596
```
96-
daily_revenue_minimal PASS
97+
daily_revenue_minimal PASS 0.84s
9798
expect expected daily_revenue PASS
9899
expect assertion no_negative_revenue PASS
99100
100-
PASS=1 FAIL=0 TOTAL=1
101+
PASS=1 FAIL=0 TOTAL=1 (0.91s)
101102
```
102103

104+
Each scenario row shows its wall time and the summary shows the total. `--json` reports the same
105+
values as `scenarios[].duration_ms` and `summary.duration_ms`.
106+
103107
Local scenarios add ERROR and SKIP statuses:
104108

105109
```
106-
PASS=2 FAIL=0 ERROR=0 SKIP=1 TOTAL=3
110+
PASS=2 FAIL=0 ERROR=0 SKIP=1 TOTAL=3 (1.20s)
107111
```
108112

109113
Failed scenarios suggest rerunning with `--retain` for inspection. Local runs always keep the DuckDB file at `target/run/scenarios/<scenario_name>/local.duckdb`.

‎src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/models/materializations.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ FROM __ref("stg_orders")
3333
GROUP BY customer_id
3434
```
3535

36-
Projects may opt into `settings.table_promotion_mode = "immediate"`. Immediate promotion replaces the destination before audits, so a failed audit does not restore the old table. Immediate promotion rejects models that require declared-type enforcement or `contract enforced`; use staged promotion for those guarantees.
36+
Projects may opt into `settings.table_promotion_mode = "immediate"`. Immediate promotion replaces the destination before audits, so a failed audit does not restore the old table. Immediate promotion rejects models that require declared-type enforcement or `contract enforced`; use staged promotion for those guarantees. A table or incremental model with `contract enforced` under immediate promotion is a `K011` compile error, so `sqb compile`, `sqb plan`, `sqb build`, and `sqb scenario test` report it before any warehouse work.
3737

3838
## Incremental
3939

‎src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/scenarios.md‎

Lines changed: 25 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -106,6 +106,10 @@ SELECT 1
106106

107107
For sources with two-part identity, use double underscores: `__source__raw__orders`.
108108

109+
To mock a contracted source, ref, or seed with no rows, use `SELECT * FROM __empty_fixture()` as
110+
the complete fixture body. SQLBuild expands it to the relation's declared, typed columns with a
111+
false filter, as in [unit tests](testing.md).
112+
109113
Every scenario must have at least one fixture CTE and at least one `__expected__` or `__assert__` CTE.
110114

111115
Scenario SQL uses macros, enums, and constants available from the scenario file's directory under
@@ -145,13 +149,32 @@ Scenario artifacts are physically isolated from production:
145149

146150
### Execution flow
147151

148-
1. Clean any existing scenario artifacts from a previous run in the effective run namespace
152+
1. Drop any scenario artifacts a previous run left in the effective run namespace (one catalog read finds them)
149153
2. Materialize source, ref, and seed fixtures as physical tables
150154
3. Load required project seeds
151155
4. Build required models in dependency order (incremental models run as full-refresh in scenarios)
152156
5. Run expected-output comparisons (order-insensitive)
153157
6. Run zero-row assertions
154-
7. Clean up this namespace's scenario-owned artifacts (unless `--retain`)
158+
7. Clean up this namespace's scenario-owned artifacts (unless `--retain`), including after a failure or Ctrl-C
159+
160+
Table models use the same promotion as `sqb build`: the project's `table_promotion_mode`, or the
161+
adapter default (staged) when it is unset. An enforced contract needs staged promotion, so its
162+
runtime contract check runs before the scenario reaches its expectations; with
163+
`table_promotion_mode = "immediate"` such a model is a `K011` compile error, reported before any
164+
scenario runs, exactly as in `sqb compile`, `sqb plan`, and `sqb build`.
165+
166+
Independent scenarios run concurrently, up to `--concurrency` or the project `concurrency`
167+
setting. Each scenario writes only its own prefixed relations, so concurrent scenarios never
168+
share an artifact. Results are reported in selection order with each scenario's wall time, and the
169+
summary includes the total time. On DuckDB, concurrent scenarios use separate connections to the
170+
same database file; an in-memory database (`:memory:`) gives each connection its own database, so
171+
fixtures still isolate every scenario.
172+
173+
On Ctrl-C, SQLBuild prints one notice, cancels scenarios that have not started, cancels the
174+
statements running scenarios have in flight, drops their relations (unless `--retain`), and only
175+
then exits. A second Ctrl-C skips that cleanup and exits immediately; the next run of the same
176+
scenarios drops whatever they left behind. `--local` replay and `sqb scenario capture` run one
177+
scenario at a time.
155178

156179
### Parallel CI runs
157180

‎src/sqlbuild/adapter/contract/classes/base_adapter.py‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -149,6 +149,18 @@ def supports_unqualified_function_fingerprints(self) -> bool:
149149
def supports_table_functions(self) -> bool:
150150
return False
151151

152+
def interrupt_connection(self, connection: Any) -> bool:
153+
"""Cancel a connection's running statement from another thread; False if unsupported."""
154+
155+
del connection
156+
return False
157+
158+
def lists_relations_authoritatively(self, *, database: str | None, schema: str) -> bool:
159+
"""Return whether ``list_relations`` reports every relation in this scope."""
160+
161+
del database, schema
162+
return False
163+
152164
def supports_concurrent_microbatch_dml(self) -> bool:
153165
"""Return whether disjoint same-target delete/insert batches may run concurrently."""
154166

‎src/sqlbuild/adapter/contract/classes/duckdb_backed_adapter.py‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -112,6 +112,18 @@ def get_columns_for_relations(
112112
)
113113
relation_grants_supported: ClassVar[bool] = False
114114

115+
def interrupt_connection(self, connection: Any) -> bool:
116+
"""Cancel a connection's running statement from another thread; False if unsupported."""
117+
118+
connection.interrupt()
119+
return True
120+
121+
def lists_relations_authoritatively(self, *, database: str | None, schema: str) -> bool:
122+
"""Return whether ``list_relations`` reports every relation in this scope."""
123+
124+
del database, schema
125+
return True
126+
115127
def supports_zero_copy_clone(self) -> bool:
116128
return False
117129

‎src/sqlbuild/adapter/contract/classes/strict_adapter.py‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -101,6 +101,16 @@ def maximum_identifier_length(self) -> int:
101101
"""Return the maximum unqualified identifier length supported by the adapter."""
102102
...
103103

104+
@abstractmethod
105+
def interrupt_connection(self, connection: Any) -> bool:
106+
"""Cancel a connection's running statement from another thread; False if unsupported."""
107+
...
108+
109+
@abstractmethod
110+
def lists_relations_authoritatively(self, *, database: str | None, schema: str) -> bool:
111+
"""Return whether ``list_relations`` reports every relation in this scope."""
112+
...
113+
104114
@abstractmethod
105115
def supports_concurrent_microbatch_dml(self) -> bool:
106116
"""Return whether disjoint delete/insert microbatches may execute concurrently."""

‎src/sqlbuild/adapter/relations/_helpers/identifier_fitting.py‎

Lines changed: 32 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
"""Identifier-limit fitting implementations for generated relation names."""
1+
"""Fit generated relation names within identifier limits measured in UTF-8 bytes."""
22

33
from __future__ import annotations
44

@@ -13,13 +13,13 @@ def fit_artifact_logical_name_impl(
1313
) -> str:
1414
"""Fit a readable logical component with a deterministic hash suffix."""
1515

16-
max_logical_length: int = identifier_limit - len(fixed_prefix)
16+
max_logical_length: int = identifier_limit - _byte_length(fixed_prefix)
1717
if max_logical_length < 1:
1818
raise SharedInputError(
1919
f"{artifact_label} prefix '{fixed_prefix}' does not fit within identifier "
2020
f"limit {identifier_limit}"
2121
)
22-
if len(logical_name) <= max_logical_length:
22+
if _byte_length(logical_name) <= max_logical_length:
2323
return logical_name
2424

2525
suffix_length: int = SHORTENED_LOGICAL_NAME_HASH_LENGTH + 1
@@ -32,4 +32,32 @@ def fit_artifact_logical_name_impl(
3232
:SHORTENED_LOGICAL_NAME_HASH_LENGTH
3333
]
3434
prefix_length: int = max_logical_length - suffix_length
35-
return f"{logical_name[:prefix_length]}_{logical_hash}"
35+
return f"{_truncate_bytes(value=logical_name, max_bytes=prefix_length)}_{logical_hash}"
36+
37+
38+
def fit_auxiliary_relation_name_impl(*, base_name: str, suffix: str, identifier_limit: int) -> str:
39+
"""Append a suffix, shortening the base with a stable hash when the whole name is too long."""
40+
41+
if _byte_length(base_name) + _byte_length(suffix) <= identifier_limit:
42+
return f"{base_name}{suffix}"
43+
hash_part: str = (
44+
"_"
45+
+ hashlib.sha256(base_name.encode("utf-8")).hexdigest()[:SHORTENED_LOGICAL_NAME_HASH_LENGTH]
46+
)
47+
kept_length: int = identifier_limit - _byte_length(suffix) - len(hash_part)
48+
if kept_length < 1:
49+
raise SharedInputError(
50+
f"auxiliary relation suffix '{suffix}' for '{base_name}' cannot fit within "
51+
f"identifier limit {identifier_limit}"
52+
)
53+
return f"{_truncate_bytes(value=base_name, max_bytes=kept_length)}{hash_part}{suffix}"
54+
55+
56+
def _byte_length(value: str) -> int:
57+
return len(value.encode("utf-8"))
58+
59+
60+
def _truncate_bytes(*, value: str, max_bytes: int) -> str:
61+
"""Keep the longest prefix whose UTF-8 encoding fits, never splitting a character."""
62+
63+
return value.encode("utf-8")[:max_bytes].decode("utf-8", errors="ignore")
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
"""Auxiliary relation name fitting entrypoint."""
2+
3+
from __future__ import annotations
4+
5+
from sqlbuild.adapter.relations._helpers.identifier_fitting import (
6+
fit_auxiliary_relation_name_impl,
7+
)
8+
9+
10+
def fit_auxiliary_relation_name(*, base_name: str, suffix: str, identifier_limit: int) -> str:
11+
"""Return ``base_name + suffix`` fitted within the limit, keeping the suffix and a base hash."""
12+
13+
return fit_auxiliary_relation_name_impl(
14+
base_name=base_name, suffix=suffix, identifier_limit=identifier_limit
15+
)

0 commit comments

Comments
 (0)