You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 8fbfce8
Browse filesBrowse the repository at this point in the historyBrowse files
docs: give maintained models their own page (#1593)
Move maintained models from the materializations page to their own page next to
Incremental and Snapshots: settings, what the plan shows, what a build does, and
what happens when a build dies. The materializations page keeps a short section
like View and Table. Drop the error-code table; A718 and A719 are named where
their situations are explained. Repoint links to the old anchors.
-[Snapshots (SCD Type 2)](concepts/snapshots.md) (`concepts/snapshots`) - Preserve row history over time using SCD Type 2 semantics with timestamp or check-based change detection.
27
+
-[Maintained Models](concepts/models/maintained.md) (`concepts/models/maintained`) - Let the warehouse keep a model up to date on a schedule, paused safely while builds run.
27
28
-[Functions](concepts/functions.md) (`concepts/functions`) - User-defined functions and table functions managed as part of your project.
28
29
-[Macros](concepts/macros.md) (`concepts/macros`) - Reuse SQL with typed SQL macros, or generate it with Python macros when it needs logic.
29
30
-[SQL Macros](concepts/macros/sql-macros.md) (`concepts/macros/sql-macros`) - Write typed, logic-free SQL fragments that the compiler checks and sees inside.
Copy file name to clipboardExpand all lines: src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/build.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -66,7 +66,7 @@ own option. `--version` works only directly after `sqb`, as in `sqb --version`:
66
66
|`--defer-sources-to`| Read managed source data from another target |
67
67
|`--plan-from`| Read warehouse state `live`, or from the target's `synced_state` checked against the warehouse first; overrides the target's `plan_from`. See [Planning from synced state](../concepts/project-configuration.md#planning-from-synced-state)|
68
68
|`--fail-fast`| Stop on first failure and skip remaining nodes |
69
-
|`--ignore-maintained-grace`| Take over [maintained models](../concepts/models/materializations.md#during-a-build) another build holds, without waiting for `[maintained] heartbeat_grace`. Needs `allow_ignore_grace = true`, else fails with `A719`|
69
+
|`--ignore-maintained-grace`| Take over [maintained models](../concepts/models/maintained.md#when-a-build-dies) another build holds, without waiting for `[maintained] heartbeat_grace`. Needs `allow_ignore_grace = true`, else fails with `A719`|
70
70
|`--concurrency`| Number of worker connections (default: 1) |
71
71
|`--test-timeout <duration>`| Time limit for each unit test the build runs, such as `30s`, or `disabled`; see [Time limits](../concepts/testing-reference.md#time-limits)|
72
72
|`--verbose`, `-v`| Show lifecycle SQL inline after each model |
A maintained model is one the warehouse keeps up to date on a schedule you set, between builds. A
10
+
build creates it, changes it when its definition changes, and otherwise leaves the refreshing to
11
+
the warehouse; on Snowflake it is a dynamic table.
12
+
13
+
```sql
14
+
MODEL (
15
+
description "One row per customer per order date",
16
+
materialized maintained,
17
+
refresh (lag 1h),
18
+
);
19
+
20
+
SELECT customer_id, CAST(ordered_at ASDATE) AS order_date, SUM(amount) AS revenue
21
+
FROM __ref("stg_orders")
22
+
GROUP BY customer_id, CAST(ordered_at ASDATE)
23
+
```
24
+
25
+
Only Snowflake supports maintained models. See [Snowflake](../adapters/snowflake.md#maintained-models)
26
+
for the grants and change tracking they need.
27
+
28
+
## Settings
29
+
30
+
| Field | Default | Description |
31
+
|-------|---------|-------------|
32
+
|`refresh (lag <duration>)`| none | How far behind its inputs the model may fall: a fixed duration such as `5m`, `1h` or `1d`, at least `1m` on Snowflake and no shorter than the lag of a maintained model it reads. Without it, the model refreshes when a maintained model that reads it refreshes |
33
+
|`refresh_mode`|`incremental`|`incremental`, `full`, `auto` or `adaptive`. Snowflake rejects `incremental` for a query it can't refresh incrementally; use `full` for that query |
34
+
|`refresh_warehouse`| the connection's warehouse | Warehouse the scheduled refreshes run on |
35
+
|`refresh_on_build`|`false`| Refresh the model in every build that selects it, not only on its schedule |
36
+
37
+
## What the plan shows
38
+
39
+
| Change | Plan | What the build does |
40
+
|--------|------|---------------------|
41
+
| First build, or the dynamic table is missing |`create`| Creates it and fills it |
42
+
| Query or `refresh_mode` changed |`recreate, full recompute`| Replaces it; every row is computed again |
43
+
|`refresh` lag or `refresh_warehouse` differs from the warehouse |`change refresh settings in place`| Alters the schedule; rows stay |
44
+
| Nothing |`maintained`| Leaves it alone |
45
+
46
+
A model the plan keeps can carry one more line:
47
+
48
+
-`recomputes in full: stg_orders is rebuilt`: the build replaces a model it reads, so its next refresh computes every row again.
49
+
-`paused: scheduled refreshes are suspended`: Snowflake lists it as suspended, so it is not refreshing.
50
+
51
+
## During a build
52
+
53
+
A build pauses the maintained models it changes, and those that read a model it rebuilds, directly
54
+
or through views, so they never refresh from half-built inputs. It resumes them when it ends,
55
+
whether it succeeds or fails.
56
+
57
+
- Dynamic tables outside the project that read a paused model pause too, until the build resumes it.
58
+
- A model someone suspended before the build stays suspended, and the build warns about it.
59
+
- A suspend made by someone else during the build is undone when the build resumes the model.
60
+
- A kept model refreshes during the build when a model built later in the same build reads it.
61
+
-`refresh_on_build` refreshes the model in every build that selects it.
62
+
- A target with `pause_maintained = true` leaves its maintained models suspended between builds; see [Targets](../project-configuration.md#targets).
63
+
64
+
## When a build dies
65
+
66
+
-**One build at a time.** A build that needs a maintained model another build holds fails with
67
+
`A718`, naming that run. Wait for it to finish.
68
+
-**A stopped build leaves its models suspended.** A build killed before it resumes them, for
69
+
example by `SIGKILL`, can't resume them itself.
70
+
-**The next build resumes them.** The next build that pauses maintained models on the target
71
+
resumes them once the stopped build has been silent for `[maintained] heartbeat_grace`, 30
72
+
minutes by default; see [Maintained](../project-configuration.md#maintained).
73
+
-**Taking over at once.** Run `sqb build --ignore-maintained-grace` to take the models over
74
+
without waiting. It needs `[maintained] allow_ignore_grace = true`; without it the flag fails
75
+
with `A719`. You can also resume the models in Snowflake by hand.
76
+
77
+
## Limitations
78
+
79
+
- Writing out the default `refresh_mode incremental` recreates the model once.
80
+
- A snapshot rebuilt in full loses change tracking; run `ALTER TABLE <snapshot> SET CHANGE_TRACKING = TRUE` before a maintained model reads it again.
81
+
- Unit tests and scenarios treat a maintained model as a table.
82
+
- On a target with `schema = "preserve"`, builds see each other's pauses only when their maintained models share a schema.
Copy file name to clipboardExpand all lines: src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/models/materializations.md
+3-61Lines changed: 3 additions & 61 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -86,75 +86,17 @@ See [Snapshots](../snapshots.md) for timestamp and check strategies, historical
86
86
87
87
## Maintained
88
88
89
-
The warehouse keeps a maintained model up to date on a schedule you set, between builds. On Snowflake it is a dynamic table; a build creates it, changes it when its definition changes, and otherwise leaves the refreshing to Snowflake.
89
+
The warehouse keeps the model up to date on a schedule you set, and builds pause it while they change its inputs:
90
90
91
91
```sql
92
-
MODEL (
93
-
description "One row per customer per order date",
94
-
materialized maintained,
95
-
refresh (lag 1h),
96
-
);
92
+
MODEL (description "One row per customer per order date", materialized maintained, refresh (lag 1h));
97
93
98
94
SELECT customer_id, CAST(ordered_at ASDATE) AS order_date, SUM(amount) AS revenue
99
95
FROM __ref("stg_orders")
100
96
GROUP BY customer_id, CAST(ordered_at ASDATE)
101
97
```
102
98
103
-
| Field | Default | Description |
104
-
|-------|---------|-------------|
105
-
|`refresh (lag <duration>)`| none | How far behind its inputs the model may fall: a fixed duration such as `5m`, `1h` or `1d`, at least `1m` on Snowflake and no shorter than the lag of a maintained model it reads. Without it, the model refreshes when a maintained model that reads it refreshes |
106
-
|`refresh_mode`|`incremental`|`incremental`, `full`, `auto` or `adaptive`. Snowflake rejects `incremental` for a query it can't refresh incrementally; use `full` for that query |
107
-
|`refresh_warehouse`| the connection's warehouse | Warehouse the scheduled refreshes run on |
108
-
|`refresh_on_build`|`false`| Refresh the model in every build that selects it, not only on its schedule |
109
-
110
-
Only Snowflake supports maintained models. See [Snowflake](../adapters/snowflake.md#maintained-models) for the grants and change tracking they need.
111
-
112
-
### What the plan shows
113
-
114
-
| Change | Plan | What the build does |
115
-
|--------|------|---------------------|
116
-
| First build, or the dynamic table is missing |`create`| Creates it and fills it |
117
-
| Query or `refresh_mode` changed |`recreate, full recompute`| Replaces it; every row is computed again |
118
-
|`refresh` lag or `refresh_warehouse` differs from the warehouse |`change refresh settings in place`| Alters the schedule; rows stay |
119
-
| Nothing |`maintained`| Leaves it alone |
120
-
121
-
A model the plan keeps can carry one more line:
122
-
123
-
-`recomputes in full: stg_orders is rebuilt`: the build replaces a model it reads, so its next refresh computes every row again.
124
-
-`paused: scheduled refreshes are suspended`: Snowflake lists it as suspended, so it is not refreshing.
125
-
126
-
### During a build
127
-
128
-
A build pauses the maintained models it changes, and those that read a model it rebuilds, directly or through views, so they never refresh from half-built inputs. It resumes them when it ends, whether it succeeds or fails.
129
-
130
-
- Dynamic tables outside the project that read a paused model pause too, until the build resumes it. One that someone had suspended before stays suspended.
131
-
- A model that was already suspended when the build started stays suspended, and the build warns about it.
132
-
- If someone suspends a model during the build, the build's resume undoes that suspend.
133
-
- A kept model is refreshed during the build when a model built later in the same build reads it, so that model reads current rows. `refresh_on_build` refreshes it in every build.
134
-
- A target with `pause_maintained = true` leaves its maintained models suspended between builds and refreshes them only in builds; see [Targets](../project-configuration.md#targets).
135
-
136
-
Two builds can't pause the same models at once. A build that needs a model another build holds fails with `A718`, naming that run. When two builds pause a model at the same moment, the one that recorded its pause first keeps it, and the other fails with `A718` without suspending or resuming anything. A build that stops without resuming, for example because its process was killed, leaves its models suspended. The next build that pauses maintained models on the target resumes them once that build has been silent for `[maintained] heartbeat_grace` (see [Maintained](../project-configuration.md#maintained)). Until then, resume them in Snowflake by hand, or run `sqb build --ignore-maintained-grace` where the project allows it.
137
-
138
-
### Error codes
139
-
140
-
| Code | When | Fix |
141
-
|------|------|-----|
142
-
|`A013`|`materialized maintained` on an adapter other than Snowflake | Use `table` or `incremental` on that adapter |
143
-
|`A014`|`materialized materialized_view` on Snowflake | Use `materialized maintained`|
144
-
|`A015`| A `refresh` lag under `1m` on Snowflake | Set a lag of at least `1m`|
145
-
|`A016`| A model with no `refresh` that no maintained model reads, so it would never refresh | Add `refresh (lag 1h)` or another lag |
146
-
|`A716`| The adapter can't build a maintained model | Use an adapter that supports them |
147
-
|`A717`| No warehouse to refresh with | Set `refresh_warehouse`, or a `warehouse` on the connection |
148
-
|`A718`| Another build holds the models this build needs | Wait for it to finish, or for its hold to pass `heartbeat_grace`|
149
-
|`A719`|`--ignore-maintained-grace` without `allow_ignore_grace = true`| Set `[maintained] allow_ignore_grace = true`, or wait for the grace to pass |
150
-
151
-
### Limitations
152
-
153
-
- Writing out the default `refresh_mode incremental` changes the model's definition, so the next build recreates it once.
154
-
- A custom materialization named `maintained` is not used; the built-in one takes the name.
155
-
- Unit tests and scenarios treat a maintained model as a table.
156
-
- On a target with `schema = "preserve"`, builds see each other's pauses only when their maintained models share a schema.
157
-
- A snapshot rebuilt in full is created without change tracking. Before a maintained model reads it again, run `ALTER TABLE <snapshot> SET CHANGE_TRACKING = TRUE`.
99
+
See [Maintained models](maintained.md) for its settings, what the plan shows and what happens when a build dies.
Copy file name to clipboardExpand all lines: src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/project-configuration.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -190,7 +190,7 @@ A target is a named build context - the database and schema you build into, plus
190
190
|`concurrency`| Worker limit for commands run against this target, replacing `[settings] concurrency` (see below) |
191
191
|`ignore_audit_failures`| Whether builds on this target let audit failures through, replacing `[settings] ignore_audit_failures`; see [Ignoring audit failures](audits.md#ignoring-audit-failures)|
192
192
|`plan_from`| Where `sqb plan` and `sqb build` read warehouse state: `live` (default) or `synced_state`; see [Planning from synced state](#planning-from-synced-state)|
193
-
|`pause_maintained`|`true` keeps this target's [maintained models](models/materializations.md#maintained) suspended between builds; each build refreshes the ones it selects. Default `false`|
193
+
|`pause_maintained`|`true` keeps this target's [maintained models](models/maintained.md) suspended between builds; each build refreshes the ones it selects. Default `false`|
194
194
195
195
```toml
196
196
[connections.warehouse]
@@ -699,7 +699,7 @@ old_name_views = "30d"
699
699
700
700
## Maintained
701
701
702
-
How builds pause [maintained models](models/materializations.md#maintained) while they run:
702
+
How builds pause [maintained models](models/maintained.md) while they run:
Copy file name to clipboardExpand all lines: website/src/content/docs/docs/cli/build.mdx
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -49,7 +49,7 @@ own option. `--version` works only directly after `sqb`, as in `sqb --version`:
49
49
|`--defer-sources-to`| Read managed source data from another target |
50
50
|`--plan-from`| Read warehouse state `live`, or from the target's `synced_state` checked against the warehouse first; overrides the target's `plan_from`. See [Planning from synced state](/docs/concepts/project-configuration#planning-from-synced-state)|
51
51
|`--fail-fast`| Stop on first failure and skip remaining nodes |
52
-
|`--ignore-maintained-grace`| Take over [maintained models](/docs/concepts/models/materializations#during-a-build) another build holds, without waiting for `[maintained] heartbeat_grace`. Needs `allow_ignore_grace = true`, else fails with `A719`|
52
+
|`--ignore-maintained-grace`| Take over [maintained models](/docs/concepts/models/maintained#when-a-build-dies) another build holds, without waiting for `[maintained] heartbeat_grace`. Needs `allow_ignore_grace = true`, else fails with `A719`|
53
53
|`--concurrency`| Number of worker connections (default: 1) |
54
54
|`--test-timeout <duration>`| Time limit for each unit test the build runs, such as `30s`, or `disabled`; see [Time limits](/docs/concepts/testing-reference#time-limits)|
55
55
|`--verbose`, `-v`| Show lifecycle SQL inline after each model |
0 commit comments