Skip to content

Commit 1ccf4d9

Browse files
authored
feat(build)!: rename maintained models to dynamic and close their hold gaps (#1594)
* fix(build): close the maintained-model hold gaps and two rebuild residuals - A model created during a build now runs the same earliest-hold check as acquire once its pause is recorded, so two builds creating the same model no longer both hold it. Our create made the model active, so it is suspended either way; the build holding it resumes it at its end, and a build that backed out resumes it itself at its end when no other build holds it, so a model never stays suspended after both back out. - When the losing build cannot close its own pause, the A718 error now names that failure too instead of being replaced by it. - Pauses are ordered by the warehouse clock when each insert starts, but an insert commits later. A build now also backs out for a rival that paused within the time its own insert took, since that rival may have read the table before our pause committed, and when a takeover closed its pause before it read the table back. Two builds that start at the same moment can both stop with A718; neither builds alongside the other. - A snapshot's first build and full rebuild turn change tracking on when a maintained model reads it, as table builds already do (new optional render_track_changes adapter method; Snowflake runs ALTER TABLE ... SET CHANGE_TRACKING = TRUE). - The planner's change check drops refresh_mode when it is the default, on both the recorded and the current side, so writing out or removing the default no longer recreates the model. Stored-status hashes are unchanged. * feat(build)!: rename maintained models to dynamic models The model kind is now called dynamic everywhere: `materialized dynamic`, the `[dynamic]` project section (still `heartbeat_grace` and `allow_ignore_grace`), the `pause_dynamic` target setting, the `--ignore-dynamic-grace` build flag, the `_sqlbuild_dynamic_pauses` state table, plan actions and wording, error text, the Dynamic models docs page and the tests. Error codes are unchanged. BREAKING CHANGE: `materialized maintained`, `[maintained]`, `pause_maintained` and `--ignore-maintained-grace` are no longer accepted; use `materialized dynamic`, `[dynamic]`, `pause_dynamic` and `--ignore-dynamic-grace`. The materialization is part of a model's version identity, so each existing model is recreated once, with a full recompute, by its first build after the rename. Builds record pauses in `_sqlbuild_dynamic_pauses` and no longer read `_sqlbuild_maintained_pauses`: let builds on the previous version finish before upgrading, resume by hand any model a killed older build left suspended, then drop the old table.
1 parent 8fbfce8 commit 1ccf4d9

163 files changed

Lines changed: 1712 additions & 1119 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
//! Error codes of the project-wide dynamic model checks.
2+
3+
/// A dynamic model that nothing would ever refresh.
4+
pub const DYNAMIC_NEVER_REFRESHED_CODE: &str = "A016";
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
pub mod never_refreshed_dynamic;

crates/sqlbuild-analysis/src/assembly/maintained_checks/main/never_refreshed_maintained.rs renamed to crates/sqlbuild-analysis/src/assembly/dynamic_checks/main/never_refreshed_dynamic.rs

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,16 @@
1-
//! Maintained models with no refresh of their own that no maintained model reads.
1+
//! Dynamic models with no refresh of their own that no dynamic model reads.
22
33
use std::collections::HashSet;
44

5-
use crate::assembly::maintained_checks::models::MaintainedModel;
5+
use crate::assembly::dynamic_checks::models::DynamicModel;
66

77
/// The first by name; without a reader or its own refresh, the warehouse never refreshes it.
8-
pub fn never_refreshed_maintained(models: &[MaintainedModel]) -> Option<&str> {
9-
let maintained: HashSet<&str> = models.iter().map(|model| model.name.as_str()).collect();
8+
pub fn never_refreshed_dynamic(models: &[DynamicModel]) -> Option<&str> {
9+
let dynamic: HashSet<&str> = models.iter().map(|model| model.name.as_str()).collect();
1010
let read: HashSet<&str> = models
1111
.iter()
1212
.flat_map(|model| model.reads.iter().map(String::as_str))
13-
.filter(|name| maintained.contains(name))
13+
.filter(|name| dynamic.contains(name))
1414
.collect();
1515
models
1616
.iter()

crates/sqlbuild-analysis/src/assembly/maintained_checks/mod.rs renamed to crates/sqlbuild-analysis/src/assembly/dynamic_checks/mod.rs

File renamed without changes.
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
//! What the project-wide dynamic model checks read.
2+
3+
/// One dynamic model: whether it sets its own refresh, and the models it reads.
4+
#[derive(Debug, Clone, PartialEq, Eq)]
5+
pub struct DynamicModel {
6+
pub name: String,
7+
pub has_refresh: bool,
8+
pub reads: Vec<String>,
9+
}

crates/sqlbuild-analysis/src/assembly/maintained_checks/tests.rs renamed to crates/sqlbuild-analysis/src/assembly/dynamic_checks/tests.rs

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
1+
#[path = "tests/test_dynamic_checks.rs"]
2+
mod dynamic_checks;
13
#[path = "tests/helpers.rs"]
24
mod helpers;
3-
#[path = "tests/test_maintained_checks.rs"]
4-
mod maintained_checks;
55
#[path = "tests/test_types.rs"]
66
mod test_types;
Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
use crate::assembly::dynamic_checks::models::DynamicModel;
2+
3+
/// A dynamic model named `name` reading `reads`.
4+
pub(crate) fn dynamic(name: &str, has_refresh: bool, reads: &[&str]) -> DynamicModel {
5+
DynamicModel {
6+
name: name.to_owned(),
7+
has_refresh,
8+
reads: reads.iter().map(|read| (*read).to_owned()).collect(),
9+
}
10+
}

crates/sqlbuild-analysis/src/assembly/maintained_checks/tests/test_maintained_checks.rs renamed to crates/sqlbuild-analysis/src/assembly/dynamic_checks/tests/test_dynamic_checks.rs

Lines changed: 13 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,49 +1,49 @@
1-
use crate::assembly::maintained_checks::main::never_refreshed_maintained::never_refreshed_maintained;
2-
use crate::assembly::maintained_checks::tests::helpers::maintained;
3-
use crate::assembly::maintained_checks::tests::test_types::NeverRefreshedTestCase;
1+
use crate::assembly::dynamic_checks::main::never_refreshed_dynamic::never_refreshed_dynamic;
2+
use crate::assembly::dynamic_checks::tests::helpers::dynamic;
3+
use crate::assembly::dynamic_checks::tests::test_types::NeverRefreshedTestCase;
44

55
#[test]
6-
fn given_maintained_chains_when_checking_then_unreached_models_without_refresh_are_named() {
6+
fn given_dynamic_chains_when_checking_then_unreached_models_without_refresh_are_named() {
77
let test_cases = [
88
NeverRefreshedTestCase {
99
description: "an upstream model refreshed for its reader",
1010
models: vec![
11-
maintained("clean_orders", false, &["orders"]),
12-
maintained("daily_totals", true, &["clean_orders"]),
11+
dynamic("clean_orders", false, &["orders"]),
12+
dynamic("daily_totals", true, &["clean_orders"]),
1313
],
1414
expected_name: None,
1515
},
1616
NeverRefreshedTestCase {
1717
description: "the last model of a chain without refresh",
1818
models: vec![
19-
maintained("clean_orders", false, &["orders"]),
20-
maintained("daily_totals", false, &["clean_orders"]),
19+
dynamic("clean_orders", false, &["orders"]),
20+
dynamic("daily_totals", false, &["clean_orders"]),
2121
],
2222
expected_name: Some("daily_totals"),
2323
},
2424
NeverRefreshedTestCase {
2525
description: "a model read only by a table, which never asks for a refresh",
26-
models: vec![maintained("clean_orders", false, &["orders"])],
26+
models: vec![dynamic("clean_orders", false, &["orders"])],
2727
expected_name: Some("clean_orders"),
2828
},
2929
NeverRefreshedTestCase {
3030
description: "two unread models report the first by name",
3131
models: vec![
32-
maintained("b_totals", false, &["orders"]),
33-
maintained("a_totals", false, &["orders"]),
32+
dynamic("b_totals", false, &["orders"]),
33+
dynamic("a_totals", false, &["orders"]),
3434
],
3535
expected_name: Some("a_totals"),
3636
},
3737
NeverRefreshedTestCase {
3838
description: "every model sets its own refresh",
39-
models: vec![maintained("daily_totals", true, &[])],
39+
models: vec![dynamic("daily_totals", true, &[])],
4040
expected_name: None,
4141
},
4242
];
4343

4444
for test_case in test_cases {
4545
assert_eq!(
46-
never_refreshed_maintained(&test_case.models),
46+
never_refreshed_dynamic(&test_case.models),
4747
test_case.expected_name,
4848
"{}",
4949
test_case.description

crates/sqlbuild-analysis/src/assembly/maintained_checks/tests/test_types.rs renamed to crates/sqlbuild-analysis/src/assembly/dynamic_checks/tests/test_types.rs

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
1-
use crate::assembly::maintained_checks::models::MaintainedModel;
1+
use crate::assembly::dynamic_checks::models::DynamicModel;
22

33
pub(crate) struct NeverRefreshedTestCase {
44
pub(crate) description: &'static str,
5-
pub(crate) models: Vec<MaintainedModel>,
5+
pub(crate) models: Vec<DynamicModel>,
66
pub(crate) expected_name: Option<&'static str>,
77
}

‎crates/sqlbuild-analysis/src/assembly/maintained_checks/constants.rs‎

Lines changed: 0 additions & 4 deletions
This file was deleted.

0 commit comments

Comments
 (0)