fix: make metric_stability run on BigQuery and ClickHouse (follow-up to #1050)

[haritamar]

Summary

Fixes the two CI failures on #1050 (this branch is #1050 + one commit on top; it can be merged as-is or the commit cherry-picked into #1050).

BigQuery – test_metric_stability_detects_restatement_with_weekly_buckets errored with No matching signature for operator >= for argument types: TIMESTAMP, DATE. bigquery__edr_timeadd returns a DATE for week/month/quarter/year parts (it uses date_add(cast(... as date), ...)), so the settled-measurement bound compared a TIMESTAMP column against a DATE:

-updated_at >= {{ edr_timeadd(min_bucket_age.period, min_bucket_age.count, "bucket_end") }}
+updated_at >= {{ edr_cast_as_timestamp(edr_timeadd(min_bucket_age.period, min_bucket_age.count, "bucket_end")) }}

Same pattern as get_start_bucket_in_data. Day-based ages were unaffected, which is why only the weekly test caught it.

ClickHouse – test_metric_stability_ignores_measurements_taken_while_settling issues a plain UPDATE on data_monitoring_metrics, which ClickHouse rejects (Lightweight updates are not supported). On that target the test now runs ALTER TABLE ... UPDATE ... SETTINGS mutations_sync = 1 instead (synchronous so the next run sees the mutated rows).

Verified locally: both tests pass on postgres; the settling test passes on ClickHouse. No BQ credentials locally — relying on CI here.

The remaining test_dbt_artifacts/test_artifacts.py::test_timings failures on the fusion jobs in #1050 are unrelated to this feature (a separate fix landed via devin/1788692120-fusion-timing-datetime).

Link to Devin session: https://app.devin.ai/sessions/2c8e65abe19f4bae97f105d9638d1065
Open in Devin Desktop: https://app.devin.ai/desktop/session/2c8e65abe19f4bae97f105d9638d1065?variant=devin
Requested by: @haritamar

Summary by CodeRabbit

• New Features

  • Added metric stability monitoring to detect unexpected changes in settled historical metric values.
  • Supports comparisons against initial or most recent measurements, configurable change thresholds, time buckets, and multiple metric columns.
  • Retains measurement history to identify gradual drift and later restatements.
  • Excludes unsettled or still-settling measurements and validates supported bucket configurations.

• Tests

  • Added comprehensive integration coverage for detection, thresholds, history retention, time buckets, and adapter-specific behavior.

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[github-actions]

👋 @haritamar
Thank you for raising your pull request.
Please make sure to add tests and document all user-facing changes.
You can do this by editing the docs files in the elementary repository.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Essentials

Run ID: 9a27be39-7f31-4c80-9f2d-8d8008126894

📥 Commits

Reviewing files that changed from the base of the PR and between 92c7c87 and 54b9725.

📒 Files selected for processing (3)

• integration_tests/tests/test_metric_stability.py
• macros/edr/data_monitoring/monitors_query/metric_stability_query.sql
• macros/edr/tests/test_metric_stability.sql

Included review availability: 3 reviews are currently available. Your included PR review attempts over the past 7 days set your current allowance at 5 reviews per hour.

---

📝 Walkthrough

Walkthrough

Adds the metric_stability dbt test and its comparison query. The implementation stores metric history, compares settled buckets against first or last checks, validates time windows, and detects threshold violations. Integration tests cover restatements, drift, history, buckets, settling periods, thresholds, and multiple columns.

Changes

Metric stability detection

Layer / File(s)	Summary
Test contract and observation window macros/edr/tests/test_metric_stability.sql	Adds the metric_stability macro, argument validation, per-column metric collection, history registration, and observation-window resolution.
Settled-bucket comparison query macros/edr/data_monitoring/monitors_query/metric_stability_query.sql	Adds settled-bucket filtering, historical measurement selection, first-check and last-check comparisons, percentage thresholds, and result projection.
Integration coverage for metric behavior integration_tests/tests/test_metric_stability.py	Adds tests for restatements, gradual drift, history retention, weekly buckets, invalid configurations, settling periods, thresholds, and multiple monitored columns.

Estimated code review effort: 4 (Complex) | ~45 minutes

Merge Risk: ⚪ Minimal · up to e4c95

The adapter-specific BigQuery timestamp cast and synchronous ClickHouse mutation address the reported metric_stability failures without a demonstrated merge-blocking risk.

Sequence Diagram(s)

sequenceDiagram
  participant IntegrationTest
  participant MetricStability
  participant MetricStabilityQuery
  participant MetricHistory
  IntegrationTest->>MetricStability: Load rows and run dbt test
  MetricStability->>MetricStabilityQuery: Pass windows, baselines, and thresholds
  MetricStabilityQuery->>MetricHistory: Read persisted measurements
  MetricHistory-->>MetricStabilityQuery: Return bucket versions
  MetricStabilityQuery-->>MetricStability: Return exceeding metrics
  MetricStability-->>IntegrationTest: Report test result

Loading

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 66.67% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 15 functions across 1 files. (2 skipped: …	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies the primary change: making metric_stability run on BigQuery and ClickHouse. It also provides relevant follow-up context.

Full details: Docstring Coverage

Explanation

Docstring coverage is 66.67% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 15 functions across 1 files. (2 skipped: 2 unsupported.)

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch devin/1788703168-metric-stability-bq-clickhouse

---

_{Comment @coderabbitai help to get the list of available commands.}