Repository navigation
Description
Problem, by example
sum by (job, instance) (rate(http_requests_total[1m])) over a source that declares series_count: 15000 but no distinct_values_per_label:
"sources": {"http_requests_total": {"series_count": 15000, "sample_interval": 15000, "metric_type": "counter"}}Plan selection prices the grouping at min(series_count, rows read) = 15,000 groups, a default for the undeclared distinct counts (group_count_bound, crates/types/src/workload/bounds.rs). The selected plan's cost rests on that number, but the plan output says nothing about it. If the deployment has 50 jobs × 20 instances = 1,000 groups, every grouped state is costed 15x too large, and a sketch plan can lose to exact (or the other way round) without the user knowing which fact to declare.
Today: no warning. Wanted: a warning per undeclared fact the selected plan's cost used, in the same channel as the DDSketch magnitude-range warning (AssumedDataFact, SelectedPlanReport::assumed_data_facts, stage3_plan_selection.warnings), each saying "this affects cost only".
Why it happens
The planner-wide rule is: when a data fact is not declared, the planner uses a written default, and the plan output names that assumption; when the accuracy guarantee depends on it, the warning says the plan may not meet its accuracy target. The first PR for this rule covers the DDSketch magnitude range and the in-order arrival of PromQL samples, the only accuracy-affecting default. The cost model still applies these other defaults without saying so (lines on main at 863153e):
| Site | Default | Undeclared fact |
|---|---|---|
crates/types/src/workload/bounds.rs group_count_bound (.unwrap_or(rows_read)) |
groups = min(declared label counts, series_count, rows read) |
distinct_values_per_label(_set) |
crates/cost/src/node/usage/mod.rs per-series groups, instant selector rows (series.unwrap_or(input.rows)) |
rows read | series_count |
crates/types/src/workload/bounds.rs source_rows_per_ms |
a source's series share of rows_per_second, or all of it |
per-source series_count + sample_interval |
crates/cost/src/node/state_bytes/mod.rs ASSUMED_HEAP_KEY_BYTES = 63, ASSUMED_VALUE_KEY_BYTES = 17 |
three 16-byte labels per heap key | label sizes |
crates/cost/src/node/state_bytes/mod.rs row_bytes |
16 bytes per string, 8 per other value | column widths |
crates/cost/src/model.rs ResourceUnitCost::illustrative, ResourceSpeed::illustrative, frozen_calibration_factor.unwrap_or(1.0) |
illustrative unit costs and speeds | measured unit costs / calibration |
crates/cost/src/plan/cost.rs merge time without a measured merge entry |
one CPU op | measured merge cost |
The missing or partial measured cost table is tracked separately in #843.
Proposal
Add one DataFactAssumption variant per row above and emit it where the default is applied (one place per kind), as NodeCost::assumed_data_facts does for the DDSketch range. Unit costs and speeds are deployment inputs, not data facts. They may fit better as a sibling warning next to #843's. Every variant here returns false from accuracy_target_depends_on_it.
Not a default (no warning needed): an undeclared sign is Unknown and costs the larger two-store DDSketch, a conservative bound rather than an assumed value. Unknown rows_per_state at ingestion time likewise sizes a state for the most it can hold.
Scope and acceptance
In scope: the rows above. For each, a test that a plan resting on the default carries its warning and a plan with the fact declared carries none, and a survey of the 197-workload set and the paper sets showing selected plans unchanged.
Status
Follows the first assumed-fact PR (DDSketch range + in-order arrival). Related: #843 (cost-table warning), #1324 (column_magnitudes).
Activity