Skip to content

Plan output: warn about each cost-only data fact the selected plan assumed (group counts, rows per source, key and row widths, unit costs) #1364

Open

Description

@zzylol

Problem, by example

sum by (job, instance) (rate(http_requests_total[1m])) over a source that declares series_count: 15000 but no distinct_values_per_label:

"sources": {"http_requests_total": {"series_count": 15000, "sample_interval": 15000, "metric_type": "counter"}}

Plan selection prices the grouping at min(series_count, rows read) = 15,000 groups, a default for the undeclared distinct counts (group_count_bound, crates/types/src/workload/bounds.rs). The selected plan's cost rests on that number, but the plan output says nothing about it. If the deployment has 50 jobs × 20 instances = 1,000 groups, every grouped state is costed 15x too large, and a sketch plan can lose to exact (or the other way round) without the user knowing which fact to declare.

Today: no warning. Wanted: a warning per undeclared fact the selected plan's cost used, in the same channel as the DDSketch magnitude-range warning (AssumedDataFact, SelectedPlanReport::assumed_data_facts, stage3_plan_selection.warnings), each saying "this affects cost only".

Why it happens

The planner-wide rule is: when a data fact is not declared, the planner uses a written default, and the plan output names that assumption; when the accuracy guarantee depends on it, the warning says the plan may not meet its accuracy target. The first PR for this rule covers the DDSketch magnitude range and the in-order arrival of PromQL samples, the only accuracy-affecting default. The cost model still applies these other defaults without saying so (lines on main at 863153e):

Site Default Undeclared fact
crates/types/src/workload/bounds.rs group_count_bound (.unwrap_or(rows_read)) groups = min(declared label counts, series_count, rows read) distinct_values_per_label(_set)
crates/cost/src/node/usage/mod.rs per-series groups, instant selector rows (series.unwrap_or(input.rows)) rows read series_count
crates/types/src/workload/bounds.rs source_rows_per_ms a source's series share of rows_per_second, or all of it per-source series_count + sample_interval
crates/cost/src/node/state_bytes/mod.rs ASSUMED_HEAP_KEY_BYTES = 63, ASSUMED_VALUE_KEY_BYTES = 17 three 16-byte labels per heap key label sizes
crates/cost/src/node/state_bytes/mod.rs row_bytes 16 bytes per string, 8 per other value column widths
crates/cost/src/model.rs ResourceUnitCost::illustrative, ResourceSpeed::illustrative, frozen_calibration_factor.unwrap_or(1.0) illustrative unit costs and speeds measured unit costs / calibration
crates/cost/src/plan/cost.rs merge time without a measured merge entry one CPU op measured merge cost

The missing or partial measured cost table is tracked separately in #843.

Proposal

Add one DataFactAssumption variant per row above and emit it where the default is applied (one place per kind), as NodeCost::assumed_data_facts does for the DDSketch range. Unit costs and speeds are deployment inputs, not data facts. They may fit better as a sibling warning next to #843's. Every variant here returns false from accuracy_target_depends_on_it.

Not a default (no warning needed): an undeclared sign is Unknown and costs the larger two-store DDSketch, a conservative bound rather than an assumed value. Unknown rows_per_state at ingestion time likewise sizes a state for the most it can hold.

Scope and acceptance

In scope: the rows above. For each, a test that a plan resting on the default carries its warning and a plan with the fact declared carries none, and a survey of the 197-workload set and the paper sets showing selected plans unchanged.

Status

Follows the first assumed-fact PR (DDSketch range + in-order arrival). Related: #843 (cost-table warning), #1324 (column_magnitudes).

Activity

  1. added
    cost-modelCost formulas, statistics, ranking, and selection
    enhancementNew feature or request
    on Nov 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    cost-modelCost formulas, statistics, ranking, and selectionenhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions