Skip to content

A global PromQL aggregate over another aggregate answers one row for an empty input #1348

Closed

Description

@zzylol

Problem

A PromQL aggregate with no grouping keys, over the output of another aggregate, answers one row when its input is empty. Prometheus answers no sample.

Over a gauge m whose values are all below 1,000, exact, planned with plan_installable and run with the executor:

Query Prometheus Today
count(m >= 1000) empty empty
count(sum by (job) (m >= 1000)) empty {} 0
count by (instance) (count by (job, zone, region) (m{job="a"}) > 10) (instance is absent after the inner by, so the outer grouping has no keys) empty {} 0

Probably the same cause (not traced) makes some nested aggregates fail to install instead, with native output type differs from Planner output. For example sum(min without (resource) (kube_resourcequota{type="hard"})) and quantile(0.9, sum without (cpu) (rate(node_cpu_seconds_total{mode="system"}[5m]))). The executor's aggregate nullability reads the same identity-column test (below).

The first two rows were measured on #1242's head before its round-5 fixes, so this is not caused by that PR; its fuzz harness (a PromQL reference evaluator) finds it through without queries that now plan, such as count without (zone) (count without (instance, job) (max by (instance, zone, job) (m{job="a"})) > 4).

Why

The executor decides between SQL and PromQL semantics by whether its input schema has a series identity column:

  • crates/executor/src/operators/summary/mod.rs, the PerGroupSummaryBuild run: emit_empty_global: !input_schema.has_promql_series_identity(). With no groups and that flag, an empty input still gets the global group, as a SQL SELECT count(*) does.
  • crates/executor/src/operators/aggregate/mod.rs (nullability of sum, min, max, avg) and aggregate/columnar.rs use the same test.

An aggregate's output ([job, sum] for sum by (job)) has no identity column, so an aggregate over it is treated as a SQL aggregate over a relation. The IR already knows which it is: OperatorNode::result_kind is InstantVector there and Relation only for SQL tables. The summary merge path already uses it (built_from_relation in crates/executor/src/physical_planner/mod.rs).

Proposal

Decide the SQL-versus-PromQL rule in the lowering, from the IR's result kind, and carry it in the operator:

  • Kind::PerGroupSummaryBuild (and the keyed and all-groups builds) gets the same opt-in the merge has (with_emit_empty_global), set only when the build's input is a Relation.
  • Operator::aggregate's nullability takes the same flag instead of reading the identity column.

Scope and acceptance

  • In scope: global aggregates and their output types over PromQL aggregate outputs.
  • Done when count(sum by (job) (m >= 1000)) answers no sample, sum(min without (resource) (kube_resourcequota)) installs and matches its reference, and SQL SELECT count(*) FROM t WHERE false still answers one row of 0.

Activity

  1. added
    bugSomething isn't working
    promqlPromQL front-end lowering (L1→L2)
    on Nov 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingpromqlPromQL front-end lowering (L1→L2)

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions