Skip to content

count by (job) (count by (job, host) (m)): the outer exact Count over a finalized count does not compile #1289

Open

Description

@zzylol

Example

A PromQL query that counts, per job, the (job, host) groups of a gauge, every 60 s (relative error 0.05, failure probability 0.01):

count by (job) (count by (job, host) (m))
  • Today: Stage 3 selects (all at query time) 0 Scan(m) → 1 TimeRange(1s, instant) → 2 SummaryAgg exact Count (OneEvaluation) → 3 FinalizeExactAccumulator → 4 SummaryAgg exact Count (OneEvaluation) over the finalized counts → 5 FinalizeExactAccumulator. Installing it fails when the executor compiles the query run: QueryRunNotExecutable { reason: "invalid DAG: node 4: invalid DAG: summary update column missing or ambiguous" } (observed with the review harness on the Stage 3 baseline branch).
  • Expected: the plan installs and answers like the exact query: per job, the number of (job, host) groups.

Why

Pass 1 offers the outer count as a SummaryAgg over the finalized inner count. The finalized inner count has the group-key columns and one value column (the count). The outer count counts rows, so its summary update reads no value column. The executor's summary build (summary_agg_build in crates/executor/src/physical_planner/) looks for the update's column in the input schema and finds none or several, so it refuses the node.

Which side is wrong is open: either Pass 1 should express a row count over a finalized accumulator with an update the executor accepts (for example a constant weight), or the executor should build a Count whose update names no column.

Proposal

  1. Decide the representation of "count the rows of a finalized accumulator" in SummaryUpdate.
  2. Make Pass 1 emit it, and the executor compile it.

Scope and acceptance

Acceptance: count by (job) (count by (job, host) (m)) plans, installs and answers equal to the exact count, as a planner integration test that fails today.

Related

Found by the functional review of the Stage 3 baseline (#1236, PR 6 of #1207), workload n18.

Activity

  1. added
    bugSomething isn't working
    promqlPromQL front-end lowering (L1→L2)
    on Nov 12, 2026
  2. zzylol commented on Nov 15, 2026

    @zzylol
    ContributorAuthor

    Still open after #1355. With #1355's measured unit costs, Stage 3 selects the plain exact plan for this query (Scan → TimeRange(1s, instant) → Aggregate by (job, host) → Aggregate by (job)), which installs and answers exactly (planned/exact query CPU 1.03×, run at #1355's head 008704da2). That only steers away from the failing plan: Pass 1 still offers the outer Count SummaryAgg over the finalized inner count, and the executor still refuses it with "summary update column missing or ambiguous" whenever that candidate is the cheapest (for example under a measured table that prices exact accumulators lower). So #1355 does not fix this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingpromqlPromQL front-end lowering (L1→L2)

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions