Skip to content

Executor: subqueries over expressions other than a selector or a per-series range function (step-grid evaluator) #1248

Open

Description

@zzylol

Problem, by example

A PromQL subquery whose inner expression is not a selector or a per-series range function of one is refused by the executor, so plan_installable fails for it:

Query (exact) Today Expected
max_over_time(sum(rate(m[1m]))[10m:1m]) installable plan: the executor cannot compile the plan's query run: … a subquery over an expression other than a selector or a per-series range function of one has no native implementation the largest of sum(rate(m[1m])) evaluated at the multiples of 1 minute in (T - 10m, T]

Nested subqueries (max_over_time(deriv(rate(x[5s])[30s:5s])[10m:]), the Prometheus docs example) need the same evaluator; today that query fails earlier, in plan selection (the selection MILP is infeasible).

Shapes that run since the time-operators PR : f(m[r])[R:s] and m[R:s] (with offset/@ on the subquery or its selector), where the inner function is a per-entity Aggregate or an exact accumulator's FinalizeExactAccumulator(SummaryAgg) over a TimeRange of a Scan.

Why it happens

physical_planner::subquery (crates/executor/src/physical_planner/mod.rs) lowers a subquery to one Operator::series_window with steps over the inner selector's Scan: it recognizes only an instant TimeRange, a per-entity range function over a range TimeRange, or an exact accumulator over one. Anything else (a cross-series aggregate, a binary operation, another subquery) would need the inner sub-DAG evaluated at every step of the grid, and the runtime runs a DAG at one evaluation time (RunContext has one Scope::Query { evaluation_time_ms }).

Proposal

A step-grid evaluator: an operator that holds the compiled inner sub-DAG and runs it once per grid step with a RunContext at that step (the steps are bounded by MAX_SUBQUERY_STEPS), stamping each result row with its step time. The current series_window lowering stays as the fast path for selectors and per-series range functions.

Alternative: rewrite common shapes (e.g. sum(rate(m[1m]))[R:s] as a grouped sum over rate(m[1m])[R:s]) in the frontend. It covers fewer shapes.

Scope and acceptance

  • max_over_time(sum(rate(m[1m]))[10m:1m]) and the nested docs example plan and run exactly, checked against the values Prometheus' definition gives (planner e2e test as in crates/planner/tests/time_operators.rs).
  • Out of scope: approximate (sketch) evaluation of subqueries.

Related

Executor gap 25 (PromQL subqueries) of the executor gap inventory; the time-operators PR (branch exec/x4-time-operators).

Activity

  1. zzylol commented on Nov 15, 2026

    @zzylol
    ContributorAuthor

    The time-operators PR (#1250) is closed as superseded, so the "Shapes that run since the time-operators PR" above do not run on main (59365ce80). The physical_planner::subquery lowering named under "Why it happens" is not on main either. Today every subquery, including max_over_time(c[10m:1m]) and max_over_time(rate(c[1m])[10m:1m]), fails at install with invalid DAG: … value operation has no native implementation.

    The selector / per-series range function fast path is now #1318, which reads the subquery window from the edge read_window (#1291/#1292). This issue's step-grid evaluator builds on #1318.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions