Skip to content

Range queries: evaluate @ end()/@ start() subtrees once per request and share them #1251

Open

Description

@zzylol

Example

Two PromQL queries over the same metric m:

Query How it runs
Q1 (dashboard) sum_over_time(m[5m]) / sum_over_time(m[5m] @ end()) range query, span 1h, step 1m, requested every 1m
Q2 (alert) sum_over_time(m[5m]) > 100 instant query, every 1m (phase 0)

The denominator of Q1 is pinned to the end of the requested range. At every one of Q1's 61 steps it reads the same rows, (end − 5m, end], so it is one value per request, divided into every step. When Q1's request ends at 12:00, that value is exactly what Q2's sum_over_time(m[5m]) computes when Q2 is evaluated at 12:00.

What should happen: the planner evaluates Q1's @ end() subtree once per request and lets it share Q2's state (or both read one tumbling-window chain). What happens after the #1207 stack: the @ end() subtree is not shared with anything. Its TimeShift stays in the state's unlifted_operators, so its aggregate_coverage definition differs from Q2's and from Q1's own numerator, and Pass 2 offers it no chain.

@ start() has the same shape: in Q1 it pins the subtree to the request's start, end − 1h.

Why it is not done today

Coverage is derived per memo group from its expression only, and a memo group can be read by queries of several evaluation contexts. In a range query, m[5m] @ end() and m[5m] are different computations: the plain one reads different rows at every step; the @ end() one reads the same rows at every step. Coverage cannot express "evaluated once, then repeated across the steps", so the #1207 stack (decided 2026-11-08) lifts @ end() / @ start() into coverage only for instant queries, where they equal the evaluation time and the PromQL front end lowers them as no @. In range queries they stay unlifted, which is correct but shares nothing.

Proposal

  • A new operator RepeatAcrossSteps { child }: child is evaluated once per request, in the context (the query's recurrence, no range evaluation), and its result is repeated for every step of the range evaluation.
  • The PromQL front end lowers X @ end() in a range query as RepeatAcrossSteps(X), with X written as in an instant query. X @ start() is RepeatAcrossSteps(X offset span), since the request starts span before it ends.
  • The subtree below RepeatAcrossSteps is then an ordinary instant-context subtree: it has the same groups and coverage as Q2's sum_over_time(m[5m]), so it shares Q2's state, and Pass 2 can offer it chains with widths from its recurrence's grid.
  • Contexts (Section 5 of the concepts doc): the subtree's consumers run in the instant context of Q1's recurrence; RepeatAcrossSteps runs in Q1's range context.
  • Executor: run the child once per request and broadcast its rows to every step.

Open: whether @ T subtrees in range queries should use the same operator (they also read the same rows at every step).

Scope and acceptance

  • In scope: the operator, PromQL lowering of @ end() / @ start() in range queries, its schema/coverage/context rules, executor support.
  • Out of scope: @ T, SQL.
  • Acceptance: with Q1 and Q2 in one workload, the memo has one state group for sum_over_time(m[5m]) read by Q2 and by the subtree under Q1's RepeatAcrossSteps; executing Q1 with a span of 1h and step 1m returns the same rows as today.

Status and related work

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    CSEenhancementNew feature or requestoptimizerWorkload optimization and plan selectionpromqlPromQL front-end lowering (L1→L2)

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions