Skip to content

Range queries with @ anchors cannot be executed or verified #1306

Open

Description

@zzylol

Problem, by example

Workload h02b: two range queries, both every 120 s at phase 0, step 60 s, with spans 180 s and 420 s. Data: 1,000 series of m, 1 sample/s, raw data kept.

sum_over_time(m[1m] @ start())

At a request time T, query 0 must return, at each of its steps, sum_over_time(m[1m]) evaluated at T − 180 s. Query 1 must return the same expression evaluated at T − 420 s.

The planner fixes for @ (#1294, and #1237's request contexts) are checked only by objective = plan_cost today. None of them can be confirmed by execution until this works.

Why it doesn't work today

  1. Install. crates/executor/src/physical_planner/mod.rs:614-622 requires a per-entity SummaryAgg (reduction: PerEntity) to read a TimeRange directly:

    let Some(NonASAPOp::TimeRange { child, .. }) = restored[input_id].non_asap() else {
        return Err(invalid("per-entity summary requires a resolved raw time range"));
    };

    The PromQL front end puts the TimeShift { at: Some(Start | End) } between them: SummaryAgg → TimeShift(@) → TimeRange → Scan. The TimeShift itself is planned as Operator::evaluate_at a few lines above.

  2. Range-request runner. RunContext::with_query_range (crates/executor/src/runtime/context.rs:70) has no caller. anchored_context (crates/executor/src/operators/time_range.rs:98) falls back to (evaluation_time, evaluation_time) when no range is set, which is right for an instant query only.

  3. Reference. The review harness has no exact range-query evaluator with Prometheus @ start() / @ end() semantics. Its reference is an instant evaluation at one t.

  4. Declared span. The planner costs and plans a range query for the span its workload entry declares. With Stage 3 shares nodes above an @ start() shift between queries whose anchors resolve to different times #1294's design, @ start() is lowered as @ end() offset by that span. A request whose actual range has a different length would get T − (declared span), not its own start, and nothing checks for this.

Proposal

  1. Install a per-entity aggregate over an @ shift. Look through the anchored TimeShift when resolving the raw time range, as below_time_shifts already does for offset shifts. Evaluate the input with evaluate_at's anchored context.
  2. A range-request entry point. Run a request [T − span, T] with with_query_range(T − span, T) and evaluate every step t, each @ input once per request.
  3. A range-query reference in the review harness. Evaluate each step exactly from the raw samples, resolving @ start() / @ end() to the request bounds. Compare at two request times.
  4. Refuse a mismatched range. The request path checks that the range length equals the span the plan was made for, and returns an error otherwise.

Scope and acceptance

Related

Activity

  1. added
    enhancementNew feature or request
    promqlPromQL front-end lowering (L1→L2)
    testingCorpus, regression, differential, or benchmark tests
    windowWindow functions, frames, panes, or temporal semantics
    on Nov 14, 2026
  2. zzylol commented on Nov 15, 2026

    @zzylol
    ContributorAuthor

    The time-operators PR (#1250) is closed as superseded. Its @ part had a per-entity aggregate over an @ shift and instant @ start() / @ end(). That part is re-filed here instead of in a new issue, because proposal items 1 and 2 above already cover it.

    What the closed PR had that is still missing on main (59365ce80):

    • Per-entity aggregate over an @ shift (proposal 1). It is still refused at install. Instant queries with @ <timestamp> now fail earlier, in the planner's install check: sum_over_time(c[1m] @ 1799999760) (exact, repeating every 60 s) gives installable plan: raw-data window of node 2 is unsupported: an @ anchor below the range (check_raw_window_below, crates/types/src/plan/materialized_view/raw_data.rs). The closed PR restricted that check to the windows a view's refresh reads. The fix here needs the same change.
    • No range-request runner (proposal 2). The closed PR did not have one either.

    Do not port the closed PR's design: its time_range(at), ExactEvaluation.at and executor walks (shifted_end, source_below, window_hull). Build on main's Operator::evaluate_at / anchored_context, and read windows from the edge read_window of the approved time-walk design (#1291/#1292). The tests an_at_timestamp_pins_the_selector_window and an_anchored_counter_window_ends_at_the_anchor in crates/planner/tests/time_operators.rs (branch exec/x4-time-operators) can be ported as instant-query acceptance tests.

    Related follow-ups filed from the same PR: #1318 (subqueries), #1319 (TimeRange over computed rows), and #748 (counter @, commented there).

  3. zzylol commented on Nov 17, 2026

    @zzylol
    ContributorAuthor

    The per-series range functions added in #1241 also stop at @. Run on #1241's branch against Prometheus 3.16.0, exact plans over raw samples, evaluated at 600 s:

    Query Executor
    rate(x[1m] @ 300), increase(x[1m] @ 300), irate(x[1m] @ 300), delta(x[1m] @ 300) compile: invalid DAG: node 2: invalid DAG: value operation has no native implementation
    the same with @ 330 offset 30s same error

    Prometheus answers each with the function over (240 s, 300 s]. The per-entity range function kernel reads its window as a fixed offset back from the evaluation time (time_range_window, crates/executor/src/physical_planner/time_window.rs), which an @ anchor is not; compile then fails on the TimeShift with an anchor. The fix belongs with this issue's @ handling, so that one resolved window feeds both the summary path and the range function kernel. Subqueries over these functions are #1318 and #1248.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestpromqlPromQL front-end lowering (L1→L2)testingCorpus, regression, differential, or benchmark testswindowWindow functions, frames, panes, or temporal semantics

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions