Repository navigation
Description
Problem, by example
An exact PromQL subquery over a selector or over a per-series range function plans, and then fails at install. Workload: one repeating query every 60 s at phase 0, exact (max_error 0), 10 series of c sampled every 10 s, 1,000 rows held, ingesting at 100 rows/s. plan_installable on main (59365ce80):
| Query (exact) | Today | Expected (Prometheus semantics) |
|---|---|---|
max_over_time(c[10m:1m]) |
installable plan: the executor cannot compile the plan's query run: invalid DAG: node 2: invalid DAG: value operation has no native implementation |
the largest of c evaluated at the multiples of 1 minute in (T - 10m, T] |
max_over_time(rate(c[1m])[10m:1m]) |
the same error, at node 4 | the largest of rate(c[1m]) evaluated at the multiples of 1 minute in (T - 10m, T] |
Concrete values, from the acceptance tests of the closed branch exec/x4-time-operators (crates/planner/tests/time_operators.rs there): a counter of job a sampled every 10 s from T - 700 s to T, growing 1 per sample, 6 per sample in (T - 300 s, T - 240 s] and 100 per sample in (T - 660 s, T - 600 s]:
Query at T |
Expected |
|---|---|
max_over_time(rate(c[1m])[10m:1m]) |
0.6 (the step T - 240 s) |
min_over_time(rate(c[1m])[10m:1m]) |
0.1 |
max_over_time(rate(c[1m])[10m:1m] offset 5m) |
10.0 (the grid T - 840 s … T - 300 s holds T - 600 s) |
And for a gauge g of job a with 100 at T - 600 s, 1 at T - 540 s, 5 at T - 300 s, 9 at T - 30 s and 50 at T + 30 s, evaluated at T + 30 s: max_over_time(g[10m:1m]) = 5, min_over_time(g[10m:1m]) = 1, count_over_time(g[10m:1m]) = 2.
Why it happens
The executor's physical_planner::bind_operation (crates/executor/src/physical_planner/mod.rs) has no arm for NonASAPOp::PromqlSubquery, so it falls to value operation has no native implementation. Operator::series_window (crates/executor/src/operators/series_window.rs) already takes steps: Option<SubquerySteps>, but no compile path builds one.
The planner no longer refuses this shape: the planner-side executor-support checks were removed, so the gap shows up at install as QueryRunNotExecutable.
Proposal
- Lower
f(m[r])[R:s]andm[R:s](withoffseton the subquery or its selector) to oneOperator::series_windowwithstepsover the inner selector'sScan. Herefis a per-entityAggregateor an exact accumulator,FinalizeExactAccumulator(SummaryAgg), over aTimeRange. A subquery without a step uses Prometheus' default evaluation interval, 1 minute. - The executor reads the subquery's window from the plan, not from its own walk. It uses the edge
read_windowof the approved time-walk design ([Bug] check_compiles accepts a plan whose view over an ingestion-time chain cannot be refreshed on its own #1291/[Refactor] Executor time walks (counter_window, node_below_shifts, below_time_shifts) should use the shared time_window walks #1292): theScanis read over(T - o - R - r, T - o]. That window is computed once in the planner, as Bounded scan windows, retention and Stage 3 pricing for subqueries, @ anchors and ranges over projected rows #1249 proposes forScanWindow. The closed branch widened the scan inside the executor (Subquery::scan_window) and added more executor time walks (shifted_end,source_below,window_hull), which the approved design deletes. Do not port those. @on a subquery is out of scope here. It goes with Range queries with@anchors cannot be executed or verified #1306 onOperator::evaluate_at.
Scope and acceptance
- In: the two queries above, with and without
offset, undermax/min/count/sum_over_timeandrate, planned exactly withplan_installableand run by the executor. Their answers match the expected values above. The planner e2e tests of the closed branch can be ported as the acceptance tests. - Out: subqueries over other inner expressions (cross-series aggregates, binary operations, nested subqueries), which need the step-grid evaluator in Executor: subqueries over expressions other than a selector or a per-series range function (step-grid evaluator) #1248; bounded retention and Stage 3 pricing of the widened window (Bounded scan windows, retention and Stage 3 pricing for subqueries, @ anchors and ranges over projected rows #1249);
@(Range queries with@anchors cannot be executed or verified #1306).
Status and related work
- Re-filed from the closed time-operators PR (feat(executor): PromQL subqueries, @ anchors and time ranges over computed rows #1250, branch
exec/x4-time-operators, its "gap 25"), which is superseded by the approvedread_windowdesign. - Depends on the [Bug] check_compiles accepts a plan whose view over an ingestion-time chain cannot be refreshed on its own #1291/[Refactor] Executor time walks (counter_window, node_below_shifts, below_time_shifts) should use the shared time_window walks #1292 time-walk PR, which provides the edge
read_window. - Related: Executor: subqueries over expressions other than a selector or a per-series range function (step-grid evaluator) #1248 (general subqueries, builds on this), Bounded scan windows, retention and Stage 3 pricing for subqueries, @ anchors and ranges over projected rows #1249 (scan window, retention and pricing), Stage 3 selects plans whose raw-data windows installation refuses (subquery over a range) #1045 (Stage 3 selecting plans whose raw-data window installation refuses, subquery over a range).
Activity