Repository navigation
Description
Problem, by example
GroupProperties::depends_on_evaluation_time decides whether a stored result can serve every evaluation. When it is false, PhysicalCandidate::delivers_on returns true for every execution context, so one stored copy is reused forever. The rule gets three cases wrong. Each example below is derived from the code at 73785fa (head of #1230), not from a planner run.
(a) A SQL table scan is marked independent of the evaluation time.
-- evaluated every minute
SELECT count(*) FROM metricsThe SQL frontend lowers this to Aggregate(count, Scan metrics), with no TimeRange and no now(). depends_on_evaluation_time is false for the Scan and the Aggregate, so a result stored once delivers on every evaluation grid, and the query returns the first minute's count forever. Rows keep arriving into metrics, so the count changes between evaluations.
| Today | Expected | |
|---|---|---|
Scan metrics |
does not depend | depends (data arrives) |
count(*) over it |
stored once, reused on every grid | recomputed or refreshed per evaluation |
(b) @ <timestamp> hides an inner @ end() / @ start().
max_over_time((rate(x[5m] @ end()))[10m:1m] @ 1700000000)
The PromQL frontend lifts the inner @ end() above rate(x[5m]) as TimeShift { at: End }, and puts the outer @ 1700000000 as TimeShift { at: Timestamp } above the subquery. The TimeShift { at: Timestamp } arm returns false without looking at its input, so the whole subquery is marked independent. In Prometheus the inner @ end() takes precedence: it resolves to the request's end, which is the evaluation time for an instant query. The inner rate therefore changes with the evaluation time.
(c) A missed optimization: @ <timestamp> below a TimeRange.
sum_over_time(x[5m] @ 1700000000)
The result is the same at every evaluation. But the frontend keeps @ T below the TimeRange (TimeRange(5m, TimeShift(@ T, Scan x))), and the TimeRange arm returns true unconditionally, so the group is marked dependent and is never stored once for all evaluations. The same holds for an instant selector x @ 1700000000 (TimeRange(Instant, TimeShift(@ T, Scan x))).
Why it happens
depends_on_evaluation_time in crates/types/src/stages/logical/group.rs (L131-L159):
Operator::NonASAP(NonASAPOp::TimeRange { .. }) => return true(L139) — true regardless of whether the input is pinned by@ T(case c).TimeShift { at: Some(AtModifier::Timestamp(_)), .. } => return false(L140-L147) — returns before the input groups are read (case b).- Every other operator depends only on
now()/time()in its scalar expressions or on its input groups, so aScanwith neither is always false (case a). The test table incrates/types/tests/memo_group_properties.rs(L387,G1 | Scan latency | no) encodes this.
The consumer is PhysicalCandidate::delivers_on in crates/types/src/stages/physical/candidate.rs (L58-L63): a non-chain AcrossEvaluations result whose group does not depend on the evaluation time delivers on every context.
The frontend shapes come from selector and anchored in crates/frontend-promql/src/promql.rs (L1172, L1193: @ start()/@ end() are lifted above the subtree, @ <ts> stays inside it below the TimeRange) and the subquery arm (L230-L247: the subquery's own @/offset become a TimeShift above PromqlSubquery).
PromQL time-series scans are not affected by (a) today: every selector is lowered under a TimeRange (range or instant lookback), which already marks it dependent.
Proposal
- Scans of arriving data depend on the evaluation time. A
Scanof a source whose data arrives (the data workload'sarrival; an unknown arrival is treated as arriving) depends on it, unless a fixed closed interval bounds it: an enclosing@ T, or a literalts BETWEEN a AND b(both bounds literal). Open question: whether a fixed interval should still be treated as dependent when late data can arrive inside it; the conservative answer is yes unless the source declares no late arrivals. @ Tpins only its own time, not inner anchors.TimeShift { at: Timestamp }is false only when no input group reads@ start()/@ end(),now()ortime(). One way: carry two bits per group, "reads the evaluation-time anchor through aTimeRange" (cleared by@ T) and "reads the request start/end or the wall clock" (never cleared by@ T).- A
TimeRangeover a subtree pinned by@ Tdoes not depend. With the two bits of (2), aTimeRangesets the first bit only when its input is not pinned.
Scope and acceptance
In scope: the rule in group.rs, its test table, and delivers_on if the property splits.
Acceptance (each fails today):
SELECT count(*) FROM metricsover an arriving source: theScanand theAggregategroups depend on the evaluation time, and a stored result of them does not deliver on a 60 s grid beyond its own times.max_over_time((rate(x[5m] @ end()))[10m:1m] @ 1700000000): the top group depends on the evaluation time.sum_over_time(x[5m] @ 1700000000)andx @ 1700000000: the top group does not depend on it.
Status and related work
Found by a semantic review of #1230 (PR 2 of #1207). Decided 2026-11-10: #1230 keeps only the core data-structure, interface and definition changes; case fixes like this one go to follow-up issues and later PRs. Related: #1251 (evaluate @ end()/@ start() subtrees once per request).
Activity
zzylol commentedon Nov 10, 2026
Scheduling: this is fixed before #1207 PR 7 (#1237, execution contexts / AcrossEvaluations), because storing a result across evaluations relies on
depends_on_evaluation_timebeing right; a Scan of a source that receives data must not be reused as time-independent.feat(stage0, types): memo hash-consing, group properties (column_sign…
fix(types): review fixes for #1264: docs on @ start()/@ end(), no-sou…