Skip to content

Aggregate coverage: refuse a TimeShift moved past a subquery Filter, and chain windows or merges that read an evaluation-time-dependent group #1266

Open

Description

@zzylol

Problem, by example

AggregateCoverage decides when two summary states compute the same thing (definition) over different rows (selection), so that one can be merged with or selected from another. Two refusals it should make are missing. Neither case is reachable from today's frontends (the SQL frontend emits no TimeShift, and the PromQL frontend emits no scalar subquery), so this is a latent soundness gap for the IR, not a wrong answer today. Derived from the code at 73785fa (head of #1230), not from a run.

(a) A TimeShift moves past an unlifted Filter that reads another group. S is a scalar subquery group (e.g. a threshold) that depends on the evaluation time.

A: SummaryAgg(TimeRange(5m, Filter(v > S, TimeShift(offset 1h, Scan m))))
B: SummaryAgg(TimeRange(5m, TimeShift(offset 1h, Filter(v > S, Scan m))))

In A, S is evaluated at the evaluation time t; in B it is inside the shift, so it is evaluated at t − 1h. The two states keep different rows. Today both get the same coverage: definition = [SummaryAgg, Filter(v > S)] over Scan m, selection = ts ∈ (t − 1h − 5m, t − 1h], so a merge or a reuse would treat them as interchangeable.

(b) A chain's windows or a merge read a group that changes with the evaluation time.

SummaryTumblingWindowChain[1m](Filter(v > S, Scan m))   -- S depends on the evaluation time
TimeRange(5m, <that chain>)                             -- coverage: same definition as SummaryAgg(TimeRange(5m, Filter(v > S, Scan m)))

Each window of the chain is built once, when the window closes, so it evaluates S at build time. A query-time SummaryAgg evaluates S at the query's evaluation time. The window selection and a SummaryMerge over such windows are accepted as covering the same definition, though their states hold different rows.

Why it happens

All in crates/types/src/stages/logical/aggregate_coverage.rs at 73785fa:

  • (a) of_aggregate (L439) lifts the TimeRange and all TimeShifts into one interval (time_constraint) and, when it lifts them (lift_time, L521), drops them from unlifted_operators (L536) wherever they sat. An unlifted Filter whose residual reads another group (Some(g) in unlifted_operators) keeps its place, so its position relative to the shift is lost.
  • (b) of_merge (L254) checks only that the inputs' definitions are equal and their selections disjoint; of_window_selection (L710) copies the chain's definition. Neither looks at whether a group the definition reads (Some(g) in unlifted_operators) has depends_on_evaluation_time set.

Proposal

Refuse conservatively, no new representation:

  • (a) In of_aggregate, do not lift the time links when an unlifted operator that reads another group sits between the TimeRange and a TimeShift; they then stay in unlifted_operators in place, and the definitions differ.
  • (b) In of_merge and of_window_selection (and therefore regrouped), return NoCoverage when any group the definition reads depends on the evaluation time.

Scope and acceptance

In scope: the two refusals and their tests in crates/types/tests/memo_aggregate_coverage.rs. Out of scope: making such builds shareable (re-evaluating the subquery per window).

Acceptance (fail today, built directly in the memo since no frontend emits these shapes):

  • A and B above get different definitions.
  • A TimeRange over a chain whose definition reads a group with depends_on_evaluation_time = true has no coverage, and a SummaryMerge over such windows is rejected.

Status and related work

Found by a semantic review of #1230 (PR 2 of #1207). Decided 2026-11-10: #1230 keeps only the core data-structure, interface and definition changes; case fixes go to follow-up issues and later PRs. Related: #1264 (depends_on_evaluation_time fixes from the same review).

Activity

  1. added
    bugSomething isn't working
    irPre-ASAP or post-ASAP intermediate representation
    rollupFine-to-coarse aggregation and summary reuse
    windowWindow functions, frames, panes, or temporal semantics
    on Nov 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingirPre-ASAP or post-ASAP intermediate representationrollupFine-to-coarse aggregation and summary reusewindowWindow functions, frames, panes, or temporal semantics

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions