Skip to content

Install and execute a TimeRange over computed rows (SQL asap_rate: TimeRange over Project over Scan) #1319

Open

Description

@zzylol

Problem, by example

SQL asap_rate over a time-indexed table plans and then fails at install. Table metrics(ts TIMESTAMP, service UTF8, latency DOUBLE), time index ts, unique key (ts, service). Workload: one repeating query every 60 s at phase 0, exact, ingesting at 100 rows/s. plan_installable on main (59365ce80):

SELECT service, asap_rate(latency, ts, 300000) AS r FROM metrics GROUP BY service
  • Today: installable plan: raw-data window of node 2 is unsupported: an operator other than TimeShift below the range.
  • Expected: the plan installs, and each service gets the rate of latency over (T - 5m, T].

The frontend lowers asap_rate as TimeRange(Project(Scan)): the Project puts the table's columns into the time-series layout the rate reads. So the TimeRange reads computed rows, not a Scan.

Why it happens

There are two refusals, in this order:

  1. Planner install check. ViewIndex::check_raw_window_below (crates/types/src/plan/materialized_view/raw_data.rs) walks down from every TimeRange that reads raw data. It accepts only TimeShifts without @ down to the Scan, so a Project gives an operator other than TimeShift below the range. The check also runs on query-time windows, which no ingestion run refreshes. It exists for windows that a materialized view's refresh reads.
  2. Executor. The executor's TimeRange lowering expects to reach a Scan through TimeShifts. This was observed on the closed branch exec/x4-time-operators, which had to extend the lowering to read any time-indexed input.

Proposal

Scope and acceptance

  • In: the query above, and a per-entity SummaryAgg over such a range. Each is planned exactly with plan_installable and run by the executor, and the answer matches the rate computed by hand over the rows in (T - 5m, T]. The closed branch has a test that can be ported: sql_asap_rate_reads_its_window_of_projected_rows in its crates/planner/tests/time_operators.rs.
  • Out: a bounded scan window, retention and Stage 3 pricing for this shape (Bounded scan windows, retention and Stage 3 pricing for subqueries, @ anchors and ranges over projected rows #1249 row 3; today the whole table is read).

Status and related work

Activity

  1. added
    enhancementNew feature or request
    sqlSQL front-end lowering (DataFusion → L2)
    windowWindow functions, frames, panes, or temporal semantics
    on Nov 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestsqlSQL front-end lowering (DataFusion → L2)windowWindow functions, frames, panes, or temporal semantics

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions