Repository navigation
Description
Example
Two cases, one SQL and one PromQL. Both were observed with the review harness on the Stage 3 baseline branch.
SQL: a table with no time window. The data arrives at 1,000 rows/s and the query runs every 10 s:
SELECT approx_percentile_cont(bytes, 0.9) FROM flows(rank error 0.01; SELECT count(DISTINCT dst) FROM flows at error 0.05 behaves the same.)
- Today: Stage 3 selects
0 Scan(flows)(IngestionTime,None) →1 Aggregateover the whole table (IngestionTime,None) →2 Project(IngestionTime,AcrossEvaluations), costed at 0.0076/s.plan_installablerefuses it:Uninstallable(ViewReadsUnwindowedSource { node_id: 2 }). Both queries get the same plan and the same refusal. - Expected: either a plan the runtime can install (for example the query-time plan, which reads the table at each evaluation), or the ingestion-time view kept, with the executor refreshing a view over the whole table as rows arrive.
PromQL: an offset over raw rows. The query runs every 60 s, raw data is not kept (raw_data_retained: false), and there are 1,000 series at 1 sample/s:
sum_over_time(m[5m] offset 90s)
- Today: Stage 3 selects
0 Scan(m)(IngestionTime,None) →1 TimeShift(90s)(IngestionTime,AcrossEvaluations,retention_ms300,000) →2 TimeRange(300s)→SummaryAgg→Finalize. TheTimeShiftsits between theScanand the range, so the materialized view at node 1 reads raw rows with no window below it.plan_installablerefuses it:Uninstallable(ViewReadsUnwindowedSource { node_id: 1 }). - Expected: the same plan installs, with the executor refreshing a shifted view of raw rows that keeps the last 5 minutes (its
retention_ms).
Why
PhysicalASAPDAG::materialized_views (crates/types/src/plan/materialized_view/views.rs, inputs_of_view) refuses a view that reads a Scan other than through a raw-data TimeRange or a chain's tumbling window. An ingestion run refreshes the window that ends at the run's end. An unwindowed SQL table has no window, and a TimeShift placed directly over the Scan has none below it either.
Stage 2 still offers these ingestion-time candidates: views over an unwindowed table, and (IngestionTime, AcrossEvaluations) TimeShift views over a raw Scan. Stage 3 does not know the installer refuses them, so it picks them when they are cheapest.
For the PromQL case, Stage 3 also under-costs the view today: it charges one ingestion run's rows instead of the rows its retention_ms covers. That cost fix is in progress in the #1207 PR 6 branch. After it, these views are charged their full retention, so the planner may select them less often. They remain valid plans, so the installer still has to accept them.
What works nearby: the same SQL queries with a time window (WHERE ts > now() - INTERVAL '5 minutes') or with a declared rows_held plan and install. For the PromQL query, the cheapest plan without ingestion-time TimeShift views is a chain plan (0.1043/s against 0.01225/s for the selected view under today's cost).
Proposal
The planner must not be restricted by today's executor. Two options:
- Executor: refresh these views in the executor.
- A view over an unwindowed SQL source becomes an all-time accumulation. Each ingestion run merges its window's rows into the stored state (a running exact aggregate or a sketch merge), and the view keeps one state.
- A shifted view of raw rows keeps the rows its
retention_mscovers, indexed by time. Each ingestion run appends its rows and drops rows older than the retention. - The installer then accepts both, with
retention_mscovering what the view keeps.
- Cost and Stage 2 (if 1 is out of reach): cost these views by what they keep and do, and keep the planner's choice. Only the executor work in 1 makes them installable.
Option 1 is the recommendation.
Scope and acceptance
In scope: an ingestion-time view that reads a Scan with no window below it. This covers SQL tables with no time window and PromQL views placed between the Scan and the range, such as a TimeShift for offset.
Acceptance: the SQL queries above and sum_over_time(m[5m] offset 90s) (with raw data not kept) plan, install (plan_installable returns Ok), and answer equal to the exact reference over the same rows. Each is a planner integration test that fails today.
Related
Found by the functional reviews of the Stage 3 baseline (#1236, PR 6 of #1207):
- SQL: workloads
s01andx13. - PromQL: workload
y13.y05,y08andy18(raw data not kept) andy02andz03(raw data kept) select the same kind ofTimeShiftview.
Executor gaps are fixed in the executor, not by restricting the planner.
Activity
zzylol commentedon Nov 12, 2026
Four more workloads of the same review hit the same refusal (
ViewReadsUnwindowedSource) on the Stage 3 baseline branch, all over the unwindowed tableflows:SELECT src, sum(bytes) AS s FROM flows GROUP BY src ORDER BY s DESC LIMIT 5SELECT src, sum(-bytes) AS s FROM flows GROUP BY src ORDER BY s DESC LIMIT 5(twice, with and without an accuracy target)SELECT (SELECT approx_percentile_cont(bytes, 0.9) FROM flows)The acceptance test should cover a grouped top-k and a scalar subquery too.
[-]An ingestion-time view over a SQL table with no time window is planned but refused at install[/-][+]An ingestion-time view with no time window below it (SQL table, PromQL offset over raw rows) is planned but refused at install[/+]zzylol commentedon Nov 15, 2026
Decision: running-aggregate views over a whole SQL table are fixed here, in the executor. The executor refreshes such a view by accumulating each ingestion run's rows into the view's state; Stage 2 keeps offering the ingestion-time candidate, because an executor gap must not restrict the planner's design.
#1320 reported the same plan shape for
SELECT sum(a) FROM t(andmax,count(*),avg,stddev) and is closed as a duplicate of this issue. Its root-cause check on main is here: #1320 (comment). The acceptance test should cover those queries too, with the ingesting arrival from #1320 (rows_per_second100,rows_held1000) giving the same answers as with"at_rest". The separatecorr(a, b)costing bug is #1321.