Skip to content

Executor: per-series deriv, predict_linear, delta and idelta have no native implementation, so the planner plans them but none installs #1357

Closed

Description

@zzylol

Problem, by example

The data is 1,000 series of a gauge latency sampled every 1 s, arriving at 2,000 rows per second. Each query below runs every 10 s with the built-in models and default CostInputs:

Query plan_installable, raw data kept plan_installable, raw data not kept
deriv(latency[1h]) the executor cannot compile the plan's query run: invalid DAG: node 2: per-entity aggregate Deriv has no native implementation no valid candidate: ... Infeasible
predict_linear(latency[1h], 60) ... per-entity aggregate PredictLinear { seconds: 60.0 } has no native implementation same infeasible selection
delta(latency[1h]) ... per-entity aggregate Delta has no native implementation same infeasible selection
idelta(latency[5m]) ... per-entity aggregate IDelta has no native implementation same infeasible selection
sum by (job) (delta(latency[1h])) ... per-entity aggregate Delta has no native implementation same infeasible selection

These come from a throwaway planner test that calls plan_installable for each query, on the head of #1355 (008704da2) and with its review fixes; the results are the same on both.

The planner plans each query as Scan → TimeRange → Aggregate { reduction: per entity, measures: [Deriv] } (and the same for the others). The executor then refuses to compile that plan. Expected: each query installs and answers as Prometheus does.

Why it happens

  • The executor's per-entity Aggregate lowering (crates/executor/src/physical_planner/mod.rs, the arm that returns per-entity aggregate ... has no native implementation) maps only Count, Sum, Avg, Min, Max, Quantile, Cardinality, FrequencyL2 and FrequencyEntropy to a Reduction. That list is agg_has_exact_per_entity_kernel (crates/types/src/ir/operator/agg_intent.rs).
  • The executor already has a windowed aggregate for some of these: Operator::window (crates/executor/src/operators/aggregate/mod.rs) accepts Rate, Deriv, PredictLinear and Increase over a (start, end) window. The physical planner never builds it, because the per-entity Aggregate over a TimeRange has no window bounds at compile time: they depend on each evaluation's time.
  • rate and increase install today only because Stage 2 offers an exact Rate / Increase accumulator (SummaryAgg + FinalizeExactAccumulator) for them. deriv, predict_linear, delta and idelta have no accumulator, so the per-entity Aggregate is their only plan.
  • agg_has_exact_per_entity_kernel's doc says "Stage 3 rejects a plan holding any other exact per-entity aggregate, so both read this one list". Only the executor reads it; Stage 3 selects these plans and installation refuses them, as the table shows.

Proposal

  • Executor: lower a per-entity Aggregate whose measure is Rate, Increase, Deriv, PredictLinear, Delta or IDelta to a kernel that reads each series' samples in time order over the evaluation's window (the TimeRange's bounds at run time), reusing Operator::window's computation for the first four. Add Delta and IDelta (delta: last minus first, extrapolated as Prometheus does; idelta: the last two samples).
  • Fix the stale claim in agg_has_exact_per_entity_kernel's doc: say that only the executor reads it, or add the Stage 3 check it describes. The planner should not be restricted by the executor's current list, so the preferred fix is the executor kernels above plus a doc that says what the list is.
  • irate, resets and changes are Executor: irate, resets and changes have no native implementation, so the planner plans them but none installs #1345.

Scope and acceptance

  • In scope: per-series rate, increase, deriv, predict_linear, delta, idelta over a range, as a per-entity Aggregate at query time over raw data.
  • Out of scope: tumbling-window chain states for deriv / predict_linear / delta (a separate accumulator design).
  • Acceptance: a planner test plans each query above with raw data kept, installs it and answers as Prometheus does on a small series. It fails today.

Related

Activity

  1. added
    bugSomething isn't working
    promqlPromQL front-end lowering (L1→L2)
    on Nov 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingpromqlPromQL front-end lowering (L1→L2)

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions