Skip to content

FinalizeExactAccumulator is charged 40 ns per group but costs about 2.5 us, so stored accumulators look cheaper than they are #1362

Closed

Description

@zzylol

Problem, by example

sum by (job) (sum_over_time(latency[5m])), 1,000 gauge series at 1 Hz in 10 jobs, evaluated every 10 s, raw data kept, default CostInputs, with the measured unit costs of #1355.

  • Selected: stored Sum windows: SummaryTumblingWindowChain(10 s) → TimeRange(5m) → SummaryMerge → FinalizeExactAccumulator → Aggregate by (job).
  • Alternative: a query-time Sum accumulator grouped by job: Scan → TimeRange(5m) → SummaryAgg(Sum, by job) → FinalizeExactAccumulator.
Node (per evaluation) Measured own CPU Predicted
stored: SummaryMerge 30,000 → 1,000 states 5.8 ms 6.9 ms
stored: FinalizeExactAccumulator over 1,000 groups 2.4 ms 0.04 ms
query-time: SummaryAgg Exact Sum, 300,000 rows into 10 groups 9.2 ms 12.0 ms
query-time: FinalizeExactAccumulator over 10 groups ≈ 0 0.0004 ms
  • Today: the query-time plan measures 1.28 cost/s and the stored plan 1.71 (0.92 query CPU, 0.22 ingestion, 0.58 memory). The planner predicts the opposite, 1.89 against 1.25, and selects the stored plan: regret 1.20. At 5m × 60 s the regret is 1.17.
  • Expected: finalize is charged about what it costs per output group, so the two plans are ranked as measured.
  • The same gap shows in all 52 finalize nodes measured over the grid: median 63× under-charged, about 2.5 µs per group.

A second per-group gap: a SummaryMerge with one input state per group (window = evaluation interval, e.g. 5m × 300 s) costs about 1.2 ms per 1,000 groups and is charged 0.23–0.35 ms.

Measured on a release build, one pinned core, each query-time node's own CPU (its sub-plan minus its inputs, median of 3 runs over warm stores).

Why it happens

  • FINALIZE_OPERATIONS_PER_GROUP = EXACT_UPDATE_OPERATIONS_PER_ROW = 40 (crates/cost/src/node/operations_per_unit.rs): finalize is charged as folding one row.
  • The executor's finalize builds one output row per group, with its series identity, which costs about 2.5 µs per group.
  • SummaryMerge is charged per input state only (merge_operations_per_state), with nothing per output group, so a merge of one state per group is charged as a single state merge.

Proposal

Scope and acceptance

  • In scope: the CPU charge of FinalizeExactAccumulator and the per-group part of SummaryMerge.
  • Acceptance: a cost test with the example's two plans in which the query-time plan is predicted cheaper; the two sum_over_time cells above select a plan measured within 1.05 of the best.

Related

Activity

  1. added
    bugSomething isn't working
    cost-modelCost formulas, statistics, ranking, and selection
    on Nov 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingcost-modelCost formulas, statistics, ranking, and selection

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions