Skip to content

Pass 2: a filter that pins a grouping key does not reuse an exact grouped quantile (no same-level view match) #1358

Open

Description

@zzylol

Problem, by example

The workload crates/integration-tests/tests/plan_sharing_fixtures/percentile_filtered_and_by_city.json, over the table metrics(ts, city, host, value) with 100 series, every 10 s, 1% target:

-- A: the filtered quantile
SELECT percentile_cont(0.9) WITHIN GROUP (ORDER BY value) FROM metrics
WHERE ts > now() - INTERVAL '5' MINUTE AND city = 'NYC'
-- B: the same quantile by city
SELECT city, percentile_cont(0.9) WITHIN GROUP (ORDER BY value) FROM metrics
WHERE ts > now() - INTERVAL '5' MINUTE GROUP BY city

Under the default CostInputs (with the measured unit costs of #1355), both queries are planned exactly (scan, keep each group's values, sort). B alone costs 2.1164 per second; A and B together cost 2.2742, 7.5% more: A scans and sorts NYC's rows again, although B's output already holds NYC's exact 0.9 quantile.

Expected: A reads B's row for city = 'NYC' (a filter on B's output, dropping the city column), and the two cost about what B costs alone. With summaries' CPU costed free (summary_cpu_free), the sketch plans are selected, A reads B's states through a compensating filter, and together they cost 2.1167 against B's 2.1164. Numbers from plan_sharing::grouped_filtered_quantile_plans (cost= lines with --nocapture).

Why it happens

  • Pass 2's rollup rule (rollup_from in crates/stage1-pass2-reuse-candidates/src/rules/rollup/rewrite.rs) does find B as A's finer group, with the compensating filter city = 'NYC' on B's key. It always builds Rollup(by A's keys)(Filter(compensation, B)).
  • For an exact Aggregate, the rollup re-aggregates each measure with rolled_up (same file), which has a rollup only for Sum, Count, Min and Max. A quantile has none, so the rule returns without a candidate.
  • But here no re-aggregation is needed: the compensating filter pins the one key A drops (city) to a single value, so at most one B row survives and that row is A's answer. View matching calls this the same-level case (the dropped group columns are all fixed by constant conditions): the substitute is a filter and projection over the view, with no aggregate, and it works for holistic measures such as a quantile.
  • For summary states the same rule already works (A merges B's NYC state), which is why the sketch plans share.

Proposal

  • In rollup_from, when every key A drops is pinned by an equality in the compensation, offer Project(Filter(compensation, B)) (A's columns, under A's names) instead of the rollup, for any measure. It is exact, so it carries no accuracy cost.
  • Keep the SQL semantics of a global aggregate over no row: if no B row matches, A returns one NULL row (a MIN-style wrapper, as the earlier filtered-group rule did), which the memo's schema check should cover.

Scope and acceptance

  • In scope: equality compensations that pin every dropped key, over an exact grouped Aggregate, SQL and PromQL. Out of scope: IN lists or ranges (they still need a rollup, which a quantile does not have).
  • Acceptance: plan_sharing::grouped_filtered_quantile_plans' last assertion (A and B together cost more than B alone under the default costs) fails because A reuses B; change it to expect together ≤ B × 1.001.

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    CSEenhancementNew feature or requestrollupFine-to-coarse aggregation and summary reuse

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions