Skip to content

Heavy-hitter promotion could renumber positional readers instead of being skipped #1347

Open

Description

@zzylol

Problem

Heavy-hitter promotion in canonicalize is skipped whenever a parent reads the ranked node by column position. For example, with one repeating PromQL query:

Query Canonical shape today Summary candidates for the ranking
topk(3, sum by (job) (m)) Aggregate { [TopK { k: 3 }] } over Aggregate { [Sum] } heavy-hitter summaries (TopK is a summary-capable intent)
sort_desc(topk(3, sum by (job) (m))) Sort { Limit { Sort { Aggregate { [Sum] } } } }: the inner Limit { Sort } stays none: only the exact sort then limit
topk(3, sum by (job) (m)) > 5 Filter { Limit { Sort { Aggregate { [Sum] } } } } none

These shapes are the lowered trees lower_promql_query_workload returns today (checked on the without grouping PR's head). The same holds under any operator that reads the top-k's columns by position, such as arithmetic (topk(3, …) * 2) or an outer aggregate.

Why

canonicalize (crates/frontend-common/src/canonicalize.rs) computes read_by_position, the nodes some consumer reads by column position, and promotes a Limit { Sort { Aggregate } } only for a node outside that set (columns_may_change). Promotion changes the node's columns, so a parent bound to the old positions would read another column. Refusing is correct but loses the rewrite.

Series identity realization (with_promql_series_identity, crates/types/src/ir/schema/series_identity.rs) solves the same problem the other way: when it inserts the identity column into a node, it renumbers every parent's column references with NonASAPOp::map_input_columns (and an Aggregate's having by the aggregate's output), and fails only when a parent reads a column the rewrite drops.

Proposal

Let promotion renumber parents instead of refusing:

  1. For a node in read_by_position, compute where each column of the old output sits in the promoted output, matched by name, as positions_after_realization does for the identity column.
  2. If every column a parent reads is kept, rewrite the node and renumber each parent's references with map_input_columns.
  3. Refuse only when a parent reads a column the promotion does not keep.

The two rewrites would then share one rule: a rewrite that moves a node's columns renumbers its readers. A shared helper (positions by name, then map_input_columns) could serve both.

Scope and acceptance

  • In scope: heavy-hitter promotion under positional readers.
  • Out of scope: other canonicalization rules, which do not change columns.
  • Done when sort_desc(topk(3, sum by (job) (m))) canonicalizes its inner ranking to Aggregate { [TopK { k: 3 }] }, a test checks the parent's sort key reads the ranked value after promotion, and the exact answers over a sample input stay equal to today's.

Related

Activity

  1. added
    canonicalizationCanonical IR normalization and invariants
    enhancementNew feature or request
    promqlPromQL front-end lowering (L1→L2)
    on Nov 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    canonicalizationCanonical IR normalization and invariantsenhancementNew feature or requestpromqlPromQL front-end lowering (L1→L2)

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions