Skip to content

Hash-consing never shares expressions with a NaN literal #1268

Open

Description

@zzylol

Problem, by example

Two queries in one workload that contain the same expression with a NaN literal:

sum(x > NaN)
max(x > NaN)

x > NaN (the comparison over x) is the same expression in both, so it should be one memo group read by both aggregates. Today it becomes two groups, and every group above it that would otherwise be shared is duplicated too. The same holds for any repeated expression carrying a Float64(NaN) literal, in PromQL or SQL. Derived from the code at 73785fa (head of #1230), not from a run (the CLI's stage output for this commit is still todo! until #1207 PR 9).

x > NaN in q1 and q2 x > 1 in q1 and q2
Today two groups one group
Expected one group one group

Why it happens

Hash-consing in crates/types/src/stages/logical/memo/hash_consing.rs compares typed equality first:

  • same_serialized_expression (L42-L47): left == right && left_serialized == right_serialized, used by Memo::find_or_add_group (memo/mod.rs L105-L125).
  • same_value (L54-L59): the same pattern, used for candidates (same_expression_and_logical_properties).

ScalarValue derives PartialEq (crates/types/src/ir/scalar/expr_ir.rs L35-L38), and f64::NAN != f64::NAN, so two operators holding ScalarValue::Float64(NaN) are never ==, whatever their serializations say. The serialization alone would match: ScalarValue::Float64 is serialized through crate::serde_f64, which writes "NaN" (and "+Inf", "-Inf") explicitly (crates/types/src/serde_f64.rs).

The comments in hash_consing.rs say non-finite floats serialize as null; that is true only for f64 fields without serde_f64, not for ScalarValue literals or the AggIntent parameters that use it.

Proposal

Decide equality by the canonical serialization (already computed once per expression), with no typed == in front of it:

  • Equal serializations make NaN == NaN for literals, which is what sharing needs.
  • Signed zero needs no typed check: serde_json writes 0.0 and -0.0 differently, so the serialization already keeps them apart (it is the typed == that conflates them, which is why the serialized check was added).
  • The one case serialization conflates is an f64 field serialized without serde_f64, where NaN and ±Inf all become null. Either move those fields of operators and logical properties to serde_f64, or keep a typed check only for them that treats NaN as equal to NaN (a total_cmp-style comparison).
  • Fix the null comments.

Scope and acceptance

In scope: hash_consing.rs and, if chosen, serde_f64 on the remaining f64 fields of operators and logical properties.

Acceptance (fails today): adding x > NaN twice to a memo returns one group; x > 0.0 and x > -0.0 stay two groups (as today); x > 1 and x > 2 stay two groups.

Status and related work

Found by a semantic review of #1230 (PR 2 of #1207). Decided 2026-11-10: #1230 keeps only the core data-structure, interface and definition changes; case fixes go to follow-up issues and later PRs.

Activity

  1. added
    bugSomething isn't working
    canonicalizationCanonical IR normalization and invariants
    on Nov 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    CSEbugSomething isn't workingcanonicalizationCanonical IR normalization and invariants

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions