Repository navigation
Description
Problem, by example
Two queries in one workload that contain the same expression with a NaN literal:
sum(x > NaN)
max(x > NaN)
x > NaN (the comparison over x) is the same expression in both, so it should be one memo group read by both aggregates. Today it becomes two groups, and every group above it that would otherwise be shared is duplicated too. The same holds for any repeated expression carrying a Float64(NaN) literal, in PromQL or SQL. Derived from the code at 73785fa (head of #1230), not from a run (the CLI's stage output for this commit is still todo! until #1207 PR 9).
x > NaN in q1 and q2 |
x > 1 in q1 and q2 |
|
|---|---|---|
| Today | two groups | one group |
| Expected | one group | one group |
Why it happens
Hash-consing in crates/types/src/stages/logical/memo/hash_consing.rs compares typed equality first:
same_serialized_expression(L42-L47):left == right && left_serialized == right_serialized, used byMemo::find_or_add_group(memo/mod.rsL105-L125).same_value(L54-L59): the same pattern, used for candidates (same_expression_and_logical_properties).
ScalarValue derives PartialEq (crates/types/src/ir/scalar/expr_ir.rs L35-L38), and f64::NAN != f64::NAN, so two operators holding ScalarValue::Float64(NaN) are never ==, whatever their serializations say. The serialization alone would match: ScalarValue::Float64 is serialized through crate::serde_f64, which writes "NaN" (and "+Inf", "-Inf") explicitly (crates/types/src/serde_f64.rs).
The comments in hash_consing.rs say non-finite floats serialize as null; that is true only for f64 fields without serde_f64, not for ScalarValue literals or the AggIntent parameters that use it.
Proposal
Decide equality by the canonical serialization (already computed once per expression), with no typed == in front of it:
- Equal serializations make
NaN == NaNfor literals, which is what sharing needs. - Signed zero needs no typed check:
serde_jsonwrites0.0and-0.0differently, so the serialization already keeps them apart (it is the typed==that conflates them, which is why the serialized check was added). - The one case serialization conflates is an
f64field serialized withoutserde_f64, where NaN and ±Inf all becomenull. Either move those fields of operators and logical properties toserde_f64, or keep a typed check only for them that treats NaN as equal to NaN (atotal_cmp-style comparison). - Fix the
nullcomments.
Scope and acceptance
In scope: hash_consing.rs and, if chosen, serde_f64 on the remaining f64 fields of operators and logical properties.
Acceptance (fails today): adding x > NaN twice to a memo returns one group; x > 0.0 and x > -0.0 stay two groups (as today); x > 1 and x > 2 stay two groups.
Status and related work
Found by a semantic review of #1230 (PR 2 of #1207). Decided 2026-11-10: #1230 keeps only the core data-structure, interface and definition changes; case fixes go to follow-up issues and later PRs.
Activity