Skip to content

[Bug] Canonicalize floating-point NaN values for Java compatibility #256

Description

@zjw1111

Search before asking

  • I searched the existing issues and found nothing similar.

Paimon-cpp version

main at 4027601e.

Minimal reproduce step

  1. Construct float or double NaN values with different sign or payload bits.
  2. Use those values in Literal hashing, Bloom filter hashing, Hive bucket hashing, BTree keys, Variant values, global-index scores, or indexed-split scores.
  3. Compare the generated hashes or serialized bytes with Java Paimon, which canonicalizes NaN through Float.floatToIntBits and Double.doubleToLongBits.

What doesn't meet your expectations?

C++ currently preserves raw NaN payload bits in several hashing and serialization paths. As a result, NaN values that compare as equivalent can produce different hashes, and C++ can emit bytes that differ from Java.

All float NaN values should use canonical bits 0x7fc00000, and all double NaN values should use canonical bits 0x7ff8000000000000, while preserving every non-NaN value unchanged.

Anything else?

No.

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions