Background
PR #5166 fixes #4966 and adds Comet SQL-file coverage for Spark 4.2 collect_set normalization.
The repository currently has no dev/diffs/4.2*.diff and no Spark 4.2 Spark SQL Tests job, so the corresponding upstream Spark tests cannot yet be enabled in the Comet diff.
Task
Once a Spark 4.2 diff and CI job are added, enable these DataFrameAggregateSuite tests:
SPARK-57298: collect_set normalizes NaN and -0.0 for floating-point types
SPARK-57298: collect_set normalizes NaN and -0.0 nested in complex types
Verify both tests pass with native collect_set.
Related: #4966, #5166, and review 4830920688.
Background
PR #5166 fixes #4966 and adds Comet SQL-file coverage for Spark 4.2
collect_setnormalization.The repository currently has no
dev/diffs/4.2*.diffand no Spark 4.2 Spark SQL Tests job, so the corresponding upstream Spark tests cannot yet be enabled in the Comet diff.Task
Once a Spark 4.2 diff and CI job are added, enable these
DataFrameAggregateSuitetests:SPARK-57298: collect_set normalizes NaN and -0.0 for floating-point typesSPARK-57298: collect_set normalizes NaN and -0.0 nested in complex typesVerify both tests pass with native
collect_set.Related: #4966, #5166, and review 4830920688.