Skip to content

Enable SPARK-57298 collect_set tests when Spark 4.2 SQL diff lands #5209

Description

@peterxcli

Background

PR #5166 fixes #4966 and adds Comet SQL-file coverage for Spark 4.2 collect_set normalization.

The repository currently has no dev/diffs/4.2*.diff and no Spark 4.2 Spark SQL Tests job, so the corresponding upstream Spark tests cannot yet be enabled in the Comet diff.

Task

Once a Spark 4.2 diff and CI job are added, enable these DataFrameAggregateSuite tests:

  • SPARK-57298: collect_set normalizes NaN and -0.0 for floating-point types
  • SPARK-57298: collect_set normalizes NaN and -0.0 nested in complex types

Verify both tests pass with native collect_set.

Related: #4966, #5166, and review 4830920688.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions