Skip to content

[GH-3267] Preserve recoverable geometry dimensions - #3268

Merged
jiayuasu merged 2 commits into
apache:masterfrom
jiayuasu:fix/java-geometry-serde-dimensions
Aug 18, 2026
Merged

[GH-3267] Preserve recoverable geometry dimensions#3268
jiayuasu merged 2 commits into
apache:masterfrom
jiayuasu:fix/java-geometry-serde-dimensions

Conversation

@jiayuasu

@jiayuasu jiayuasu commented Aug 17, 2026

Copy link
Copy Markdown
Member

Did you read the Contributor Guide?

Is this PR related to a ticket?

  • Yes, and the PR name follows the format [GH-XXX] my subject.

Closes #3267.

What changes were proposed in this PR?

Sedona's compact Java geometry format already supports XY, XYZ, XYM, and XYZM coordinate-layout tags. However, GeometrySerializer selected one of those tags by inspecting only a geometry's first coordinate and checking whether its Z and M values were finite. This could silently discard recoverable ordinates before a geometry crossed a Spark GeometryUDT or Flink serializer boundary. For example, LINESTRING Z (0 0 NaN, 1 1 3) was encoded as XY because its first Z value is NaN, so the later finite Z value was lost.

This PR makes dimensional inference and empty reconstruction loss-aware:

  • Inspect every coordinate sequence, including all polygon rings and multipart members, instead of only the first coordinate.
  • Use CoordinateSequence measure metadata to preserve XYM and XYZM layouts even when measure values are NaN.
  • Scan ambiguous three-spatial-dimension sequences for a finite Z beyond the first coordinate.
  • Preserve recoverable M and ZM metadata when empty Point and Polygon values, and their multipart children, are deserialized.
  • Reject recoverably heterogeneous layouts inside one Polygon or multipart geometry. The compact format has one coordinate-layout header for the entire value, so silently selecting or promoting a layout cannot preserve every component's identity. Heterogeneous layouts remain representable with a GeometryCollection.
  • Reject coordinate-sequence layouts wider than the format's existing XYZM model instead of silently truncating them.

The wire format and its four existing coordinate-layout tag values do not change, so previously serialized data remains readable.

There is one deliberate JTS boundary. JTS 1.20's default sequence factory represents both ordinary XY coordinates and declared XYZ coordinates whose Z values are all NaN as dimension=3, measures=0. Once JTS has made those representations identical, the serializer cannot recover the original declaration. This PR keeps the existing XY normalization for that ambiguous case instead of promoting ordinary XY payloads to XYZ. Consequently, an XYZ buffer produced by Python/GEOS whose Z ordinates are all NaN is indistinguishable after JTS reconstruction and normalizes to XY if it is serialized again on the JVM. Zero-member multipart and geometry-collection values similarly expose no child sequence from which to recover a declared dimension.

This is a pre-existing GeometrySerde issue identified while reviewing #3266, and is intentionally separate from the new equality predicate.

How was this patch tested?

  • mvn -pl common test: 1,299 tests passed.
  • Focused GeometryDimensionSerdeTest: 7 tests passed.
  • Spotless, Markdown lint, repository commit hooks, and git diff --check passed.

The new regression suite covers leading-NaN XYZ/XYM/XYZM values, later multipart members establishing a layout, all-NaN measure metadata, recoverable typed empty values and children, heterogeneous GeometryCollection children, mixed multipart and Polygon-ring layout rejection, and ordinary default-JTS XY normalization.

SerDe performance

The compact format was introduced to avoid the coordinate-by-coordinate byte conversion and stream growth costs of WKB. This PR adds one extra coordinate scan only for JTS's ambiguous dimension=3, measures=0 representation, so that path was benchmarked directly against the previous implementation and JTS WKB.

The table reports median-of-three-fork operation latency. “Full SerDe” is the sum of separately measured serialization and deserialization medians.

Geometry Coordinates Previous custom SerDe This PR Change JTS WKB This PR vs. WKB
LineString 1,000 4.173 µs 4.742 µs +13.6% 41.277 µs 8.7× faster
LineString 10,000 39.020 µs 45.036 µs +15.4% 404.198 µs 9.0× faster
Polygon 1,000 4.245 µs 4.896 µs +15.3% 45.392 µs 9.3× faster
Polygon 10,000 37.857 µs 45.880 µs +21.2% 416.244 µs 9.1× faster

The scan affects serialization only. For 1,000-coordinate default-JTS XY inputs, serialization changed from 1.043 to 1.541 µs for a LineString and from 1.244 to 1.787 µs for a Polygon. Non-empty deserialization is unchanged and showed no repeatable regression. Canonical CoordinateXY, XYZ, XYM, and XYZM paths were also effectively unchanged.

The benchmark used Corretto 17.0.13, JTS 1.20.0, Sedona's default Unsafe-backed geometry buffer, a fixed 512 MB G1 heap, prebuilt geometry/input objects, warmup followed by recalibration, nine samples per fork, and three fresh JVM forks. Every returned byte array or geometry escaped through a volatile sink, and all decoded dimensions and ordinates were validated outside the timed region. The WKB writer and reader were reused and preconfigured with the intended coordinate layout outside timing, which favors WKB. These are operation-level microbenchmarks; they do not include Spark SQL execution, parsing, network, or storage costs.

The result is a measurable cost on the ambiguous JTS representation, but it does not remove the compact format's performance advantage: the corrected implementation remains approximately 9× faster than generously configured WKB over full SerDe for these workloads. The format's fast deserialization path, which motivated the original geometry SerDe optimization, is unchanged.

Did this PR include necessary documentation updates?

  • Yes. The Sedona 2.0.0 release notes document the new mixed-layout rejection, its potential effect on ST_Collect, and the GeometryCollection workaround.

@jiayuasu jiayuasu added this to the sedona-2.0.0 milestone Aug 17, 2026
@jiayuasu
jiayuasu marked this pull request as ready for review August 17, 2026 22:48
@jiayuasu
jiayuasu merged commit d793315 into apache:master Aug 18, 2026
45 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Preserve recoverable coordinate dimensions in Java GeometrySerializer

1 participant