[GH-3267] Preserve recoverable geometry dimensions - #3268
Merged
jiayuasu merged 2 commits intoAug 18, 2026
Conversation
jiayuasu
marked this pull request as ready for review
August 17, 2026 22:48
This was referenced Aug 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Did you read the Contributor Guide?
Is this PR related to a ticket?
[GH-XXX] my subject.Closes #3267.
What changes were proposed in this PR?
Sedona's compact Java geometry format already supports XY, XYZ, XYM, and XYZM coordinate-layout tags. However,
GeometrySerializerselected one of those tags by inspecting only a geometry's first coordinate and checking whether its Z and M values were finite. This could silently discard recoverable ordinates before a geometry crossed a SparkGeometryUDTor Flink serializer boundary. For example,LINESTRING Z (0 0 NaN, 1 1 3)was encoded as XY because its first Z value is NaN, so the later finite Z value was lost.This PR makes dimensional inference and empty reconstruction loss-aware:
CoordinateSequencemeasure metadata to preserve XYM and XYZM layouts even when measure values are NaN.GeometryCollection.The wire format and its four existing coordinate-layout tag values do not change, so previously serialized data remains readable.
There is one deliberate JTS boundary. JTS 1.20's default sequence factory represents both ordinary XY coordinates and declared XYZ coordinates whose Z values are all NaN as
dimension=3, measures=0. Once JTS has made those representations identical, the serializer cannot recover the original declaration. This PR keeps the existing XY normalization for that ambiguous case instead of promoting ordinary XY payloads to XYZ. Consequently, an XYZ buffer produced by Python/GEOS whose Z ordinates are all NaN is indistinguishable after JTS reconstruction and normalizes to XY if it is serialized again on the JVM. Zero-member multipart and geometry-collection values similarly expose no child sequence from which to recover a declared dimension.This is a pre-existing GeometrySerde issue identified while reviewing #3266, and is intentionally separate from the new equality predicate.
How was this patch tested?
mvn -pl common test: 1,299 tests passed.GeometryDimensionSerdeTest: 7 tests passed.git diff --checkpassed.The new regression suite covers leading-NaN XYZ/XYM/XYZM values, later multipart members establishing a layout, all-NaN measure metadata, recoverable typed empty values and children, heterogeneous GeometryCollection children, mixed multipart and Polygon-ring layout rejection, and ordinary default-JTS XY normalization.
SerDe performance
The compact format was introduced to avoid the coordinate-by-coordinate byte conversion and stream growth costs of WKB. This PR adds one extra coordinate scan only for JTS's ambiguous
dimension=3, measures=0representation, so that path was benchmarked directly against the previous implementation and JTS WKB.The table reports median-of-three-fork operation latency. “Full SerDe” is the sum of separately measured serialization and deserialization medians.
The scan affects serialization only. For 1,000-coordinate default-JTS XY inputs, serialization changed from 1.043 to 1.541 µs for a LineString and from 1.244 to 1.787 µs for a Polygon. Non-empty deserialization is unchanged and showed no repeatable regression. Canonical
CoordinateXY, XYZ, XYM, and XYZM paths were also effectively unchanged.The benchmark used Corretto 17.0.13, JTS 1.20.0, Sedona's default Unsafe-backed geometry buffer, a fixed 512 MB G1 heap, prebuilt geometry/input objects, warmup followed by recalibration, nine samples per fork, and three fresh JVM forks. Every returned byte array or geometry escaped through a volatile sink, and all decoded dimensions and ordinates were validated outside the timed region. The WKB writer and reader were reused and preconfigured with the intended coordinate layout outside timing, which favors WKB. These are operation-level microbenchmarks; they do not include Spark SQL execution, parsing, network, or storage costs.
The result is a measurable cost on the ambiguous JTS representation, but it does not remove the compact format's performance advantage: the corrected implementation remains approximately 9× faster than generously configured WKB over full SerDe for these workloads. The format's fast deserialization path, which motivated the original geometry SerDe optimization, is unchanged.
Did this PR include necessary documentation updates?
ST_Collect, and theGeometryCollectionworkaround.