Skip to content

Merge coincident nodes when constructing any Grid - #1692

Merged
cmdupuis3 merged 57 commits into
mainfrom
rajeeja/coincident-nodes
Nov 15, 2026
Merged

cmdupuis3 merged 57 commits into
mainfrom
rajeeja/coincident-nodes

Conversation

@rajeeja

@rajeeja rajeeja commented Aug 19, 2026 •

Copy link
Copy Markdown
Contributor

Closes #865

  • Grids with duplicate node indices were rejected outright; the dual is now built after canonicalizing those indices in the face-node connectivity.
  • geoflow-small previously could not produce a dual at all and now yields 3840 faces; the test asserts this and fails on main.
  • _find_duplicate_nodes is vectorized with np.unique instead of a per-node dict.
  • Grid.get_dual(check_duplicate_nodes=...) is now ignored and deprecated rather than removed, so existing callers keep working.
  • UxDataArray.get_dual and UxDataset.get_dual still raise GridInvalidError, since node-centered data cannot be remapped onto a merged node set.
  • The search is skipped for formats whose faces index an already-unique vertex list (MPAS, HEALPix), where it cannot find anything to merge. Formats storing per-face corners (SCRIP, Exodus, GEOS-CS) still run it, and UGRID is deliberately not skipped since it permits either layout.
  • Nodes coincident on the sphere but differing in (lon, lat), such as poles and the antimeridian, are handled by Merge coincident pole and antimeridian nodes in structured grids #1690, now merged into main.
  • A user-facing merge tolerance is tracked separately in Allow specifying the coincident-node merge tolerance #1770.

Match nodes in Cartesian space rather than the lon/lat plane so pole and
antimeridian nodes are recognized as the same point, and store the resulting
polar faces as triangles instead of quads with a repeated corner.
Canonicalize duplicate node indices in the face-node connectivity before
building the dual, so grids with repeated nodes produce a correct dual instead
of being rejected. Also vectorize the duplicate lookup and deprecate the now
redundant check_duplicate_nodes argument.
@rajeeja rajeeja self-assigned this Aug 19, 2026
@rajeeja
rajeeja requested a review from Sevans711 August 19, 2026 22:06

@Sevans711 Sevans711 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly looks like a good fix! I think it still needs a little bit of extra work, and I left some inline comments accordingly.

Comment thread uxarray/grid/validation.py Resolved
Comment thread uxarray/grid/validation.py Resolved
Comment thread uxarray/grid/connectivity.py Resolved
Comment thread test/grid/grid/test_core.py Resolved
Comment thread test/grid/grid/test_core.py Resolved
@Sevans711

Copy link
Copy Markdown
Collaborator

Actually, apologies for not including this during the original review, comment but one more thought: does this actually fully close the original issue? The issue writeup makes it sound to me like duplicate nodes should be handled immediately upon constructing the grid, not just during one functionality (get_dual). Is there a reason that duplicate nodes should be handled only during get_dual, instead of immediately? (Do all other current/planned functions work properly regardless of whether there are duplicate nodes?)

@Sevans711 Sevans711 added the bug Something isn't working label Aug 21, 2026
rajeeja and others added 11 commits August 21, 2026 11:00
float32 input (e.g. real climate datasets) silently ran the whole
xyz/tolerance pipeline at float32 precision, causing pole/antimeridian
merges to fail or merge only partially.
Match nodes in Cartesian space rather than the lon/lat plane so pole and
antimeridian nodes are recognized as the same point, and store the resulting
polar faces as triangles instead of quads with a repeated corner.
float32 input (e.g. real climate datasets) silently ran the whole
xyz/tolerance pipeline at float32 precision, causing pole/antimeridian
merges to fail or merge only partially.
Extend #865's fix beyond the dual mesh: canonicalize duplicate/coincident
node indices in connectivity for every Grid construction path, not just
construct_dual. Detection is now tolerance-based (unit-sphere chordal
distance) instead of exact lon/lat match, so pole-degenerate duplicates
are also caught. Node coordinate arrays are left untouched by design;
only connectivity is remapped to canonical indices, with any resulting
repeated face corners collapsed.
construct_dual no longer needs its own per-call duplicate detection and
remap, and get_dual() no longer needs to hard-gate on duplicate node
indices, since Grid construction now canonicalizes them structurally
before any of this code runs.
Since duplicate node coordinates are intentionally left unreferenced by
connectivity, a node KDTree/BallTree built over the raw coordinate array
could select an index no face actually points to, silently returning
empty or wrong nearest-neighbor results. Build the "nodes" tree only
over live (referenced) indices and translate query results back to
original index space.
polars' unique() with maintain_order unset does not guarantee row order
across runs, so the node index assigned to a given corner coordinate
could vary between reads of the same file. This is normally harmless,
but it made canonical-node selection for coincident duplicates
(e.g. pole points with differing longitude) flaky from run to run.
test_dual_duplicate: validate() now succeeds since connectivity is fully
canonicalized (duplicate coordinates remain by design, but nothing
references a dead index anymore). test_grid_nn_subset: max valid k for a
node search is now bounded by the live node count, not raw node count.
test_to_geodataframe_preserves_antimeridian_faces: pole-coincident
corners with differing longitude are now also merged, shifting the
antimeridian face count.
@rajeeja
rajeeja requested a review from Sevans711 August 21, 2026 19:58
Merging pole-adjacent duplicate nodes was collapsing each face's own
locally-meaningful longitude at the pole into one arbitrary canonical
value, which corrupted lat/lon bounds and broke zonal weight
computation for cube-sphere grids near the poles.

@Sevans711 Sevans711 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: please change the title of this PR to reflect the full scope. Duplicate nodes are now handled directly whenever constructing a Grid, not just when constructing the dual mesh. I was confused about why _check_duplicate_nodes_indices checks were removed despite construct_dual() being unchanged. I think the reason is because all Grid objects are now guaranteed to not have duplicate nodes.

I tried leaving a full review but I kept getting the feeling that something weird was happening, a nagging feeling like "hey I think I've looked at this code before and left an inline comment, why am I reviewing it again?" Then I realized many of the changes here are also in #1690 which hasn't been merged yet. That led to duplicating review work and will probably lead to needing to apply fixes multiple times. I'm not sure the cleanest way forwards at this point, but I might suggest waiting to merge this until after #1690 gets merged. At least, I will want to do another close review of this PR after that PR merges, because I lost track of which things I already looked at closely and which I need to consider again.

Comment thread test/io/test_structured.py Resolved
Comment thread uxarray/grid/validation.py Resolved
Comment thread uxarray/io/_structured.py Resolved
…des' into rajeeja/coincident-nodes

# Conflicts:
#	test/io/test_structured.py
#	uxarray/core/dataarray.py
#	uxarray/core/dataset.py
#	uxarray/grid/grid.py
#	uxarray/io/_structured.py
"Coincident nodes" is the domain term used by TempestRemap, MOAB, and our own
validation module; "dedupe" reads like a dataframe operation. Also note in the
docstring that we keep the redundant coordinates and only remap connectivity,
where TempestRemap and MOAB delete and renumber.
_coincident_node_canonical_indices exempts pole nodes from merging so each face
touching a pole keeps its own longitude. The mask used np.isclose(|z|, 1.0,
atol=tolerance), which is wrong twice: tolerance is a chord radius everywhere
else in the function, and np.isclose's default rtol=1e-5 swamped atol entirely.
The carve-out spanned 1 - |z| <= 1.001e-5, a chord of 4.5e-3 or ~28 km on Earth,
so any node within 28 km of a pole was silently exempt from merging.

For a point at colatitude t, 1 - |z| = 1 - cos(t) = chord**2 / 2, so the correct
cap is tolerance**2 / 2 with rtol pinned to zero.
Comment thread uxarray/core/dataarray.py Resolved
Comment thread uxarray/grid/connectivity.py Resolved
Comment thread uxarray/grid/connectivity.py Resolved
Comment thread uxarray/grid/connectivity.py Resolved
Comment thread uxarray/grid/connectivity.py Resolved
Comment thread uxarray/grid/validation.py Resolved

@Sevans711 Sevans711 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like some great progress! All of my previous inline comments have been addressed, the benchmarks aren't showing slowdowns, and my previous timing test of ux.open_grid() on a 15km MPAS grid now shows extremely similar speeds (within 1%) on this PR and on main.

I will plan to take one more look through these changes before approving, but I would prefer to delay that until at least a majority of @cmdupuis3's inline comments have been addressed. (Currently I see many unresolved inline comments, and no direct replies, so at a glance I can't tell whether they have been addressed yet.) Please feel free to re-request a review from me once more of those points have been resolved!

@cmdupuis3 cmdupuis3 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some snippets worth investigating:

ux.open_grid("test/meshfiles/ugrid/geoflow-small/grid.nc").get_dual().n_face             # 3842
ux.open_grid("test/meshfiles/ugrid/geoflow-small/grid.nc", chunks={}).get_dual()          # GridInvalidError
ux.Grid.from_topology(np.array([0., 1, 1, 0, np.nan]), np.array([0., 0, 1, 1, np.nan]),
                      np.array([[0, 1, 2, 3]]))
# main: builds; branch: ValueError: Node XYZ coordinates must contain only finite values.
F = 1e20   # face 1 is a triangle padded with the fill value
clon = np.array([[0., 1, 1, 0], [1, 2, 1, F]]); clat = np.array([[0., 0, 1, 1], [0, 0, 1, F]])
xr.Dataset({
    "grid_center_lon": ("grid_size", [0.5, 1.33]), "grid_center_lat": ("grid_size", [0.5, 0.33]),
    "grid_corner_lon": (("grid_size", "grid_corners"), clon, {"units": "degrees", "_FillValue": F}),
    "grid_corner_lat": (("grid_size", "grid_corners"), clat, {"units": "degrees", "_FillValue": F}),
    "grid_imask": ("grid_size", np.array([1, 1], np.int32)), "grid_dims": ("grid_rank", np.array([2], np.int32)),
}).to_netcdf("scrip_fill.nc")

ux.open_grid("scrip_fill.nc")
# main:   opens (face_lon from grid_center_lon; the padded face's area is NaN, as before)
# branch: ValueError: Node XYZ coordinates must contain only finite values.
p = ux.open_grid("test/meshfiles/exodus/outCSne8/outCSne8.g")
fnc = np.vstack([p.face_node_connectivity.values, p.face_node_connectivity.values[:1]])  # repeated face
g = ux.Grid.from_topology(p.node_lon.values, p.node_lat.values, fnc)
g.get_dual()                                                                  # still uses node 384
ux.UxDataArray(np.zeros(g.n_node), dims=["n_node"], uxgrid=g).get_dual()      # 384 merged into 0
g.get_dual().get_dual()                                                       # GridInvalidError

This one's more of a robustness issue, not necessarily blocking; one bad face renders the whole grid unopenable.

ux.open_grid("bowtie.nc")                                      # GridInvalidError
ux.open_dataset("bowtie.nc", "bowtie.nc")                      # GridInvalidError
ux.Grid.from_dataset(ds, merge_coincident_nodes=False)         # GridInvalidError (flag isn't forwarded)
ux.open_grid("bowtie.nc", merge_coincident_nodes=False)        # TypeError (goes to the xarray backend)

Comment thread uxarray/grid/validation.py Resolved

@Sevans711 Sevans711 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After one final read through, this looks reasonable to me. All of my comments have been addressed, and most other comments have been addressed as well. I'm assuming that @cmdupuis3 would take a closer look at the remaining suggestions to investigate the snippets above (in #1692 (review)); I didn't take a closer look at them, personally. I'm happy to approve this one, thank you for all your efforts on it!

@cmdupuis3 cmdupuis3 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's an issue with the merge_coincident_nodes setting's persistence:

import numpy as np, xarray as xr, uxarray as ux

# outCSne8 plus one extra node that duplicates node 0; one face uses the copy.
p = ux.open_grid("test/meshfiles/exodus/outCSne8/outCSne8.g")
lon = np.append(p.node_lon.values, p.node_lon.values[0])
lat = np.append(p.node_lat.values, p.node_lat.values[0])
fnc = p.face_node_connectivity.values.copy()
r, c = np.argwhere(fnc == 0)[0]
fnc[r, c] = lon.size - 1
ds = xr.Dataset({
    "mesh": ((), 0, {"cf_role": "mesh_topology", "topology_dimension": 2,
                     "node_coordinates": "lon lat", "face_node_connectivity": "faces"}),
    "lon": ("nodes", lon, {"standard_name": "longitude"}),
    "lat": ("nodes", lat, {"standard_name": "latitude"}),
    "faces": (("faces_dim", "corners"), fnc, {"start_index": 0}),
})

ux.open_grid(ds,         merge_coincident_nodes=False).get_dual()  # GridInvalidError: Duplicate nodes found
ux.open_grid(ds.chunk(), merge_coincident_nodes=False).get_dual()  # 386 faces: the opt-out is ignored

Probably just need to save the setting:

@@ Grid.__init__
         self._is_subset = is_subset
+        self._merge_coincident_nodes = merge_coincident_nodes

@@ Grid._prepare_for_dual
         return type(self)(
             grid_ds,
             self.source_grid_spec,
             source_dims_dict=self._source_dims_dict,
+            merge_coincident_nodes=self._merge_coincident_nodes,
         )

Also, merge_coincident_nodes=False is now how users open a file that fails with "nonconsecutive repeated corners", but neither open_grid nor Grid.from_dataset lists it in their docstrings. The comment in Grid.__init__ (grid.py:194) also still calls it internal. Both functions take it through **kwargs, so a user can't find the name from the signature. A short parameter entry in both docstrings would fix this.

@rajeeja

rajeeja commented Nov 12, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the review and for catching this. Fixed in 673018d: deferred dual preparation now preserves the merge setting, and both entry points document the opt-out.

@cmdupuis3 cmdupuis3 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The last bug is fixed, approving

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working run-benchmark Run ASV benchmark workflow

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Handling Duplicate Node Indices

3 participants