Skip to content

Support geopandas files in open_grid, and fix bug in MultiPolygon .shp file reads - #1904

Open
Sevans711 wants to merge 7 commits into
mainfrom
sevans/open_grid-geopandas
Open

Sevans711 wants to merge 7 commits into
mainfrom
sevans/open_grid-geopandas

Conversation

@Sevans711

@Sevans711 Sevans711 commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #1754

Overview

Adds support for geopandas files in open_grid by dispatching to Grid.from_file if appropriate based on file extension. (There are no other types of files currently supported by Grid.from_file but not open_grid.)

Large expansion of scope:

  • Also fixed the bug mentioned off-handedly in the initial issue report, i.e. "…Grid.from_file() might separately have bug(s) in it, because … plotting it doesn't give results which look like the US…". It turns out this was being caused by a simple typo in _read_multipolygon which kept swapping node_lon and node_lat lists while iteratively reading through the polygons in the underlying file. Added corresponding regression tests in test_geopandas.py which now pass but failed before this fix.

Here's a quick plot of ux.UxDataArray(xr.DataArray(np.arange(grid.n_face), dims="n_face"), uxgrid=grid) using grid = ux.Grid.from_file(os.path.join(..., "cb_2018_us_nation_20m.shp")) before and after the fix:
Screenshot 2026-11-16 at 8 37 07 PM

Minor expansion of scope:

  • Removed the obsolete test_read_shpfile in test/grid/grid/test_initialization, which simply pointed at a nonexistent file (grid_fire.shp).
  • Clarified in Grid.from_file docstring what happens to extra kwargs (they are silently discarded in the geopandas case).

PR Checklist

General

  • An issue is created and linked
  • Added appropriate labels (if your uxarray repo permissions allow it)
  • Filled out Overview and Expected Usage (if applicable) sections

Testing & Benchmarking

  • There is adequate test coverage of changes from this PR (add new tests if needed)
  • [N/A] If this PR could affect performance, ran ASV benchmarks and confirmed they show expected behavior (add a new benchmark if necessary)

Documentation and Examples

  • Docstrings updated with any function changes, and included in all new functions
  • [N/A] User (public) functions added to docs/api.rst; internal (private) function names start with an underscore (_)
  • [N/A] If touched any notebook files, cleared the output of all cells before committing
  • [N/A] If added new notebook files, put into appropriate directories and referenced in appropriate files

AI Disclosure

AI Usage: Claude for planning, some suggestions about where to look, and generating the test_multipolygon_preserves_lon_lat test. Some GitHub Copilot's inline code suggestions while implementing the fixes and tests.

  • I have tested and take responsibility for all AI-generated content in my PR.

(Grid.from_file currently only adds special support for geopandas files; otherwise it dispatches to xarray open_dataset just like open_grid does.)
@Sevans711 Sevans711 added the bug Something isn't working label Nov 17, 2026

@erogluorhan erogluorhan left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does what #1754 asks: ux.open_grid() now opens .shp/.geojson files, and the lon/lat swap fix in _read_multipolygon makes the US shapefile read correctly. The one-line root-cause fix and its targeted regression test are nicely done.

  • Ran the touched tests on the PR (test/io/test_geopandas.py, test/core/test_api.py, test/grid/grid/test_initialization.py): 130 passed. ruff check/format clean.
  • Mutation: with _geopandas.py reverted to main, test_read_shpfile and test_multipolygon_preserves_lon_lat both fail, so the regression tests catch the bug.
  • Swept open_grid on base vs PR: str and Path inputs, .geojson, chunks, merge_coincident_nodes, a missing file, an upper-case extension, and open_dataset(shp, data, ...). The problems found are in the inline comments.

Before merge: please look at the three inline comments. The upper-case extension is a nit.

Comment thread uxarray/core/api.py Outdated
and os.path.splitext(grid_filename_or_obj)[1] in Grid._GEOPANDAS_FILE_EXTENSIONS
):
# Special case for geopandas files (reject other kwargs, then use geopandas backend)
_got_kw = list(kwargs.keys()) + ([] if chunks is None else ["chunks"])

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two chunks cases don't reach this check correctly:

  • ux.open_grid(shp, chunks={"n_face": 10}) raises OSError: NetCDF: Unknown file format, not this TypeError. match_chunks_to_ugrid (line ~100) opens the file with xarray before this branch.
  • ux.open_dataset(shp, data, chunks=-1, chunk_grid=False) raises this TypeError even though the user asked not to chunk the grid. Checking grid_chunks instead of chunks would fix that.

Moving the geopandas branch above the chunk handling should fix both. The test only covers chunks="auto", which skips match_chunks_to_ugrid.

Comment thread uxarray/core/api.py
f"open_grid() for geopandas files (extension one of: {Grid._GEOPANDAS_FILE_EXTENSIONS}) "
f"got unexpected keyword argument(s) {_got_kw!r}, when filepath={grid_filename_or_obj!r}."
)
grid = Grid.from_file(grid_filename_or_obj, backend="geopandas")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A missing .shp used to raise FileNotFoundError. It now raises pyogrio.errors.DataSourceError, a RuntimeError subclass. The removed test_read_shpfile in test_initialization.py was catching exactly this: restored on this branch, it fails with DataSourceError. That test was still working, since a missing file is its intended input. Could you raise FileNotFoundError here (e.g. an os.path.exists check) and keep the test?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a new test, test_open_grid_raises_when_file_not_found, with a clearer docstring and purpose, instead of restoring the old test directly.

Comment thread uxarray/core/api.py Outdated
and os.path.splitext(grid_filename_or_obj)[1] in Grid._GEOPANDAS_FILE_EXTENSIONS
):
# Special case for geopandas files (reject other kwargs, then use geopandas backend)
_got_kw = list(kwargs.keys()) + ([] if chunks is None else ["chunks"])

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(non-blocking) merge_coincident_nodes is popped above, so the kwarg check never sees it. It is also never applied on this path: open_grid(shp, merge_coincident_nodes=False) succeeds silently, and the default True doesn't merge either. sample_chicago_buildings.geojson gives 181 nodes but only 178 unique (lon, lat) pairs. At least the docstring for merge_coincident_nodes should say it doesn't apply to geopandas files.

Comment thread uxarray/core/api.py Outdated

elif (
isinstance(grid_filename_or_obj, (str, os.PathLike))
and os.path.splitext(grid_filename_or_obj)[1] in Grid._GEOPANDAS_FILE_EXTENSIONS

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(nit) The extension match is case-sensitive: CB_2018.SHP still goes to xarray and fails with the original #1754 error. os.path.splitext(...)[1].lower() would handle that.

@Sevans711

Copy link
Copy Markdown
Collaborator Author

@erogluorhan Thank you for the review. I believe I was able to address all your comments via the latest changes. Please let me know if you notice anything I missed!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ux.open_grid() crashes on files supported by Grid.from_file()

2 participants