Repository navigation
Description
Proposed new feature or change:
While diagnosing #1444 and developing #1667, we noticed an increasing scope. It started with being concerned about a specific instance of time leaking onto a grid object, causing downstream issues while using subset.
After finding a fix for this scenario, we decided we wanted to make a more robust grid object cleaning function, that can clean grid objects in a way that avoids other potentially damaging situations like this. Doing so we found other interactions with the IO, like #1717 that showed we need to further investigate making rules around cleaning, as there is a surprisingly unique range of incoming data variables, with characteristics that are uncommon, but comparable to what we imagine "leaked" data to look like.
A few main goals:
- Reduce the amount of potential incoming grid
data variables- Based on what we don't need or don't expect to have after a grid file is read in
- Reduce the amount of potential incoming
coordinates- Based on what we don't need or don't expect to have after a grid file is read in, and if they are 0 dimensional (dims=() or "scalar")
- Make a method that can run without needing model/format specific solutions
- Ensure that no models have their unique attributes removed if they are needed
- Ensure that the function is placed within the file read process so that it doesn't disrupt other existing parts
Added notes:
- We are interested in keeping at a minimum our existing
GRID_DIMS:
['n_edge', 'n_face', 'n_max_edge_edges', 'n_max_edge_faces',
'n_max_edge_nodes', 'n_max_face_edges', 'n_max_face_faces',
'n_max_face_nodes', 'n_max_node_edges', 'n_max_node_faces',
'n_node', 'two']- We want cases like we've seen, and cases we've not seen, to be covered by this fix (in these examples,
timeshould be assumed to be any variable we don't want/need):timepresent as a dimension coordinate (1-D, time(time), ndim == 1)timepresent as a scalar (0-dimensional) coordinate (dims == (), ndim == 0)timepresent as a dimension without a coordinate (a bare time dimension carrying data variables, no time coordinate)- Meaningful dimensions (dimensions within
DIM_NAMES) that should be there, but aren't, due to it's unique IO process
These are some of the instances we've seen and want to keep them all in mind when finding a solution. This may be blocked until we understand #1717 better.
Metadata
Metadata
Assignees
Labels
Type
Fields
Priority
Projects
- StatusShow more project fields🏗 In progress
Activity
Sevans711 commentedon Aug 31, 2026
Thank you for opening an issue on this! This description mostly matches my understanding of the problem. I have a few tiny comments below. We might already be on the same page about these but I'm including them here to clarify:
Rather, I was thinking the idea was to put the function directly within the Grid object constructor (i.e.
Grid.__init__). I wouldn't consider this to be part of the file read process; it acts on the dataset that gets provided toGrid, regardless of whether the grid is being created from file data or from an xarray.Dataset provided directly asGrid(grid_ds, ...).These are good examples. If the goal is a comprehensive list I would add a few kinds of coordinates that could theoretically pop up and aren't covered above:
time_since_epochpresent as a non-dimension coordinate (e.g. 1D coordinate withtimedimension)special_indexpresent as a (non-dimension) coordinate with N>1 dimensions (e.g.special_indexwith dims "time" and "elevation", with valuelen(time) * time_index + elevation_index; the value itself isn't important, just hoping to clarify that 2D+ coords across non-grid dims are possible)zpresent as a (non-dimension) coordinate with N>1 dimensions, at least one of which is not a grid dimension (e.g.x,y,zrepresenting geocentric coordinates of data could each have dimensions "node_lon", "node_lat", "elevation"; elevation is not a grid dimension so these coords would need to be removed.)I agree this would be blocked by #1717, though to me it seems like 1717 is already a well-understood and clearly-reported issue with a simple fix (rename "n2" to "two" in fesom io functions), so I would maybe rephrase to "until #1717 is solved and merged" instead. Is 1717 more complicated than I think?
dylannelson commentedon Aug 31, 2026
@Sevans711
I don't know, we'll see as I get into it, haven't started yet.
What is the value of discussing semantics like this? Saying "X is blocked by Y" and "X is blocked until Y is solved and merged" sounds basically the same. I need to do X before Y. That's all I'm trying to say. Not making any statement on it's difficulty, and if I was, I'd do it on that issue's discussion, no?
Sevans711 commentedon Aug 31, 2026
I think the only value of it here is just to make sure we agree on the underlying ideas. For some reason the phrasing "blocked until we understand 1717" sounded to me like a claim about the difficulty of 1717. Your reply above helped me see that wasn't actually your intention, so I believe we are on the same page here - thank you for clarifying! I agree it makes the most sense to put claims about an issue's difficulty directly on that issue's discussion.