Skip to content

fix(indexing): make every view executable with its source grid - #4349

Draft
d-v-b wants to merge 16 commits into
zarr-developers:mainfrom
d-v-b:codex/indexing-partition-execution
Draft

fix(indexing): make every view executable with its source grid#4349
d-v-b wants to merge 16 commits into
zarr-developers:mainfrom
d-v-b:codex/indexing-partition-execution

Conversation

@d-v-b

@d-v-b d-v-b commented Sep 13, 2026

Copy link
Copy Markdown
Contributor

🤖 AI text below 🤖

Make every LazyArray view executable through projection-aware readers. Previously, part.view.result() and unpartitioned reads bypassed planning and passed projection=None, so they failed with readers that needed chunk-local coordinates.

LazyArray.result() now plans every nonempty read and supplies both the source-global transform and a chunk projection. Partition views retain their source grid and reader, so they can be read, indexed, or repartitioned like any other view. Callers schedule part.view.result() and assemble its values at part.out_selection; partitions describe the plan and placement rather than providing a second execution API.

Remove the special partition-local source window. All views now use the full source shape as base_shape, including partition views. with_parts and with_parts_per_axis therefore consistently describe the source grid. unpartitioned() treats the source as one grid cell and still supplies a projection. Independently executed views plan for their own output domain; their placement transform need not equal the parent's placement transform. Prepared parent plans remain reusable through view.result(parts=parts).

Validation: 1,672 tests and doctests passed, no skips, including Dask and TensorStore coverage; strict documentation build passed; Pyright reports 0 errors and 90 warnings. Tests cover independent, nested, reversed, and repartitioned views, masked output, reader errors, scalars, repeated indices, and empty selections. Six reader-contract cases reproduced the missing-projection failure before the change. Applicable repository hooks passed; the unrelated root mypy hook was skipped (it previously reported four unused-ignore errors in files unchanged by this PR). Independent review found no actionable issues and validated 8,222 randomized nested and repartitioned reads.

Correct mathematical API documentation to match supported coordinate, grid, and chunk projection contracts.

Assisted-by: Codex:GPT-6
Retain the existing unsigned selector fix and update the unsupported mixed-dependency error assertion for general intersection routing.

Assisted-by: Codex:GPT-6
Use lexicographic tuple grouping when chunk indices contain negative values. Cover shared one-axis and two-axis array dependencies, repeated points, and extreme signed coordinates.

Assisted-by: Codex:GPT-6
@github-actions github-actions Bot added the needs release notes Automatically applied to PRs which haven't added release notes label Sep 13, 2026
@read-the-docs-community

read-the-docs-community Bot commented Sep 13, 2026

Copy link
Copy Markdown

Remove implementation history and unsupported historical claims from source and test docstrings. Distinguish immutable coordinate mappings from mutable source values.

Assisted-by: Codex:GPT-6
Preserve current wire bounds validation and Dask tokenization while retaining the partition execution feature.

Assisted-by: Codex:GPT-6
@codecov

codecov Bot commented Sep 14, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.34%. Comparing base (0de6077) to head (ee341db).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4349   +/-   ##
=======================================
  Coverage   94.34%   94.34%           
=======================================
  Files          92       92           
  Lines       12935    12935           
=======================================
  Hits        12203    12203           
  Misses        732      732           
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Remove Partition.result and partition-local source windows. Preserve the source grid in derived views and supply projections for every LazyArray reader call, including unpartitioned reads.

Assisted-by: Codex:GPT-6
@d-v-b d-v-b changed the title feat(indexing): execute partitions with their reader context fix(indexing): make every view executable with its source grid Sep 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs release notes Automatically applied to PRs which haven't added release notes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant