Skip to content

feat(indexing): make LazyArray indexing lazy by default - #4350

Draft
d-v-b wants to merge 38 commits into
zarr-developers:mainfrom
d-v-b:codex/indexing-lazy-default
Draft

feat(indexing): make LazyArray indexing lazy by default#4350
d-v-b wants to merge 38 commits into
zarr-developers:mainfrom
d-v-b:codex/indexing-lazy-default

Conversation

@d-v-b

@d-v-b d-v-b commented Sep 13, 2026

Copy link
Copy Markdown
Contributor

🤖 AI text below 🤖

LazyArray currently requires .lazy at each selection step; ordinary indexing materializes immediately. This revision makes the wrapper lazy by default: view[...][...], view.oindex[...], and view.vindex[...] return composed views, and iteration yields lazy first-axis views. result() and NumPy conversion materialize selected values.

Adds synchronous view.write(values) and assignment through the composed view. These update the original source, snapshot the RHS before mutation, and apply repeated destinations in C order with the last value winning. Affine writes use ascending basic slices; more general selections use scalar assignments. Writes are not transactional, and backend errors can leave partial writes. No asynchronous execution engine is introduced.

A separate EagerArrayAdapter supplies eager indexing for consumers such as Dask's from_array. Documentation, examples, and existing tests use the revised interface. This changes the experimental zarr-indexing wrapper; it does not add or change a zarr.Array.lazy accessor or claim full NumPy advanced-indexing compatibility.

Stack

Depends on #4345, #4346, #4347, #4348, and #4349. Their current heads are included in this branch. GitHub targets main because the prerequisite branches live in the fork; the full PR diff therefore includes those prerequisites. Rebase after they merge.

Focused LazyArray implementation commit

Validation

  • Full zarr-indexing suite, including doctests: 1,681 passed.
  • Recording-source tests verify indexing and iteration defer reads.
  • Write tests cover NumPy and Zarr, composed selections, negative strides, duplicate destinations, broadcasting, source aliases, masks, empty selections, and randomized selection chains checked against an independent coordinate oracle.
  • Documentation examples and strict MkDocs build passed.
  • Pyright: zero errors (100 warnings); applicable repository hooks passed.

Correct mathematical API documentation to match supported coordinate, grid, and chunk projection contracts.

Assisted-by: Codex:GPT-6
Retain the existing unsigned selector fix and update the unsupported mixed-dependency error assertion for general intersection routing.

Assisted-by: Codex:GPT-6
Use lexicographic tuple grouping when chunk indices contain negative values. Cover shared one-axis and two-axis array dependencies, repeated points, and extreme signed coordinates.

Assisted-by: Codex:GPT-6
Add synchronous writes through composed selections and an explicit eager adapter for array consumers. Cover deferred reads, source mutation, masks, aliases, and repeated destinations.

Assisted-by: Codex:GPT-6
The prerequisite runtime fixes are already present via the tested integration commits. Preserve the lazy-default revision while recording current prerequisite heads and their numbered changelogs.

Assisted-by: Codex:GPT-6
@github-actions github-actions Bot added the needs release notes Automatically applied to PRs which haven't added release notes label Sep 13, 2026
@read-the-docs-community

read-the-docs-community Bot commented Sep 13, 2026

Copy link
Copy Markdown

Remove implementation history and unsupported historical claims from source and test docstrings. Distinguish immutable coordinate mappings from mutable source values.

Assisted-by: Codex:GPT-6
The prerequisite runtime changes and the shared docstring correction are already integrated and tested. Record the updated heads while preserving the revised LazyArray API.

Assisted-by: Codex:GPT-6
@codecov

codecov Bot commented Sep 13, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.34%. Comparing base (ba883a5) to head (a6f9949).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4350   +/-   ##
=======================================
  Coverage   94.34%   94.34%           
=======================================
  Files          92       92           
  Lines       12935    12935           
=======================================
  Hits        12203    12203           
  Misses        732      732           
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs release notes Automatically applied to PRs which haven't added release notes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant