feat(indexing): make LazyArray indexing lazy by default

packages/zarr-indexing/CONTRIBUTING.md

@@ -8,16 +8,16 @@ just test        # run the test suite (extra args go to pytest)
 just lint        # ruff, same invocation as CI
 just typecheck   # pyright, same invocation as CI
 just docs-check  # strict build of the docs site
-just check       # all of the above
+just check       # the checks above plus TensorStore parity
 just docs-serve  # serve the docs site locally
 ```
 
-Run them from this directory, or from anywhere in the repository as
+Run them from this directory, or from the repository root as
 `just packages/zarr-indexing/<recipe>`.
 
-The test recipe runs against the workspace-root environment, because the
-chunk-resolution tests exercise this package against `zarr`'s chunk grids and
-`zarr` is deliberately not a dependency of this package.
+The test recipe layers this package into the repository-root environment.
+Chunk-resolution tests use this package’s own grids; the Dask example also
+uses `zarr`, which is not a dependency of the base indexing package.
 
 ## License
 

packages/zarr-indexing/README.md

@@ -5,19 +5,19 @@ Composable, lazy coordinate transforms for Zarr array indexing.
 Documentation: <https://zarr-indexing.readthedocs.io/>
 
 This package implements TensorStore-inspired index transforms. The core idea:
-every indexing operation (slicing, fancy indexing, etc.) produces a coordinate
-mapping from user space to storage space. These mappings compose lazily — no
-I/O until you explicitly read or write.
+each supported indexing operation (slicing, fancy indexing, etc.) produces a coordinate
+mapping from user space to storage space. These mappings compose without reading selected source values. `LazyArray`
+materializes them on request; the transform algebra itself performs no source I/O.
 
 Key types:
 
-- `LazyArray` — wraps a system-memory/basic-indexing source and adds a `.lazy`
-  accessor: `LazyArray.from_numpy(numpy_array).lazy[10:50, ::2].lazy.oindex[[3, 1, 1], :]`
+- `LazyArray` — wraps a system-memory/basic-indexing source with lazy indexing:
+  `LazyArray.from_numpy(numpy_array)[10:50, ::2].oindex[[3, 1, 1], :]`
   composes a transform and returns a new view without reading data, and
   `result()` materializes it into owned system memory. `LazyArray(source)` uses
-  the conservative basic reader; `from_numpy` explicitly selects NumPy's
-  optimized reader. Device arrays require an explicit custom reader responsible
-  for transferring values into the supplied system-memory output buffer.
+  the basic reader; `from_numpy` selects `numpy_reader`, which currently uses
+  the same slab-and-gather implementation. Device sources that refuse NumPy
+  conversion need a custom reader to transfer values into the output buffer.
 - `Reader` — the explicit backend execution boundary: transforms say which
   values belong in the result, while readers say how a backend obtains them
 - `IndexDomain` — a rectangular region of integer coordinates
@@ -32,9 +32,11 @@ Key types:
   dimension can depend on the input
 - `compose` — chain two transforms into one
 
-The package depends only on NumPy and the standard library; it does not import
-`zarr`. It is developed in the [zarr-python](https://github.com/zarr-developers/zarr-python)
-repository and consumed by `zarr` to resolve array indexing operations.
+The base package depends on NumPy and the standard library; its optional testing
+module also requires Hypothesis. The package does not import `zarr`. It is developed
+in the [zarr-python](https://github.com/zarr-developers/zarr-python) repository,
+and its examples include reading Zarr arrays through Dask. Installing it does not
+replace Zarr's indexing implementation.
 
 ## Installation
 

packages/zarr-indexing/benchmarks/README.md

@@ -22,6 +22,19 @@ whether input arrays and transform construction are included. Repeated local
 column access measures cache reuse, which trades allocation against retained
 memory. Bounded coordinate batches avoid constructing the full coordinate array.
 
+The current script reports `peak_mib` as the incremental peak tracked by
+`tracemalloc` during a separate invocation after the timed calls. It is not
+process RSS or total retained memory. Inputs and most transforms are constructed
+before measurement. The case walks construct plans and consume projections
+without retaining them; `all_coordinates` materializes the complete coordinate
+array. There is no bounded-batch coordinate workload in this script.
+
+`local_rows` reuses one prepared table: its first timed invocation populates the
+local-coordinate cache, while later timed invocations and the allocation probe
+reuse that cache. Its median mixes a cold first call with warm calls, and its
+reported peak excludes the already-retained cache. A separate fresh-table
+measurement would be needed to quantify cold cache construction.
+
 These scripts measure planning rather than codec or storage throughput. Repeat
 measurements with alternating operation order before interpreting small timing
 differences. Preserve raw benchmark output as an experiment artifact rather than

packages/zarr-indexing/benchmarks/chunk_planning.py

@@ -25,6 +25,11 @@
 
 
 def measure(operation: Callable[[], Any], repeats: int) -> dict[str, float]:
+    """Time repeated calls, then trace one additional call's incremental peak.
+
+    State retained by earlier calls can affect the extra call; this is not a
+    measurement of process RSS or all memory owned by the operation's inputs.
+    """
     samples = []
     for _ in range(repeats):
         start = time.perf_counter()
@@ -84,6 +89,8 @@ def read_local_rows() -> None:
         for row in range(len(table)):
             table.local[table.run(row)]
 
+    # The first timed call fills table.local; later calls, including the traced
+    # allocation probe, reuse it. The table and retained cache are not re-created.
     results["local_rows"] = measure(read_local_rows, args.repeats)
     part = plan_chunks(
         IndexTransform.from_shape((100, 100, 100)),

packages/zarr-indexing/changes/4345.bugfix.md

@@ -0,0 +1 @@
+Reject invalid wire index-array values and unrepresentable normalized bounds/ranks, and raise an explicit error for unsupported intersections sharing an affine and lookup input axis instead of returning incorrect coordinates. Group negative chunk-coordinate tuples without merging distinct chunks.

packages/zarr-indexing/changes/4345.doc.md

@@ -0,0 +1,3 @@
+Correct indexing, reader, serialization, cache, and integration descriptions to match supported behavior; qualify NumPy/TensorStore compatibility and performance claims.
+
+Describe current contracts in source and test docstrings instead of narrating prior implementations. Clarify that immutable index coordinates do not snapshot source values.

packages/zarr-indexing/changes/4346.misc.md

@@ -0,0 +1 @@
+Expand planner property tests across signed origins and chunk IDs, custom grids, mixed affine and lookup dependencies, duplicate coordinates, and empty domains. Verify exact request coverage and storage mapping with an independent pointwise oracle.

packages/zarr-indexing/changes/4347.bugfix.md

@@ -0,0 +1 @@
+Reject constrained `index_array_bounds` when lowering JSON to an engine transform or output map, instead of silently dropping the constraint. Message normalization continues to preserve bounds for consumers that support them.

packages/zarr-indexing/changes/4348.bugfix.md

@@ -0,0 +1 @@
+Define an explicit source token contract: hash plain NumPy arrays without object fields, delegate source hooks and propagate their errors, and reject unsupported sources without converting or serializing them. Remove optional-Dask and per-call UUID fallbacks. Document hashing costs, mutation limits, and `name=False` for Dask wrapping opaque sources.

packages/zarr-indexing/changes/4349.feature.md

@@ -0,0 +1 @@
+Add `Partition.result()` to execute a planned partition independently with the same global transform and chunk projection supplied during parent assembly.

packages/zarr-indexing/changes/4350.feature.md

@@ -0,0 +1,5 @@
+Make `LazyArray` indexing lazy by default: use `view[...]`, `view.oindex[...]`,
+and `view.vindex[...]` directly, without a `.lazy` accessor. Iteration yields
+lazy views; `result()` and NumPy conversion materialize values. Add synchronous
+`write(values)` and assignment through composed views, and an explicit
+`EagerArrayAdapter` for consumers such as Dask that require eager indexing.

packages/zarr-indexing/docs/api/grid.md

@@ -11,8 +11,9 @@ objects whose `shape` is the valid data size and whose `codec_shape` preserves
 the full codec-buffer size at a regular-grid boundary.
 
 `dimension_grids_from_chunks` returns these compact dimensions: integer chunk
-shapes become `FixedDimension` instances and explicit per-axis edge sequences
-become `VaryingDimension` instances. `DimensionGridLike` remains the narrow
+shapes become `FixedDimension` instances and positive per-axis edge sequences
+become `VaryingDimension` instances. An empty-axis sequence of zeros (including
+an empty sequence) becomes `FixedDimension(size=0, extent=0)`. `DimensionGridLike` remains the narrow
 protocol used by the chunk planner, while `EdgeDimensionGrid` is kept for
 explicit edge-based and coordinate-origin examples.
 

packages/zarr-indexing/docs/api/index.md

@@ -25,12 +25,11 @@ and the wire format built on top of it.
 - [`zarr_indexing.domain`](domain.md) — `IndexDomain`, a rectangular region of
   integer coordinates with an explicit (possibly non-zero) origin
 - [`zarr_indexing.output_map`](output_map.md) — `ConstantMap`, `DimensionMap`,
-  and `ArrayMap`: three representations of a set of integer coordinates, one
-  per storage dimension
+  and `ArrayMap`: three coordinate mappings that preserve order and duplicates,
+  one per storage dimension
 - [`zarr_indexing.transform`](transform.md) — `IndexTransform`, which pairs a
   domain with output maps, plus the indexing (`[...]`, `.oindex`, `.vindex`),
-  `intersect`, and `translate` operations, and `selection_to_transform`
-  transforms into one
+  `intersect`, `translate`, and `compose` operations, and `selection_to_transform`
 
 **Chunk resolution**
 
@@ -49,7 +48,7 @@ and the wire format built on top of it.
 **Lazy arrays**
 
 - [`zarr_indexing.lazy_array`](lazy_array.md) — `LazyArray`, a wrapper for
-  system-memory/basic-indexing sources that adds a `.lazy` accessor for
+  system-memory/basic-indexing sources with lazy indexing for
   TensorStore-style deferred indexing, plus `Partition` and `parts()` /
   `with_parts()`, which determine the boxes a read is broken into. Device
   sources require an explicit custom reader that transfers into the supplied

packages/zarr-indexing/docs/api/lazy_array.md

@@ -2,26 +2,39 @@
 title: lazy_array
 ---
 
-`LazyArray.lazy[...]` is metadata-only: every derived view keeps the same
+`LazyArray[...]` is metadata-only: every derived view keeps the same
 reader and composes its transform without reading data. `result()` allocates
 owned system memory, then calls that reader once for each projected part.
 Rectangular parts write directly into their final slices; advanced placement
 may first use an owned dense temporary. `LazyArray(source)` assumes only basic
-indexing, while `LazyArray.from_numpy(array)` explicitly selects NumPy's
-optimized reader.
+indexing, while `LazyArray.from_numpy(array)` selects `numpy_reader`. Both
+currently use the same slab-and-gather implementation.
 
 The built-in readers lower through NumPy system memory and support sources
-whose basic reads can be converted there. They do not implicitly transfer
-device arrays; a device source needs an explicit custom reader that transfers
-into the supplied system-memory output. Derived views and parts share their
+whose basic reads can be converted there. A device source that refuses NumPy
+conversion needs a custom reader that transfers into the output buffer. Derived views and parts share their
 reader and part views may be materialized concurrently, so stateful readers
 must synchronize their own mutable state.
 
 Every public `Partition.view.transform` directly maps that view's zero-origin
 coordinates into its raw `Partition.view.array`, including for non-first
 partitions. `Partition.projection.chunk_transform` intentionally stays local to
-the selected chunk. During materialization the reader receives both frames in
-one `ReadContext`: the public global transform in `context.transform` and the
-same local plan in `context.projection`.
+the selected chunk. During parent materialization (`view.result(parts=parts)`) the reader receives
+both frames in one `ReadContext`: the public global transform in `context.transform` and the
+same local plan in `context.projection`. Use `part.result()` to execute a
+partition independently with both frames. Direct `part.view.result()` calls
+resolve the general view with no projection.
+
+`view[key]`, `view.oindex[key]`, and `view.vindex[key]` return lazy views,
+and iteration yields lazy first-axis views. `result()` and `numpy.asarray(view)`
+materialize values. `view.write(values)` synchronously writes to the original
+source through the composed transform and returns `None`; `view[key] = values`
+writes a selected sub-view. Writes require a writable source.
+
+For consumers requiring eager indexing, import `EagerArrayAdapter` from
+`zarr_indexing` and wrap the view. The adapter delegates shape, rank, dtype,
+NumPy conversion, and tokenization to the view, but materializes each
+`adapter[key]`. Use it with `dask.array.from_array`; direct lazy indexing
+is not a reliable Dask block-read interface.
 
 ::: zarr_indexing.lazy_array

packages/zarr-indexing/docs/api/reader.md

@@ -12,7 +12,8 @@ ownership.
 `Reader.read_into(source, context, out)` receives a `ReadContext` whose
 `transform` maps zero-origin output-buffer coordinates to global coordinates in
 `source`, with `context.transform.domain.shape == out.shape`. Its optional
-`projection` is the existing plan for a partitioned read. The projection's
+`projection` is the existing plan for a partitioned read. Both parent
+assembly and independent `Partition.result()` calls supply that projection. The projection's
 `chunk_transform` remains chunk-local, its `cell_transform` describes result
 placement, and its `chunk_domain` describes the grid cell. The global read
 transform and the projection's chunk transform deliberately use different
@@ -34,7 +35,7 @@ class RecordingReader:
         self.calls = []
 
     def read_into(self, source, context, out, /):
-        self.calls.append((source, context, out))
+        self.calls.append((source, context, out.shape, out.dtype))
         self.inner.read_into(source, context, out)
 
 
@@ -44,7 +45,8 @@ view = LazyArray.from_numpy(array).with_reader(outer)
 values = view.result()
 ```
 
-Both wrappers observe the same three objects, in outer-to-inner order. This
+Both wrappers observe the same arguments, in outer-to-inner order, and log
+output metadata without retaining the output buffer. This
 delegation pattern supports policies such as logging and caching without
 library-defined wrapper primitives.