Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions packages/zarr-metadata/changes/4635.bugfix.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Strategies in `zarr_metadata.testing.strategies` that draw codec pipelines, such as `array_documents` and `pipelines`, now draw the same examples on every derandomized Hypothesis run. A codec was sampled from options that hold strategies built during the draw, which Hypothesis labels by memory address, so the examples its mutator produced varied from one process to the next.
1 change: 1 addition & 0 deletions packages/zarr-metadata/changes/4636.bugfix.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
The `rectilinear` chunk grid now refuses an empty list of chunk edge lengths, such as `"chunk_shapes": [[]]`, even along a dimension of length 0. zarr-extensions#78 made the list form non-empty; a dimension with no chunks writes a bare integer instead. The JSON Schema says `minItems: 1` for the list, and `chunk_grid_configurations` no longer draws an empty list.
1 change: 1 addition & 0 deletions packages/zarr-metadata/changes/4636.feature.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
`zarr_metadata.typed_json.check` and definitions' configurations now read more annotated-types constraints, as pydantic does: `MinLen`, `MaxLen` and `Len` on an array or a string, and `MultipleOf` on an integer. A value outside them is an `invalid_value` problem whose `ctx` uses pydantic's names (`min_length`, `multiple_of`), and the JSON Schema writes the matching keyword, such as `minItems`, `minLength` or `multipleOf`. A constraint on a type it does not apply to raises `TypeError`. Constant rules now live on the types, so the JSON Schemas say them too: non-empty `codecs`, sharding `codecs` and `index_codecs`, rectilinear edge lists, `struct` fields and v3 `struct` field names; `length_bytes` of at least 1 for `null_terminated_bytes` and zarr-python `raw_bytes`, and a positive multiple of 4 for `fixed_length_utf32`. Raw `r<N>` declares its bits a positive multiple of 8 on its type, and a refused name's message now names them: `bits in "r12": ...`. `check` now also refuses `"codecs": []`.
6 changes: 4 additions & 2 deletions packages/zarr-metadata/docs/extensions.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,8 +82,10 @@ names `acme.lz4` is then checked against this definition.
- a `NewType` or a type alias.

A number may carry bounds from annotated-types: `Gt`, `Ge`, `Lt`, `Le` or
`Interval`. Any other `Annotated` metadata is refused when the definition
is built.
`Interval`, and an integer a step, `MultipleOf`. An array or a string may
bound its length with `MinLen`, `MaxLen` or `Len`, so
`Annotated[tuple[int, ...], MinLen(1)]` is never empty. Any other
`Annotated` metadata is refused when the definition is built.

**The rules function**

Expand Down
8 changes: 5 additions & 3 deletions packages/zarr-metadata/src/zarr_metadata/_json.py
Original file line number Diff line number Diff line change
Expand Up @@ -127,9 +127,11 @@ class ValidationProblem:
document, `input` shows the change. `ctx` holds what was expected,
when that is more than a type:

- `gt`, `ge`, `lt` and `le`: the bounds from the value's type or from
a rule, with the names pydantic uses. For example `{"ge": 0, "le": 9}`
for a gzip `level`, whose type is `Annotated[int, Interval(ge=0, le=9)]`.
- `gt`, `ge`, `lt`, `le`, `multiple_of`, `min_length` and
`max_length`: the constraints from the value's type or from a rule,
with the names pydantic uses. For example `{"ge": 0, "le": 9}` for a
gzip `level`, whose type is `Annotated[int, Interval(ge=0, le=9)]`,
or `{"min_length": 1}` for an array document's `codecs`.
- `expected`: the values of a closed set, like zod's `values`, in the
order the message lists them. For example the values of a `Literal`,
of `node_type`, or of `zarr_format`.
Expand Down
7 changes: 4 additions & 3 deletions packages/zarr-metadata/src/zarr_metadata/_pydantic_schema.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
"""Private input types used only to generate accurate Pydantic JSON schemas.

These repeat the document TypedDicts in a stricter form: extension names
match the spec's pattern, `must_understand` is `true` at an extension
point, and `codecs` is not empty. They exist because pydantic builds a
match the spec's pattern, and `must_understand` is `true` at an extension
point. They exist because pydantic builds a
schema from Python types and cannot embed a JSON Schema that has its own
`$defs`, such as the ones `zarr_metadata.json_schema` writes; the v3
document schemas are recursive, so they cannot be inlined either.
Expand All @@ -16,6 +16,7 @@
from collections.abc import Mapping # resolved by Pydantic at runtime
from typing import Annotated, Literal, NotRequired

from annotated_types import MinLen
from pydantic import Field
from typing_extensions import TypedDict

Expand Down Expand Up @@ -57,7 +58,7 @@ class ZarrV3MandatoryNamedConfigJSON(TypedDict, closed=True):

ZarrV3DeclarationJSON = ExtensionName | ZarrV3NamedConfigJSON
ZarrV3MandatoryDeclarationJSON = ExtensionName | ZarrV3MandatoryNamedConfigJSON
ZarrV3CodecPipelineJSON = Annotated[tuple[ZarrV3MandatoryDeclarationJSON, ...], Field(min_length=1)]
ZarrV3CodecPipelineJSON = Annotated[tuple[ZarrV3MandatoryDeclarationJSON, ...], MinLen(1)]
ZarrV2FilterPipelineJSON = tuple[ZarrV2CodecMetadata, ...]


Expand Down
Loading
Loading