Skip to content

fix(zarr-metadata): compare the declarations a configuration holds whatever order it writes them in - #4641

Merged
d-v-b merged 2 commits into
zarr-developers:mainfrom
d-v-b:fix/zarr-metadata-nested-declaration-order
Nov 17, 2026
Merged

d-v-b merged 2 commits into
zarr-developers:mainfrom
d-v-b:fix/zarr-metadata-nested-declaration-order

Conversation

@d-v-b

@d-v-b d-v-b commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

This PR ensures that the order of elements in a configuration dict for a nested declaration does not matter for the identity of the outer declaration.

🤖 AI text below 🤖

Two declarations that hold other declarations, such as two sharding_indexed codecs, are now equal when they differ only in the order their configurations write their members in. The declarations a configuration holds were compared in the order it writes them, so a shard that writes index_codecs before codecs was not equal to one that writes them the other way round, and neither were the array models that hold them, although the order of the members of a JSON object means nothing. The items of an array, such as the codecs of a pipeline, are still compared in order.

How it was found, and the change

The agent found this while testing canonical documents: a v3 array document and the same document with the members of every object reversed read as two unequal models when it held a shard. Attributes, extra fields, unclaimed configurations and a configuration's own members are compared as sorted JSON text, so their order never mattered. Only _nested_key in definition/_definition.py kept written order, because it iterated the nested mapping. It now sorts by location, with a sort key that never compares an object key with an array index.

refines already compared nested declarations by location, so it is unchanged. hash follows ==, since both use the key.

Tests and checks
  • tests/v3/test_definitions.py: three cases added to the table of test_two_declarations_are_equal_when_they_read_the_same. A shard with its members reversed is equal (this case fails without the fix), a shard with its index codecs in another order is not, and a codec with no nested declarations and its members reversed is equal, as before.
  • Run locally: the package suite, ruff 0.17.3 and pyright.
Stack bookkeeping

🤖 Generated with Claude Code

…atever order it writes them in

`declaration_key` paired each nested declaration's key with its location,
in the order the configuration wrote them. So two `sharding_indexed`
codecs that differed only in whether `codecs` or `index_codecs` came
first had different keys, and were not equal, although the order of the
members of a JSON object means nothing and the rest of a configuration
is already compared as sorted JSON text. The array models that held them
were not equal either.

The nested keys are now sorted by location. An index of an array sorts
by its number, so the items of an array, such as the codecs of a
pipeline, are still compared in order.

Assisted-by: ClaudeCode:claude-fable-5-2
@github-actions github-actions Bot added needs release notes Automatically applied to PRs which haven't added release notes zarr-metadata Specific to the zarr-metadata sub-package labels Nov 16, 2026
@d-v-b
d-v-b marked this pull request as ready for review November 17, 2026 16:01
@d-v-b
d-v-b merged commit b0020f2 into zarr-developers:main Nov 17, 2026
45 checks passed
d-v-b added a commit to d-v-b/zarr-python that referenced this pull request Nov 17, 2026
…canonical-documents

Main now carries the chunk-aware canonical hook (zarr-developers#4640) and the nested declaration order fix (zarr-developers#4641). The one conflict was in model/_array_v3.py, where this branch adds canonical_array_document_v3 next to codecs_for_chunks; both are kept.

Assisted-by: ClaudeCode:claude-fable-5-2
d-v-b added a commit to d-v-b/zarr-python that referenced this pull request Nov 17, 2026
…at are not the default

A review found that the Hypothesis tests passed with a canonicalize that returned the model's own document, that no test noticed a model losing its scope, and that four behaviours had no case: a claimed extra field, a v2 fill value, a chunk grid and a storage transformer that are respelled.

The parametrized cases are now read in scopes that are not the default ones. The generated documents are compared with the same document reordered at every depth and, for arrays, with each declaration in its own simplest spelling. The reordering helper reverses every object, which the fix in zarr-developers#4641 allows.

Assisted-by: ClaudeCode:claude-fable-5-2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs release notes Automatically applied to PRs which haven't added release notes zarr-metadata Specific to the zarr-metadata sub-package

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant