Repository navigation
perf(data): node cache for component readers - #196
Merged
Merged
Conversation
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
janickm
added this pull request to stack #201
October 14, 2026 07:28
Collaborator
Author
|
/ok to test dcfb70a |
janickm
force-pushed
the
stack/zarr3/03-node-cache
branch
from
October 14, 2026 07:45
dcfb70a to
d435c3b
Compare
Collaborator
Author
|
/ok to test d435c3b |
janickm
force-pushed
the
stack/zarr3/03-node-cache
branch
from
October 14, 2026 10:02
d435c3b to
d3b4b36
Compare
Collaborator
Author
|
/ok to test d3b4b36 |
janickm
force-pushed
the
stack/zarr3/03-node-cache
branch
from
October 14, 2026 10:09
d3b4b36 to
ac24f4b
Compare
Collaborator
Author
|
/ok to test ac24f4b |
janickm
force-pushed
the
stack/zarr3/03-node-cache
branch
from
October 14, 2026 11:58
ac24f4b to
064254f
Compare
Collaborator
Author
|
/ok to test 064254f |
janickm
force-pushed
the
stack/zarr3/03-node-cache
branch
from
October 14, 2026 14:16
064254f to
f5c3f7a
Compare
Collaborator
Author
|
/ok to test f5c3f7a |
janickm
force-pushed
the
stack/zarr3/03-node-cache
branch
from
October 14, 2026 14:43
f5c3f7a to
4a20fac
Compare
Collaborator
Author
|
/ok to test 4a20fac |
janickm
force-pushed
the
stack/zarr3/03-node-cache
branch
from
October 14, 2026 15:32
4a20fac to
375b6de
Compare
Collaborator
Author
|
/ok to test 375b6de |
janickm
force-pushed
the
stack/zarr3/03-node-cache
branch
from
October 15, 2026 07:55
375b6de to
a47b3d7
Compare
Collaborator
Author
|
/ok to test a47b3d7 |
janickm
force-pushed
the
stack/zarr3/03-node-cache
branch
from
October 15, 2026 10:20
a47b3d7 to
e7d77cc
Compare
Collaborator
Author
|
/ok to test e7d77cc |
Collaborator
Author
|
/ok to test bdb9a8e |
janickm
force-pushed
the
stack/zarr3/03-node-cache
branch
from
October 16, 2026 13:37
bdb9a8e to
0e68e7e
Compare
Collaborator
Author
|
/ok to test 0e68e7e |
janickm
force-pushed
the
stack/zarr3/03-node-cache
branch
from
October 19, 2026 08:56
0e68e7e to
fe94b83
Compare
Collaborator
Author
|
/ok to test fe94b83 |
janickm
force-pushed
the
stack/zarr3/03-node-cache
branch
from
October 19, 2026 11:52
fe94b83 to
f82f7ee
Compare
Collaborator
Author
|
/ok to test f82f7ee |
janickm
force-pushed
the
stack/zarr3/03-node-cache
branch
from
October 19, 2026 11:58
f82f7ee to
79dd9ef
Compare
Collaborator
Author
|
/ok to test 79dd9ef |
janickm
force-pushed
the
stack/zarr3/03-node-cache
branch
from
October 21, 2026 08:37
79dd9ef to
0f552ff
Compare
Collaborator
Author
|
/ok to test 0f552ff |
janickm
force-pushed
the
stack/zarr3/03-node-cache
branch
from
October 21, 2026 08:53
0f552ff to
517461d
Compare
Collaborator
Author
|
/ok to test 517461d |
janickm
force-pushed
the
stack/zarr3/03-node-cache
branch
from
October 21, 2026 09:11
517461d to
a65383a
Compare
janickm
force-pushed
the
stack/zarr3/03-node-cache
branch
from
October 21, 2026 14:15
a65383a to
8a2cea3
Compare
Collaborator
Author
|
/ok to test 8a2cea3 |
Opening a group / array constructs its zarr-python object, which decodes the node's metadata on every access (~0.5 ms per frame lookup with zarr-python 2, dominating reads of small frame arrays). Component readers now cache the nodes they open (with their decoded metadata, but not their data) in a thread-safe nodes.NodeCache shared by all member groups of the component. The cache size is configurable via SequenceComponentGroupsReader(node_cache_size=...): None (default) caches all accessed nodes, N the N most recently used nodes, and 0 disables caching. With the cache, repeated frame lookups take ~2 us instead of ~470 us. Component stores are opened concurrently (hiding the latency of remote archive index reads), their hierarchies are now opened from their consolidated metadata sequentially on the calling thread, each as soon as its store is opened (while the remaining stores are still being opened): opening hierarchies is pure-Python CPU work, which concurrent threads only slow down.
janickm
force-pushed
the
stack/zarr3/03-node-cache
branch
from
October 21, 2026 18:01
8a2cea3 to
4044682
Compare
Collaborator
Author
|
/ok to test 4044682 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Opening a group / array constructs its zarr-python object, which decodes the node's metadata on every access (~0.5 ms per frame lookup with zarr-python 2, dominating reads of small frame arrays).
Component readers now cache the nodes they open (with their decoded metadata, but not their data) in a thread-safe nodes.NodeCache shared by all member groups of the component. The cache size is configurable via SequenceComponentGroupsReader(node_cache_size=...): None (default) caches all accessed nodes, N the N most recently used nodes, and 0 disables caching.
With the cache, repeated frame lookups take ~2 us instead of ~470 us.
Component stores are opened concurrently (hiding the latency of remote archive index reads), their hierarchies are now opened from their consolidated metadata sequentially on the calling thread, each as soon as its store is opened (while the remaining stores are still being opened): opening hierarchies is pure-Python CPU work, which concurrent threads only slow down.
Measurements
Full test sequence (35 stores, 447 MB; reading 320 camera / lidar frames: encoded images, lidar directions, distances and valid masks), local SSD, 64 cores, median of 5. "First pass" reads each frame once after opening (node cache cold), "later passes" read them again (node cache warm). Metadata reads count node metadata records read from the archives.
open_consolidatedand store-opening threads (max_threads)zarr-python 2 (2.18.7), this PR (
7091fb18):open_consolidatedmax_threadszarr-python 3 (3.4.1), with this PR's node cache (measured on #197
9aee757d, which adds zarr-python 3 support):open_consolidatedmax_threadsFor comparison,
main(1de0192e, zarr-python 2, no node cache): open 65 ms / 42 ms (consolidated / unconsolidated), 819 / 869 us/frame on the first pass, 791 / 864 us/frame on later passes.main: ~790 to 860 us/frame, re-opening each node on every access).open_consolidated=Falseopens ~8x faster and is faster overall (open + first pass 211 vs 488 ms). zarr-python 2 constructs nodes on access, so the two modes are close.open_consolidated=True(the default) reads 0 node metadata records,Falsereads one per accessed node (70 at open, 779 / 1671 during the first pass, each a request on remote storage).Store-opening threads with remote latency
50 ms simulated latency per store opening (the remote index read), open only:
open_consolidatedmax_threads62f9767d/ #197f15e3ff7)7091fb18/ #1979aee757d)Concurrent store opening hides the latency (35 x 50 ms = 1.75 s sequentially). Opening each hierarchy as soon as its store is opened overlaps the CPU work of opening hierarchies with the remaining store openings, most visible with zarr-python 3's consolidated hierarchies and limited threads (639 -> 454 ms with 8 threads).