Support open-ended ranges in getInputStream, fail loudly on unreadable zip archives - #103
Merged
Conversation
…e zip HttpStore.getInputStream formatted "bytes=%d-%d" unconditionally, so the default Store.getInputStream(keys) — which delegates to (keys, 0, -1) — sent "bytes=0--1" and returned null. Emit "bytes=<start>-" when end is negative, mirroring get(keys, start). S3Store had the identical bug. ReadOnlyZipStore.ensureCache treated a null stream from the underlying store as an empty archive, turning a read failure into valid-but-empty data. Throw StoreException.readFailed instead; a genuinely empty archive still parses to an empty index through the normal path. Fixes #100 Fixes #101 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #100 and #101.
HttpStore.getInputStreamand open-ended ranges (#100)getInputStream(keys, start, end)formattedbytes=%d-%dunconditionally. SinceStore.getInputStream(keys)delegates togetInputStream(keys, 0, -1), that becameRange: bytes=0--1— which no server answers, so the method returnednull. It now emitsbytes=<start>-whenendis negative, mirroring whatget(keys, start)already did.S3Store.getInputStreambuilt the same range string and had the identical bug, so it is fixed the same way (and now validates a non-negativestart, matching the other stores).ReadOnlyZipStoresilently reporting an unreadable archive as empty (#101)ensureCache()treated anullstream from the underlying store as a successfully-parsed empty archive, so a read failure surfaced as valid-but-empty data — concretely, a misleadingNo Zarr group found at …several layers from the cause. It now throwsStoreException.readFailed(...), as it already did for anIOExceptionwhile parsing the ZIP directory. A genuinely empty archive still parses to an empty index through the normal path, so the two cases stay distinguishable.Tests
StoreTest.testInputStreamOpenEnded— shared test asserting the defaultgetInputStream()returns the full object; runs against every store subclass.HttpStoreTest.testOpenEndedRangeHeader/testBoundedRangeHeader— MockWebServer tests pinning the exactRangeheader (bytes=0-andbytes=1-3).ReadOnlyZipStoreTest.testUnreadableArchiveThrows— assertsStoreExceptionfor a missing archive.Verified against the reproducer from #100 (
https://static.webknossos.org/misc/6001240.ozx):🤖 Generated with Claude Code