Repository navigation
fix(reader): stray bytes that read as an element no longer pass for a cut or cost media - #34
Open
gravity-zero wants to merge 9 commits into
Open
gravity-zero wants to merge 9 commits into
gravity-zero wants to merge 9 commits into
Conversation
…lement, so a complete file is not reported cut short nor loses its first Clusters
…ement behind it, and the resyncing rewrite and the keyframe walk keep what sits behind head bytes that read as an element
…nts closing the Segment end the block walk instead of reading as a cut
…ed bytes are a field, and the strict refusals of bytes read as an element name the offset and the command that works
…re passed over, and analyze says what it read past
… it, a Cluster with a shrunk size is not stray, and padding counts for its excess
…bject's buffer grows with its data, and a nil subtitle index refuses to marshal
…quence ends where the next starts, and a picture shown again is PNG-encoded once
…was read past undecodable bytes, and exit 3 under --exit-damaged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stray bytes that happen to decode as an element header made a complete file read as cut short, or lose media without a word. Found on a real file: 6 bytes of an old tag string left ahead of the first Cluster by an in-place edit from before 0.56.0 (fixed there by
990a7cb; reproduced here with a binary built just before it).What changes, one commit each:
d5ba131- the block walk resynchronizes past head bytes that read as an element, asReadandReadMetaalready did.analyze,ingest -analyzeandvalidateno longer say "the source stops before its declared end" on a complete file, nor lose the first Clusters with exit 0.5f5a5fb- a Void or CRC-32 declaring a few bytes too many no longer swallows the element behind it;reindex --resyncandsalvagekeep the Info and Tracks behind such bytes (the output had no track, exit 0);keyframes -walkkeeps its keyframes.cdd2211- up to 64 stray bytes between the last Cluster and the Cues or Tags closing the Segment end the walk as the end of the media. A longer run is still refused: it may hide a Cluster.0da6d98-diagnosewalks the head to the first Cluster (it answered healthy when the Cues and Tags sat ahead of the bytes); the strict refusals ofreindex,reindex-inplaceandretimename the offset andmkvgo reindex --resync.30ee97d- bytes that read as a Cluster too small to hold a block are passed over;analyzewarns when it read past undecodable bytes.Then an audit of the branch and of the bitmap-subtitle decoders, never audited (two read-only reviewers, every finding reproduced by a failing test before its fix):
2031d2e- padding declaring too much ahead of a Cluster that is not the first resumes on that Cluster (the last one was dropped with no error, a defect ofcdd2211); a Cluster whose size was damaged but that opens with a Timestamp is not taken for stray bytes; padding counts for its excess, not its whole length;matroska.WithWholeHead.1039fc9- a block's composites share a two-plane budget (381 MiB allocated for a 200-byte block before); a PGS object's buffer grows with its data (400 MiB for 1.4 KB before); a nilSubtitleIndexrefusesMarshalBinary.f0e7345- a PGS block hands back two planes at most (256 MiB for 115 bytes before); a VobSub control sequence with no end command stops where the next one starts;extract-subtitle -format pngencodes a picture shown again once.2a551ce-to-mp4,to-webm,to-hls,demux,splitandremove-trackwarn when the Matroska source was read past undecodable bytes, and exit 3 under--exit-damaged.Added:
Diagnosis.ResyncedBytes(resynced_bytes) andreader.WithWholeHead. gorelease asks for v0.71.0.Checks, on
2a551ce:make preflight RELEASE=v0.71.0exit 0 in a container (gorelease, linux/windows/darwin/wasm/arm builds, lint, tests, 386, poisoned buffers, wasm smoke).go test -raceonmkv/reader,mkv/ops,mkv/subtitleand the commands, exit 0; the fuzz targets ofmkv/readerand of the subtitle decoders, 20 s each, exit 0.reindex --resyncrecovers them.Not covered:
-raceon the whole module (CI runs it).Known and left as is:
reindexandretimereport the offset where the read breaks, not where the stray bytes start;