Repository navigation
🐛 fix(cache): write compact interpreter JSON - #237
Merged
Merged
Conversation
Indented cache writes amplify nested candidate responses, and the streaming encoder can reject input that the JSON parser accepted. Write compact JSON before replacing the cache entry so its size follows the response size. Fixes tox-dev#204
Compact writes leave old amplified files in place. Discard entries above the size a bounded query can produce, allowing for Unicode escaping, so discovery re-queries and replaces them.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The DiskCache writer indented interpreter responses. Nested metadata inflated cache entries and could abort a lookup with
RecursionError. The writer now produces compact JSON, and the reader discards oversized old entries. Fixes #204.Nested JSON expanded in the cache
The old
json.dump(..., indent=2)call added whitespace at every nesting level. On the base commit, an 800-level value serialized from 1,600 bytes to a 1,283,213-byte entry. The compact write keeps entry size proportional to the response while preserving the atomic file replacement.The streaming encoder rejected deeper input
On the base commit, the 16,000-level testcase in #204 raised
RecursionErrorduring the cache write. The compact encoder writes that response without the streaming encoder's recursion failure. In the same container reproduction, flat and deep 34,653-byte answers each produced a 34,824-byte cache entry; the deep lookup peaked at 30 MiB.Old amplified entries survived the writer change
The old cache reader loaded an existing entry regardless of its size, so an old 512 MB file would still cost memory on every lookup. The reader now discards entries above 16 MiB before parsing. The probe accepts at most 1 MiB of output; the limit leaves room for supplementary Unicode to expand to 12 MiB when JSON escapes it.
Behavior changes
New cache entries use compact, single-line JSON. Existing indented entries up to 16 MiB remain readable; the reader removes larger entries and discovery queries the candidate again. An interpreter response too deep for the caller's JSON parser still fails before a cache write.
Reported by Anthropic's OSS Scanner: ANT-2026-ZO7AGAS5.