Repository navigation
Cap the size of logged request and response bodies. - #11
Merged
Merged
Conversation
Whole bodies were logged, so a long document produced a line longer than the 16 KB the awslogs driver accepts, which splits it into fragments that are no longer JSON, and every byte of it reached the log. A body longer than the limit (2048 bytes by default, set through withBodyLimit) is now logged as an excerpt within the limit, next to body_truncated_from_bytes. The excerpt stays an object or an array with its field names instead of a string of cut JSON, so the entry is valid JSON and redactions keyed by field name still apply to it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The middleware logged every request and response body whole. A consumer serving a long document (the full text of a legal instrument, about 18 KB) wrote entries longer than the 16 KB line the Docker
awslogsdriver accepts. The driver splits such a line into fragments that are no longer JSON, so structured parsing breaks, and every byte of the body reaches the log.Every logged body is now capped:
tiny-blocks/loggeruses).body_truncated_from_bytes, the size of the whole body.…, an object or array where it falls is read the same way one level down, and everything after is left out. Escaped characters (quotes, control characters) count at their encoded size, so the excerpt never exceeds the limit.withBodyLimit(bytes: 0)logs only the size of every body. A negative limit raises the newBodyLimitOutOfRangeat build time.LogMiddleware::build()gains?int $bodyLimit = null, placed after$loggerby the parameter-ordering rule. Named-argument callers are unaffected.Why an excerpt rather than a truncated string
The library has no redaction of its own. Consumers redact in the logger, after this middleware, by field name at any depth (
tiny-blocks/loggerEmailRedaction,NameRedaction,GenericRedactionand so on). Turning a long body into a cut JSON string would hide its fields from those redactions and put unmasked values into the first 2 KB of the log, which is worse than today. The excerpt stays a JSON object or array with its field names, so the entry stays valid JSON and those redactions keep applying. A test with a redacting logger covers it.Release
Recommended release:
1.5.0(minor). The public API only gains a builder method, an optional parameter and an exception. The visible change is that bodies longer than 2048 bytes are now logged as excerpts by default. That is the fix the release exists for, and a consumer who needs whole bodies can raise the limit.Related issue
None. The need came from a consumer whose entries for long documents were split by the log driver.
Checklist
make reviewpasses.make testspasses: 70 tests, 134 assertions, 115 mutants, all killed, MSI 100%.