Repository navigation
Fix memory leaks in signature and key handling - #534
Merged
Merged
Conversation
mtrojnar
force-pushed
the
fix-memory-leaks
branch
7 times, most recently
from
November 15, 2026 17:18
45a016a to
d023046
Compare
Release signatures, timestamp credentials and APPX buffers on error, and free unused catalog members. Repeated hashes, private keys and command options must release the values they replace. Finish initialized engines when forced login fails. Handle failed ownership transfers to OpenSSL attributes and stacks, preserve MSI buffers after failed growth, and unmap the complete input when PE or script processing shortens its content length. Check digest BIO allocation before linking sink BIOs. Route built-in TSA loading failures through common cleanup. Preserve the failure result so unreadable or invalid TSA parameters no longer leave an apparently successful output without the requested timestamp.
Make success-only ownership transfers and always-consuming helper contracts explicit at call boundaries. Explain signature cleanup, context-owned APPX buffers and immutable mapping sizes so future changes do not reintroduce leaks or double frees. Comments only; no behavior changes.
mtrojnar
force-pushed
the
fix-memory-leaks
branch
from
November 15, 2026 20:44
d023046 to
705c9dc
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request Type
Related Issue
Issue number: N/A
Current Behavior
Error paths, repeated values and failed OpenSSL ownership transfers leave signatures, credentials and buffers allocated. PE and script processing can leave input mappings partially allocated. TSA parameter-loading failures incorrectly report success. Base64 BIO allocation failures can also leak a timestamp response BIO or crash script extraction and timestamp decoding, and signature content creation can leak the PKCS#7 structure or crash when an allocation fails.
New Behavior
Release owned objects on failure and before replacement, preserve MSI buffers when growth fails, and unmap the complete original input. Invalid or unreadable TSA parameters now fail the command and remove its output. Base64 BIO allocation failures abort decoding cleanly, and signature content creation reports allocation failures without leaking or crashing.
Scope of Changes
Cover signature processing, digest BIOs, credential/provider loading, engine references and repeated options. Catalog member, duplicate APPX hash and private key selection remain unchanged. The libcurl header fix applies only to legacy OpenSSL < 3.0 builds. A separate comments-only commit documents ownership transfers, cleanup responsibility and mapping-length invariants.
Testing
All 424 existing CTest tests passed on Linux with both OpenSSL 3.6 and 1.1.1. The modern build's 407 Valgrind process checks and 192-run fault-injection matrix (132 triggered failures) reported zero errors. Separate legacy libcurl checks covered both timestamp protocols and failure of each header append, with no definite or indirect leaks. Another 21 focused script/timestamp decoding checks passed under Valgrind, including filter and memory-BIO allocation failures. A further 25 fault-injection checks covered signature content creation across PE, MSI, CAB, APPX and script formats, with no leaks or crashes. Focused checks covered failed writes, malformed signatures, engine login, MSI reallocations and complete input unmapping. Repeated APPX hash extraction and allocation failures passed Valgrind checks for all five hash types. GitHub CI, including the CodeQL security gate and Linux/macOS/Windows builds, passed.
Additional Notes
Validation used temporary local reproductions; no test files are included.
License Declaration