Fix/mem alloc mismatch - #11397
Conversation
wc_MemFailCount_AllocMem() increments mem_fail_allocs before the registered allocator runs. When a caller-installed failing allocator (via wolfSSL_SetAllocators(), used by several OOM unit tests) or a genuine out-of-memory returns NULL, the allocation is counted but no block exists to free. The mem-fail nightly then reports a spurious "Free/Alloc mismatch" (Total one higher than Frees), even in the baseline counting phase, which is why it repeats identically across shards. Undo the count when the allocator returns NULL so Total stays balanced with Frees. The injected-failure path returns before the allocator is called and so is unaffected. All changes are under WOLFSSL_MEM_FAIL_COUNT and do not affect production builds. Signed-off-by: Sameeh Jubran <sameeh@wolfssl.com>
…_bounds The first test vector nulls ssl->buffers.certChain and ssl->buffers.certificate to drive the "chain == NULL" path of ProcessChainOCSPRequest(). The SSL owns the certificate DER copy that wolfSSL_new() allocated (weOwnCert), so clearing the pointer outright leaked 1536 bytes and the mem-fail nightly reported a Free/Alloc mismatch. Save the owned buffers, clear them for the call under test, then restore them so wolfSSL_free() releases them. Signed-off-by: Sameeh Jubran <sameeh@wolfssl.com>
wolfSSL-Fenrir-bot
left a comment
wolfSSL-Fenrir-bot
left a comment
There was a problem hiding this comment.
Fenrir Automated Review — PR #11397
Scan targets checked: wolfcrypt-bugs, wolfcrypt-src, wolfssl-bugs, wolfssl-src
Findings: 1
Required changes (1)
WOLFSSL_TRAP_MALLOC_SZ early return still leaves a counted allocation
File: wolfcrypt/src/memory.c:374
Function: wolfSSL_Malloc
Category: Incorrect error handling
The WOLFSSL_TRAP_MALLOC_SZ oversize check returns NULL directly, bypassing both the new res == NULL decrement at line 430 and the WOLFSSL_FORCE_MALLOC_FAIL_TEST one, so mem_fail_allocs keeps a count for a block that never exists. The Total/Frees mismatch this PR fixes persists in WOLFSSL_TRAP_MALLOC_SZ + WOLFSSL_MEM_FAIL_COUNT builds.
Related known finding #9957 (similar but distinct): Both affect wolfSSL_Malloc error paths under optional memory-test configurations, but this candidate's trap-size early return omits the allocation-failure count decrement, while #9957 frees an adjusted interior pointer in the forced-failure path with memory-zero checking. The operations, root causes, and required patches differ.
Recommendation: Call wc_MemFailCount_AllocFailed() under #ifdef WOLFSSL_MEM_FAIL_COUNT before this return NULL, or route the trap check through the shared NULL-return path.
Referenced code: wolfcrypt/src/memory.c:374-377 (4 lines)
This review was generated automatically by Fenrir. Reported findings require changes before merge.
3 participants
Description
Fixes spurious
Free/Alloc mismatchfailures reported by the mem-fail nightly (WOLFSSL_MEM_FAIL_COUNT). Two independent fixes:memory.c— don't count a failed allocation.wolfSSL_Malloc()callswc_MemFailCount_AllocMem()(which incrementsmem_fail_allocs) before invoking the registered allocator. When a caller-installed failing allocator (wolfSSL_SetAllocators(), used by several OOM unit tests) or a genuine OOM returnsNULL, the allocation was counted but no block exists to free, soTotalends up one higher thanFrees. Because this happens in the baseline counting phase, it reproduces identically in every shard. The fix undoes the count when the allocator returnsNULL. All changes are under#ifdef WOLFSSL_MEM_FAIL_COUNTand do not affect production builds.test_ProcessChainOCSPRequest_bounds— restore SSL-owned cert buffers. The first vector nulledssl->buffers.certChain/certificateto drive the "chain == NULL" path, but the SSL owns the certificate DER copy (weOwnCert), so nulling it outright leaked 1536 bytes (a real leak, confirmed withleaks). The buffers are now saved, cleared for the call under test, and restored sowolfSSL_free()releases them.Affected Class A tests now balanced:
test_TLSX_SecureRenegotiation_parse,test_TLSX_CSR_parse,test_TLSX_SupportedGroups_parse,test_TLSX_KeyShare_gen,test_tls_msgtype_tca_new_alloc,test_TLSX_CSR_Parse_bounds,test_ProcessChainOCSPRequest_bounds.Fixes zd# (N/A — tracked via the mem-fail nightly / jenkins-supervisor; add the ticket number if there is one)
Testing
Built with a near-nightly config and the mem-fail flag:
test_TLSX_SecureRenegotiation_parse→Total 184 / Frees 183).Total == Frees; unrelated tests unaffected (8/8,282/282,19/19).MEM_FAIL_CNT=1..MAX) on a fixed test: every iteration balanced, no new crashes.leaks --atExit) ontest_ProcessChainOCSPRequest_bounds:1 leak (1536 bytes)before →0 leaksafter.Checklist