src: fix use-after-free in CleanupHookThunkRun - #65196
Conversation
CleanupHookThunkRun() read thunk->isolate/fun/arg from the CleanupHookThunk after invoking thunk->fun(). For every node::ObjectWrap alive at teardown, thunk->fun is ObjectWrap::CleanupHook, which deletes the wrap; ~ObjectWrap() calls RemoveEnvironmentCleanupHook() itself, erasing the CleanupHookThunk from the registry and freeing the node it lives in. The subsequent read of thunk->isolate/fun/arg to make the (now redundant) second RemoveEnvironmentCleanupHook() call was therefore a use-after-free. Cache the fields before running the hook so nothing is read from `thunk` once it may have been freed. Fixes: nodejs#65195
|
cc @addaleax (you wrote both #63642 and #63985, whose interaction causes this) and @legendecas (reviewed #63642) — would appreciate a look when you have a chance. |
|
cc @nsavoire — thanks for the thorough root-cause writeup and repro in the issue, it made this a straightforward fix to apply. If you get a chance to re-run your valgrind repro (test/addons/worker-addon-exit) against this branch, that would help confirm it before a maintainer looks at it. |
|
#65195 claims the issue is 100% deterministic, shouldn't we be adding a test? Also, there's a commit linter failure to fix here |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #65196 +/- ##
==========================================
- Coverage 90.32% 90.30% -0.03%
==========================================
Files 760 760
Lines 248525 248528 +3
Branches 46894 46895 +1
==========================================
- Hits 224488 224430 -58
- Misses 15469 15522 +53
- Partials 8568 8576 +8
🚀 New features to boost your workflow:
|
8월 15일부터 CI가 7연속 빨간불이었다. Node 22는 늘 통과했고 24만 죽었다. 8월 12일 마지막 성공 런의 러너는 24.18.0이었고 첫 실패부터 24.19.0이다. 그 사이 우리 커밋은 룰북과 스킬 문서뿐이라 네이티브 쪽을 건드리지 않았다. 24.19.0이 node::ObjectWrap에 정리 훅을 붙이면서(nodejs/node#63642) better-sqlite3 11.x의 Database 소멸자가 RemoveEnvironmentCleanupHook의 CHECK_NOT_NULL(env)에서 abort한다. 스택이 정확히 그 경로다. Node 쪽 수정(nodejs/node#65196)은 아직 미머지고 24.19.0 뒤로 24.x 릴리즈도 없어서 기다릴 수가 없다. better-sqlite3 13.x가 N-API로 옮겨가며 이 경로를 없앴지만 darwin-arm64에서 Node 20/22가 깨지는 회귀(#1514)가 열려 있어 검증 없이는 못 올린다. 우선 러너를 묶어 불을 끄고 업그레이드는 따로 본다. 커버리지 업로드 조건도 함께 고친다 — 24로 비교하던 자리라 핀을 넣으면 영영 안 걸린다. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Node 24.19.0 regressed process teardown with a use-after-free in CleanupHookThunkRun, which crashes better-sqlite3 11.10.0 AVA workers in RemoveEnvironmentCleanupHook (Assertion failed: (env) != nullptr). The floating 24.x matrix advanced 24.18.0 -> 24.19.0 mid-cycle and turned 'test (24.x, ubuntu-latest)' red across unrelated PRs. Pin the four 24.x matrix entries to 24.18.0 (last known-good) so the float stops. Lift once the Node fix ships. Refs: nodejs/node#65195 Refs: nodejs/node#65196
Node 24.19.0 regressed process teardown with a use-after-free in CleanupHookThunkRun, which crashes better-sqlite3 11.10.0 AVA workers in RemoveEnvironmentCleanupHook (Assertion failed: (env) != nullptr). The floating 24.x matrix advanced 24.18.0 -> 24.19.0 mid-cycle and turned 'test (24.x, ubuntu-latest)' red across unrelated PRs. Pin the four 24.x matrix entries to 24.18.0 (last known-good) so the float stops. Lift once the Node fix ships. Refs: nodejs/node#65195 Refs: nodejs/node#65196
|
I added a test and fixed the lint error - I'm happy to open new PR but didn't want to step on @sreehariannam's contribution: I'm seeing wide spread crashes in node v24.19.0 that should be fixed by #65042 which is blocked by this |
CleanupHookThunkRun() read thunk->isolate/fun/arg from the CleanupHookThunk after invoking thunk->fun(). For every node::ObjectWrap alive at teardown, thunk->fun is ObjectWrap::CleanupHook, which deletes the wrap; ~ObjectWrap() calls RemoveEnvironmentCleanupHook() itself, erasing the CleanupHookThunk from the registry and freeing the node it lives in. The subsequent read of thunk->isolate/fun/arg to make the (now redundant) second RemoveEnvironmentCleanupHook() call was therefore a use-after-free. Cache the fields before running the hook so nothing is read from `thunk` once it may have been freed. Taken over from nodejs#65196, which has been inactive; the original change is unmodified apart from the added comment. Fixes: nodejs#65195 Refs: nodejs#65196 Assisted-by: a closed-source coding agent Co-authored-by: Sreehari Annam <sreehari.annam@gmail.com> Signed-off-by: Caleb Everett <everett.caleb@gmail.com>
Add a cctest that registers an environment cleanup hook which removes itself while the cleanup queue is drained. It exercises CleanupHookThunkRun(), which must not read the CleanupHookThunk after invoking the hook, because the hook has already erased and freed it. The hook is registered directly rather than through node::ObjectWrap. ObjectWrap is what makes this reachable for addons since nodejs#63642, because its destructor removes its own hook, but the existing test/addons/worker-addon-exit does not report the use-after-free even under ASan, so exercising the self-removal directly is what catches it. The use-after-free is silent in ordinary builds and is caught by the ASan/Valgrind CI, which is how the original assertion (nodejs#63923) surfaced. Verified locally with an ASan build: the test reports heap-use-after-free in CleanupHookThunkRun() without the preceding commit and passes with it. Refs: nodejs#65195 Refs: nodejs#65196 Assisted-by: a closed-source coding agent Co-authored-by: Sreehari Annam <sreehari.annam@gmail.com> Co-authored-by: nsavoire <19255994+nsavoire@users.noreply.github.com> Signed-off-by: Caleb Everett <everett.caleb@gmail.com>
Add a cctest that registers an environment cleanup hook which removes itself while the cleanup queue is drained. It exercises CleanupHookThunkRun(), which must not read the CleanupHookThunk after invoking the hook, because the hook has already erased and freed it. The hook is registered directly rather than through node::ObjectWrap. ObjectWrap is what makes this reachable for addons since nodejs#63642, because its destructor removes its own hook, and nodejs#65195 reproduces the fault that way with test/addons/worker-addon-exit. That reproducer needs an addon build and depends on when the wrapper is collected, whereas this test drives the self-removal directly. The use-after-free is silent in ordinary builds and is caught by the ASan/Valgrind CI, which is how the original assertion (nodejs#63923) surfaced. Verified locally with an ASan build: without the preceding commit both this test and test/addons/worker-addon-exit report heap-use-after-free in CleanupHookThunkRun(); both are clean with it. Refs: nodejs#65195 Refs: nodejs#65196 Assisted-by: a closed-source coding agent Co-authored-by: Sreehari Annam <sreehari.annam@gmail.com> Co-authored-by: nsavoire <19255994+nsavoire@users.noreply.github.com> Signed-off-by: Caleb Everett <everett.caleb@gmail.com>
CleanupHookThunkRun() read thunk->isolate/fun/arg from the CleanupHookThunk after invoking thunk->fun(). For every node::ObjectWrap alive at teardown, thunk->fun is ObjectWrap::CleanupHook, which deletes the wrap; ~ObjectWrap() calls RemoveEnvironmentCleanupHook() itself, erasing the CleanupHookThunk from the registry and freeing the node it lives in. The subsequent read of thunk->isolate/fun/arg to make the (now redundant) second RemoveEnvironmentCleanupHook() call was therefore a use-after-free. Cache the fields before running the hook so nothing is read from `thunk` once it may have been freed. Taken over from nodejs#65196, which has been inactive; the original change is unmodified apart from the added comment. This also unblocks nodejs#65042, the backport of the cleanup hook registry to v24.x. Without that registry ~ObjectWrap() asserts during garbage collection, so every 24.x runtime aborts for ObjectWrap addons (nodejs#65446), as do 26.x runtimes before 26.4.0 when used with newer headers (nodejs#65262). Fixes: nodejs#65195 Refs: nodejs#65196 Refs: nodejs#65042 Refs: nodejs#65446 Refs: nodejs#65262 Assisted-by: a closed-source coding agent Co-authored-by: Sreehari Annam <sreehari.annam@gmail.com> Signed-off-by: Caleb Everett <everett.caleb@gmail.com>
Add a cctest that registers an environment cleanup hook which removes itself while the cleanup queue is drained. It exercises CleanupHookThunkRun(), which must not read the CleanupHookThunk after invoking the hook, because the hook has already erased and freed it. The hook is registered directly rather than through node::ObjectWrap. ObjectWrap is what makes this reachable for addons since nodejs#63642, because its destructor removes its own hook, and nodejs#65195 reproduces the fault that way with test/addons/worker-addon-exit. That reproducer needs an addon build and depends on when the wrapper is collected, whereas this test drives the self-removal directly. The use-after-free is silent in ordinary builds and is caught by the ASan/Valgrind CI, which is how the original assertion (nodejs#63923) surfaced. Verified locally with an ASan build: without the preceding commit both this test and test/addons/worker-addon-exit report heap-use-after-free in CleanupHookThunkRun(); both are clean with it. Refs: nodejs#65195 Refs: nodejs#65196 Assisted-by: a closed-source coding agent Co-authored-by: Sreehari Annam <sreehari.annam@gmail.com> Co-authored-by: nsavoire <19255994+nsavoire@users.noreply.github.com> Signed-off-by: Caleb Everett <everett.caleb@gmail.com>
Summary
CleanupHookThunkRun()readthunk->isolate/thunk->fun/thunk->argfrom the
CleanupHookThunkafter invokingthunk->fun(). For everynode::ObjectWrapstill alive at teardown,thunk->funisObjectWrap::CleanupHook, which deletes the wrap. Since #63642,~ObjectWrap()callsRemoveEnvironmentCleanupHook()itself, which erasesthe
CleanupHookThunkfromcleanup_hook_registryand frees the node itlives in. The subsequent read to make the (now redundant) second
RemoveEnvironmentCleanupHook()call was therefore a use-after-free — thisis now the ordinary teardown path for every
ObjectWrap-based addon, not anedge case.
The fix caches
isolate/fun/argbefore invoking the hook, so nothing isread from
thunkonce it may have already been freed.Root cause and fix as diagnosed in the issue.
Fixes: #65195
Test plan
test/addons/worker-addon-exit(built withnode-gyp, run undervalgrind) is the existing repro described in the issue.