Repository navigation
Conversation
subprocess_tree_rss_measures_contained_descendants compared two independently spawned process trees, one idle and one whose descendant touched 128 MiB, and required the loaded total to be at least 96 MiB above the idle total. Two separate trees need not start from the same footprint. On the hosted Windows CLANG64 leg they did not: the loaded tree measured only 78 MiB above the idle one (167 MiB vs 245 MiB), and the test failed on two unrelated PRs within two days (#2965 on 889170c, #2975 run 43499140720). The diagnosis that the two separate trees are the cause comes from #2965. The test now samples ONE tree twice, and every wait in the handshake is a blocking pipe read instead of a poll of a ready file: - The test creates two pipes, "up" (tree to test) and "down" (test to tree): FIFOs in its private temporary directory on POSIX, named pipes named after that directory's unique leaf on Windows. The __cbm_rss_tree_root role connects both as its first action and starts one __cbm_rss_tree_hold descendant that inherits the connected ends (fork on POSIX, an explicit two-handle inherit list on Windows); the root then drops its own copies. - The descendant writes 'I'; the test takes the idle sample and writes 'L'; the descendant touches every page of a 192 MiB load and writes 'R'; the test takes the loaded sample and closes its ends, which releases the descendant to free the load and exit. - A descendant that dies closes its ends, so the waiting read returns end-of-file instead of blocking. No handshake step has a sleep, a poll interval or a deadline. The floor stays at 96 MiB and is now half the load: the root does not grow between the two samples, so a probe that counts only the root misses the floor by 96 MiB, and a probe that counts the descendant clears it with 96 MiB to spare. The two member-fault tests use the same handshake and keep their assertions. Production code is unchanged. Evidence: - macOS arm64, ASan: subprocess suite 60 passed, 2 skipped (the Windows-only Job Object probes); the rewritten test passed 20 of 20 consecutive runs, idle 43 MiB and loaded 259 MiB in every run. - Binding: with the macOS probe temporarily limited to the root process, the test fails at the floor assertion (idle 33 MiB, loaded 33 MiB); restoring the probe returns the suite to green. - Linux arm64 container, gcc ASan: subprocess suite 60 passed, 2 skipped; the rewritten test passed 20 of 20 consecutive runs, idle 462 MiB and loaded 654 MiB in every run. - Windows: both changed files compile with -fsyntax-only -Wall -Wextra -Werror under llvm-mingw for x86_64 and aarch64. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
4 tasks done
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Makes
subprocess_tree_rss_measures_contained_descendants(tests/test_subprocess.c) deterministic. The test checks that the process-tree memory probe (cbm_subprocess_tree_rss_bytes) counts the contained descendants of a supervised process, not only its root.Why it was flaky
The test sampled two independently spawned process trees, one idle and one whose descendant touched 128 MiB, and required the loaded total to be at least 96 MiB above the idle total. Two separate trees need not start from the same footprint. On the hosted Windows CLANG64 leg they did not, and the test failed twice in two days on PRs that do not touch this code:
The diagnosis that the two separate trees are the cause comes from #2965 (commit ce65401 by Bowen-Xue-ac2). Thank you for tracking it down.
What changed
One tree, sampled twice, with the test and the descendant moving in lock step over two pipes:
FILE_FLAG_FIRST_PIPE_INSTANCE,PIPE_REJECT_REMOTE_CLIENTS).__cbm_rss_tree_rootrole connects both ends as its first action, then starts ONE__cbm_rss_tree_holddescendant that inherits the connected ends (fork on POSIX; on Windows an explicit two-handlePROC_THREAD_ATTRIBUTE_HANDLE_LIST, so nothing else is inherited). The root then drops its own copies, so the descendant alone holds the tree's side.I; the test takes the idle sample and writesL; the descendant touches every page of a 192 MiB allocation (one volatile store per 4 KiB) and writesR; the test takes the loaded sample and closes its ends, which releases the descendant to free the load and exit.rss_tree_wait_ready, 10 ms interval) is removed; no handshake step has a sleep, a poll interval or a deadline.The floor stays at 96 MiB and is now half the load. The root does not grow between the two samples, so a probe that counts only the root misses the floor by the full 96 MiB, while a probe that counts the descendant clears it with 96 MiB to spare (page rounding, allocator bookkeeping, a working-set trim between the two samples).
The two member-fault tests (
subprocess_tree_rss_fails_when_a_live_member_cannot_be_measured,subprocess_tree_rss_skips_a_member_that_exited_mid_probe) use the same handshake and keep their assertions; the injected fault still applies to a sample taken while the descendant is alive.Not changed: production code (only
tests/test_main.candtests/test_subprocess.cchange), andrss_tree_finish, which still drives the tree to its terminal state throughcbm_subprocess_poll, the subprocess API's only way to advance supervision.Evidence
build/c/test-runner subprocesspasses (60 passed, 2 skipped: the two Windows-only Job Object probes). The rewritten test, 20 consecutive runs: 20/20 pass, idle 43 MiB and loaded 259 MiB in every run.scripts/test.sh --suites subprocessin the test-infrastructuretestimage): 60 passed, 2 skipped. The rewritten test, 20 consecutive runs: 20/20 pass, idle 462 MiB and loaded 654 MiB in every run.-fsyntax-onlyand the test build's warning flags (-Wall -Wextra -Werror) under llvm-mingw for x86_64 and aarch64 (the image pinned intest-infrastructure/Dockerfile.mingw). The handshake itself first runs on Windows in this PR's CI; the Windows leg should show the test passing with itstree rss: idle=... loaded=...line.Checklist
git commit -s)subprocesssuite on macOS and on Linux arm64 (see Evidence); the full suite is left to PR CIsrc/, which is unchanged. On the changed test lines, clang-format (Homebrew LLVM) is clean and cppcheck with thelint-cppcheckflags reports nothing. No NOLINT added.