Repository navigation
fix(tests): keep the stuck-launcher survivor clear of the job's CPU limit - #12260
Merged
Merged
Conversation
…imit The executor test for a launcher that outlives its kill failed twice in CI on 2026-10-23 with `stuck == 0` and `wall_s` just over the 1.0 s timeout. The fake launcher execs the harness, so the process the test keeps alive is the harness itself and carries the run's RLIMIT_CPU: one second for a one-second job (`run_cpu_seconds`). The harness start (matplotlib, a fresh font cache per test) costs about 0.33 CPU seconds locally and about 2.4 times that on the CI runner, so once it spent the full second the kernel's SIGKILL ended the "stuck" launcher inside the grace window. A real `sandbox do` runs on the host, outside that limit, so the executor's `stuck` count is right and only the test raced. The test now gives the run the host's full CPU cap and asserts that limit on the survivor with `resource.prlimit`, so the test fails every time without the fix rather than once in a while: 31 of 31 failures with the fix removed, idle and under 48 busy loops; 50 of 50 passes with it. Also takes ruff's PEP 758 `except` form in two lines from #12244 that main's Lint job flags since the 3.14 move (the same hunks as #12259). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0175hA62juZoajMcqKxWFGza
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0175hA62juZoajMcqKxWFGza
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
The reviewed changes have no unresolved blocking issues and the test stabilization is covered by the supplied validation.
No open issues 🎉
What changed in this PR
This pull request makes the stuck-launcher renderer test deterministic by preventing the fake survivor process from inheriting the job’s one-second CPU limit. It also updates the adaptability checker syntax for Python 3.14 and documents the fix.
Changes:
- Applies and verifies the configured CPU limit for the test survivor.
- Updates both adaptability checker copies to use PEP 758 exception syntax.
- Adds a changelog fragment for the test reliability fix.
| File | Description |
|---|---|
tests/unit/agents/renderer/test_executor.py |
Makes the stuck-launcher test deterministic and verifies the survivor’s CPU limit. |
core/adaptability.py |
Applies the Python 3.14-compatible exception syntax. |
changelog.d/executor-stuck-test.md |
Documents the renderer test flake fix. |
automation/scripts/adaptability_check.py |
Keeps the generated checker mirror synchronized with the source checker. |
Review effort: Lite
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
MarkusNeusinger
added a commit
that referenced
this pull request
Oct 23, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0175hA62juZoajMcqKxWFGza
4 tasks done
MarkusNeusinger
added a commit
that referenced
this pull request
Oct 23, 2026
…#12259) ## Summary - **Playwright 1.55.1 → 1.64.1** in the root `package.json` and `package-lock.json`. GitHub's `ubuntu-latest` now resolves to Ubuntu 26.04 (image 20261012.163.1). Playwright 1.55.1 has no `ubuntu26.04-x64` host platform, so `npx playwright install --with-deps chromium` in `.github/actions/setup-node` failed (run 39938127853). Every `impl-generate`/`impl-repair` run for chartjs, d3, echarts, highcharts and muix died in setup. 1.64.1 ships the 26.04 download paths and apt dependency list. The lock diff is playwright + playwright-core plus the darwin-only `fsevents` optional entry that 1.64 no longer declares. esbuild stays at 0.28.1. Integrity hashes match the registry. This also aligns with the `playwright-core@1.64.1` pin that `scripts/site-smoke.mjs` and `ci-image.yml` already use. - **Ruff format fix no longer part of this PR.** This branch first carried the two PEP 758 `except` lines that main's lint needed. #12260 landed the identical lines, and after merging main they collapsed, so the diff is Playwright only. - Supersedes Dependabot #12251 for the Playwright half; its esbuild 0.28.2 half is left to Dependabot. ## Plan N/A ## Test plan - [x] **The action's own steps on Ubuntu 26.04, in a container.** `ubuntu:26.04` (26.04.1 LTS, resolute) with Node 22.23.3 (the action's default `node-version: "22"`). Every `run:` step of `.github/actions/setup-node/action.yml` ran verbatim against a minimal checkout: - **1.64.1:** `npm ci`, `npx playwright install --with-deps chromium` (apt deps + Chrome for Testing 156.0.8078.4) and both smoke renders all pass. The chartjs PNG and the MUI X React-path PNG are both 3200×1800. - **1.55.1 (negative control):** the install fails with exactly the CI error, `Error: ERROR: Playwright does not support chromium on ubuntu26.04-x64`. - [x] Render comparison: 1.55.1 (Chromium 140.0.7339.186) against 1.64.1 (Chromium 156.0.8078.4). Each harness was confirmed to launch its own build. The sample is 17 catalogue snippets × 2 themes = 34 pairs, rendered the way the pipeline calls the harness: - chartjs: 4 snippets. d3: 3. echarts: 4. highcharts: 3. muix: 3. Six of them are square-orientation. - **All 34 pairs are pixel-identical:** 0.000 % of pixels differ and the content bounding boxes match. Only the PNG encoding bytes differ. - [x] **MUI X sample excluded two snippets.** Two sampled muix snippets (`spiral-timeseries`, `biplot-pca`) import `@mui/x-charts/ChartContainer`. MUI X 9.16 (#12243) no longer exports that path, so both fail identically under BOTH Playwright versions. This is the known #12243 blast radius, and the muix bulk regeneration covers it. The muix sample therefore uses three snippets that still bundle. - [x] `ruff check`, `ruff format --check`, `mypy api core`, `pytest tests/unit tests/integration` (12715 passed) and `tools.changelog check` are green locally. ## Known gaps - **No PR check runs `.github/actions/setup-node`.** Only `impl-generate.yml` and `impl-repair.yml` use it, and neither runs on pull requests. A green PR therefore does not prove the install on the real runner. The 26.04 container run above is the closest local proof: same OS release, same Node major, same verbatim steps. The GitHub runner image adds preinstalled software on top of that. - **The real test is the first `impl-generate` run for a JavaScript library after the merge.** The muix bulk-regeneration pilot will be that run. - **Steps after setup are also untested on 26.04.** No `impl-generate` run has yet passed setup on the new image. Python, uv and the pngquant apt install already pass there (run 39938127853, steps 9–13). Claude, image processing and the GCS upload have not run on 26.04 yet. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
TestRender::test_a_launcher_that_outlives_its_kill_blocks_renders_until_it_exitsflake (two CI failures on 2026-10-23, runs 39932413037 and 39940278486). The fake launcher execs the harness, so the process the test keeps alive is the harness and carries the run'sRLIMIT_CPU, one second for a one-second job (wire.run_cpu_seconds). The harness start (matplotlib import, a fresh font cache per test) costs about 0.33 CPU s locally and roughly 2.4 times that on the CI runner (the plain render test takes 0.54 s here and 1.31 s there). Whenever it spent the full second, the kernel's SIGKILL ended the "stuck" launcher about 50 ms after the timeout, inside the 0.2 s grace window, sostuckread 0. The failing run'swall_s=1.054is that death: a survivor keeps_stopwaiting the full grace, about 1.22 s.sandbox doruns on the host, outside the limit the harness sets inside the sandbox, andstuckcounts only launchers still alive. The test now gives the run the host's full CPU cap and asserts that cap on the survivor withresource.prlimit, so it fails every time without the fix instead of once in a while.exceptform in two lines from feat(core): adaptability checker for Python implementations #12244 that main's Lint job flags since the 3.14 move. These are the same hunks as fix(js-render): bump Playwright to 1.64.1 for the Ubuntu 26.04 runner #12259, so whichever PR lands first, the other merges cleanly.Ruled out: the fake binary's
delete --forceonly logs and exits 0. The sleeper writes nothing, so it gets no SIGPIPE. Asyncio cancellation of the readers does not close the transport, andPYTHONASYNCIODEBUG=1with-X devreports no "close running child" or unclosed-transport warning. Coverage does not follow subprocesses (pytest-cov 7, nopatch = subprocess). Python 3.14 makes the harness start only about 5 % heavier than 3.13 (0.33 against 0.32 CPU s, same harness, same matplotlib), so it narrowed the margin but did not cause the flake.Plan
N/A
Test plan
assert (True and 0 == 1),wall_sabout 1.02, launcher exit -9): the old test body with code that spends more than one CPU second before it sleeps.uv run pytest tests/unit tests/integration -n 4 --dist worksteal: 12,715 passed, 3 skipped.tests/unit/agents/rendererunderpython -X devwithPYTHONASYNCIODEBUG=1: 179 passed, no asyncio warnings.ruff check .,ruff format --check .,mypy api core agentsandtools.changelog check --base origin/mainare clean.