Skip to content

test: bound synchronous spawns in tests and name the command on timeout (HACK-2247) - #857

Merged
roodboi merged 4 commits into
nextfrom
test/hack-2247-bounded-sync-spawns
Nov 15, 2026
Merged

roodboi merged 4 commits into
nextfrom
test/hack-2247-bounded-sync-spawns

Conversation

@roodboi

@roodboi roodboi commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

What

HACK-2247 item 1. A synchronous spawn blocks the event loop, so Bun's per-test timeout cannot fire while it waits. On macOS a test child hung before it started (the system policy daemon crashed while it was being started) and tests/python-free-bootstrap.test.ts waited for five hours.

  • tests/helpers/bounded-spawn-sync.ts: spawnSyncBounded(cmd, options) is Bun.spawnSync with a default 30 s bound. At the bound it kills the child with SIGKILL and throws SyncSpawnTimeoutError, whose message names the command (for example git commit -m … did not finish within 30000 ms and was killed). Every other result is returned unchanged.
  • tests/python-free-bootstrap.test.ts: every spawn goes through the helper. Most already had a 10 s timeout, but a timed-out Bun.spawnSync only showed up later as an unexplained exit-code mismatch, and the sh -n syntax check had no bound.
  • Shared helpers bounded: fixture-git.ts runFixtureGitSync, secret-fixture.ts runFixtureGit, and the ps probes in lifecycle-mux-fixture.ts (including the one in its generated stand-in script). checkout-write-guard.ts was already bounded.
  • tests/sync-spawn-bound.test.ts: a lint-style test. It parses tests/**/*.ts and scripts/check-*.ts with the TypeScript compiler API and fails on any new Bun.spawnSync / spawnSync / execFileSync / execSync call that has no object-literal argument with its own timeout property. Strings and comments neither hide a call nor give it a bound. The 65 existing sites in 42 files are listed per file as a ratchet: a file may not gain a site, and a file that loses one must lower its count.

There is no product change. Items 2 to 4 of the ticket need a Mac and stay with the lead.

Limits

  • Options built elsewhere and passed as a variable count as unbounded (use the helper or an explicit timeout). Spawns inside generated fixture scripts (code held in a string) are not seen. Async Bun.spawn + await exited is not covered.
  • SIGKILL ends a stopped or sleeping child, but a process stuck in an uninterruptible kernel wait can still hold spawnSync while it waits to reap. Whether the macOS hang (a child at its first instruction while the policy daemon is down) is killable is not verified here; that needs the Mac.

Validation (Linux)

  • tests/bounded-spawn-sync.test.ts: a sleep 30 child is killed at 200 ms and the error names sleep 30. A child that SIGSTOPs itself before doing anything (a stand-in for "never starts") is killed at the bound. Normal and failing results come back unchanged.
  • Negative check: appending one unbounded Bun.spawnSync(["/bin/sh", "-c", "printf (("]) to a test file makes sync-spawn-bound fail with tests/python-free-bootstrap.test.ts: 1 unbounded (0 known). Reverted. The scanner's own test covers the reviewer's cases: parentheses inside strings, timeout in a comment or string, a nested timeout, and a shorthand { timeout }.
  • Every test file that uses the changed helpers, plus the three above: 194 pass across 12 files.
  • bun run typecheck, tsc -p packages/cli (it includes tests/), bun run check, and biome check on the changed files.

Not verified: macOS.

Follow-up: remaining unbounded synchronous spawns (tests and check scripts)

42 files, 65 sites (the ratchet list in tests/sync-spawn-bound.test.ts)
  • tests/background-command-journal.test.ts (1)
  • tests/background-command-process.test.ts (1)
  • tests/ci-change-scope.test.ts (5)
  • tests/compose-startup-phases.test.ts (1)
  • tests/daemon-detach.test.ts (1)
  • tests/docker-context.test.ts (1)
  • tests/doctor-adopted-generated-files.test.ts (1)
  • tests/doctor-fix-adopted-project.test.ts (1)
  • tests/e2e-shard.test.ts (2)
  • tests/e2e/run-dependency-cache.test.ts (1)
  • tests/env-protection.test.ts (1)
  • tests/env-value-stdin-cli.test.ts (1)
  • tests/fixture-hermeticity.test.ts (1)
  • tests/gateway-tokens.test.ts (1)
  • tests/git-info-exclude.test.ts (1)
  • tests/lifecycle-command-child.test.ts (2)
  • tests/lifecycle-json.test.ts (1)
  • tests/lifecycle-private-output.test.ts (3)
  • tests/mcp-native-adapter.test.ts (3)
  • tests/mcp-startup-supervision.test.ts (2)
  • tests/native-compose-adoption-worktrees-fixture.test.ts (1)
  • tests/native-interactive-launcher.test.ts (1)
  • tests/native-preflight-cli.test.ts (1)
  • tests/native-project-branch-cli.test.ts (1)
  • tests/native-up-already-running-cli.test.ts (1)
  • tests/portable-bootstrap-env-refusal.test.ts (1)
  • tests/pre-commit-hook.test.ts (2)
  • tests/prerelease-plan.test.ts (1)
  • tests/project-down-worktree-safety.test.ts (1)
  • tests/project-env-layer-lock.test.ts (1)
  • tests/project-env-worktree-local.test.ts (1)
  • tests/project-target-inspect.test.ts (1)
  • tests/projects-registry-worktrees.test.ts (1)
  • tests/projects-registry.test.ts (1)
  • tests/qualify-provider-pin.test.ts (1)
  • tests/registry-touch-worktrees.test.ts (1)
  • tests/run-exec-branch-default.test.ts (1)
  • tests/session-list-stop-tmux.test.ts (2)
  • tests/test-shard.test.ts (4)
  • tests/tmux-backend-locale.test.ts (2)
  • tests/upgrade-apply.test.ts (1)
  • tests/upgrade-check.test.ts (7)

For routing: product paths that wait on a child with no bound

This is a read-only audit of src/ and index.ts at origin/next 8e7242d. Line numbers point at the spawn. Nothing here is changed by this PR.

file:line API runs note
src/ui/gum.ts:172 Bun.spawnSync gum log/style the only unbounded sync spawn in product code; on the logger path, so a log line can block
src/ui/gum.ts:185 Bun.spawn non-interactive gum (style/join/format/table/version) awaits exited
src/daemon/launchd.ts:127 Bun.spawn launchctl kill timer only inside withProcessDeadline; macOS
src/lib/shell.ts:65 (exec), :187 (run); :526 execOrThrow delegates to exec Bun.spawn shared helpers timeoutMs is optional and unbounded by default outside withProcessDeadline. Roughly 160 callers pass none. macOS examples: security at src/lib/nodes-registry.ts:492 and src/commands/global.ts:4468, brew --prefix at src/lib/native-hack-local-route-host.ts:384, ps at src/lib/project-lifecycle-controller.ts:1530
src/ui/clipboard.ts:49 Bun.spawn pbcopy / wl-copy / xclip awaits exited
src/daemon/process.ts:249 Bun.spawn ps -axo
src/backends/native-project-recovery.ts:141 Bun.spawn /bin/ps -axo
src/lib/secrets-hygiene.ts:675 Bun.spawn git rev-parse
src/lib/lifecycle-command-child.ts:62 Bun.spawn sh -c lifecycle hook only an optional abort signal
src/commands/gateway.ts:1403, :1414, :1430, :1447 Bun.spawn hack daemon start/stop, re-exec
src/commands/remote.ts:740, src/control-plane/extensions/enable.ts:165, src/control-plane/extensions/supervisor/commands.ts:1487 Bun.spawn hack re-exec
src/control-plane/extensions/tailscale/commands.ts:333, src/control-plane/extensions/cloudflare/commands.ts:1046 Bun.spawn tailscale / cloudflared probes
src/tui/hack-tui.ts:2219, :2516, :2738 Bun.spawn hack logs one-shot and actions
src/ui/docker-logs.ts:391, :452 Bun.spawn docker compose logs / tail (non-follow)

Intentionally unbounded (interactive or long-lived by design) and not listed: foreground run/tty paths, docker compose up in the foreground, log follow, the docker events watcher, supervisor jobs, fzf/gum prompts, and sudo password prompts. A kill timer only helps once Bun.spawn has returned; a hang inside the spawn call itself would need a different guard.

Release signal

test: test-only.

Refs HACK-2247 (item 1).

🤖 Generated with Claude Code

…ut (HACK-2247)

A synchronous spawn blocks the event loop, so Bun's per-test timeout cannot
fire while it waits. On macOS a test child hung before it started and
python-free-bootstrap waited for hours.

- tests/helpers/bounded-spawn-sync.ts: spawnSyncBounded, a Bun.spawnSync
  with a default 30 s bound that kills with SIGKILL and throws an error
  naming the command.
- python-free-bootstrap uses it for every spawn (the `sh -n` check had no
  bound).
- The shared fixture helpers (fixture-git runFixtureGitSync, secret-fixture
  runFixtureGit, lifecycle-mux-fixture ps probes) are bounded.
- tests/sync-spawn-bound.test.ts fails on any new synchronous spawn without
  a timeout in tests/ or scripts/check-*.ts; the 80 existing sites in 49
  files are a ratchet that may only go down.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Nov 15, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
hack-docs Ready Ready Preview Nov 15, 2026 4:47am UTC

Request Review

@linear-code

linear-code Bot commented Nov 15, 2026

Copy link
Copy Markdown

HACK-2247

@roodboi

roodboi commented Nov 15, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-15T04:49:28.717366Z 6dc8bf0 Manual request
🔒 Security Review ✅ Completed 2026-11-15T04:51:45.611477Z 6dc8bf0 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Chef's kiss.

Reviewed commit: 9421b16f1e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Review of #857: counting parentheses in raw text let a `(` or `)` inside a
string move a call's boundary, so a bounded call could be reported and an
unbounded one could borrow the next call's timeout. The check now parses
each file with the TypeScript compiler API and counts a call as bounded
only when one of its arguments is an object literal with its own `timeout`
property. Strings and comments that mention spawnSync or timeout no longer
count, so the baseline drops to 65 sites in 42 files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: 9421b16f1e

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@roodboi

roodboi commented Nov 15, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Delightful!

Reviewed commit: 78f14a4bcd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: 78f14a4bcd

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@roodboi

roodboi commented Nov 15, 2026

Copy link
Copy Markdown
Contributor Author

Ready for lead: head 78f14a4bcd3a703b651fa2c6bea57323e3948016 (base next 8e7242d).

Ran

  • tests/bounded-spawn-sync.test.ts: a sleep 30 child is killed at 200 ms and the error names it. A child that SIGSTOPs itself before doing anything (a stand-in for "never starts") is also killed at the bound. Normal and failing results come back unchanged.
  • tests/sync-spawn-bound.test.ts: passes on this head. Negative check: appending one unbounded Bun.spawnSync to a test file fails it with 1 unbounded (0 known) (reverted).
  • Every test file that uses the changed helpers, plus the new tests: 194 pass across 12 files at 9421b16. The only change since then is the scanner in sync-spawn-bound.test.ts, which was rerun.
  • tsc -p packages/cli (it includes tests/), bun run typecheck, bun run check, and biome check on the changed files.
  • Hosted CI run 43656635020: both required checks pass. All 8 test shards pass, including macOS shards 1/8 to 3/8 (macos-15), and so do Candidate core and Config compiler on macOS. docker-e2e 2/4 and 4/4 were still running when I posted; this PR changes only tests that do not run there.
  • Independent Codex reviewer (gpt-6.1-sol, same thread for the unit): one P2 at 9421b16. Parentheses inside strings moved the text scanner's call boundaries. Fixed in 78f14a4 by parsing with the TypeScript compiler API, with the reviewer's cases added as regression tests. Re-review: CLEAN at 78f14a4.
  • Codex bot: no major issues at 78f14a4; its automatic security review found none. computer pr review-status: CLEAR at 78f14a4.

Not verified

  • macOS behaviour of the original hang. Whether SIGKILL ends a child stuck at its first instruction while the system policy daemon is down is not shown here. Items 2 to 4 of HACK-2247 stay with you.

For you to route

  • The PR body lists the 65 remaining unbounded test and check-script sites (42 files) for follow-up.
  • The PR body also has a table of product paths that wait on a child with no bound, with every line number checked against 8e7242d. The worst for macOS are src/ui/gum.ts:172 (the only unbounded sync spawn in product code, on the logging path) and src/daemon/launchd.ts:127. The exec/run helpers in src/lib/shell.ts have no default bound, and about 160 callers pass none.

Test-only; no product code changed.

@roodboi

roodboi commented Nov 15, 2026

Copy link
Copy Markdown
Contributor Author

Lead, one fact for the "not verified" item: on the Mac where the original hang happened, the stuck child (thread still at its first instruction after 5 h 20 min) ended within two seconds of a plain SIGTERM, and its parent test then failed normally. So a bound that signals the child does release this kind of hang. This PR goes into the next Mac batch.

🤖 Generated with Claude Code

roodboi and others added 2 commits November 15, 2026 04:45
After merging next, tests/sync-spawn-bound.test.ts failed on two files
that landed after this branch's base: tmux-socket-guard.test.ts (12) and
nss-read-safety.test.ts (1). Bun.spawnSync sites use spawnSyncBounded; it
still calls Bun.spawnSync, so the tmux socket guard under test still
applies. The Node execFileSync/spawnSync/execSync calls, and the
Bun.spawnSync call that must throw from the guard, keep their API and get
an explicit timeout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@roodboi

roodboi commented Nov 15, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Can't wait for the next one!

Reviewed commit: 6dc8bf0872

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: 6dc8bf0872

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@roodboi

roodboi commented Nov 15, 2026

Copy link
Copy Markdown
Contributor Author

Ready for lead: head 6dc8bf0872adbd71ddf7fdb8e88945a00c3abcfd (normal merge of next e47681e in 4d46a09, then 6dc8bf0).

Answer: #797, #795, #852 and #858 add no unbounded synchronous spawn. I merged each PR head onto 6dc8bf0 in a throwaway checkout, each alone and then all four together, and ran tests/sync-spawn-bound.test.ts. Green every time, with no new and no lowered entries. Heads: #797 969bf07215 (newer than 5cdbd3a), #795 177cc9ffef, #852 6aba7f074f, #858 0c33970830. #795 edits two files that already contain counted spawns (tests/native-preflight-cli.test.ts, tests/upgrade-check.test.ts, both already on the baseline) without adding one. Nothing was pushed to their branches.

What changed since 78f14a4

  • tests/tmux-socket-guard.test.ts (12 sites) and tests/nss-read-safety.test.ts (1), both added by fix(test): isolate tmux servers and guard default sockets #845/fix(trust): guard observe-only NSS queries (HACK-2218) #847 after this branch's base, are now bounded.
  • The Bun.spawnSync sites use spawnSyncBounded. It still calls Bun.spawnSync, so the tmux socket guard under test still applies.
  • The Node execFileSync/spawnSync/execSync calls, and the object-form Bun.spawnSync that must throw tmux-socket-guard, keep their API and get timeout: 10_000.
  • The Bun.spawnSync calls inside generated-script strings are not counted by the lint and are unchanged.

Ran (Linux)

  • tests/sync-spawn-bound.test.ts: passes at this head, with no change to the baseline.
  • tmux-socket-guard, nss-read-safety, sync-spawn-bound, bounded-spawn-sync: 75 pass. tmux is installed, so the tmux tests ran rather than skipped.
  • tsc -p packages/cli (it includes tests/) and biome check on the changed files.
  • Hosted CI run 43678285012 at this head: required checks pass. Linux test shards 4/8 to 8/8 pass, as do macOS test shard 2/8, Candidate core (macOS and Ubuntu), Config compiler (macOS, Ubuntu, arm), docker-e2e 1/4 to 4/4 and linux-process-lifetime.
  • Session Codex reviewer (gpt-6.1-sol, same thread): CLEAN at 6dc8bf0, covering the merge and the 13 bounded spawns. It confirmed the timeout options survive the guard's forwarding wrappers and that the refusal test still requires tmux-socket-guard.
  • Codex bot: no major issues at 6dc8bf0, and its automatic security review found none. computer pr review-status: CLEAR at 6dc8bf0.

Not waited for

  • Hosted macOS test shard 1/8 (in progress) and test shard 3/8 (queued) when this was posted.
  • The macOS behaviour of the original hang, beyond your note that a plain SIGTERM released it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant