You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
A test run that hits the 120 s limit scores worse than --skip-tests, leaves vitest running, and may score stale coverage #106
On a pnpm monorepo, npx vitest run --reporter=json --coverage for one workspace project hit the fixed limit (tool run exitCode: null, durationMs: 120078). The testing check reported an error "app vitest command failed" although no test failed. Score and grade depend on machine speed, not code.
src/runners/exec.ts:131-146 — a timeout (e.code === "ETIMEDOUT", e.signal === "SIGTERM") is recorded as a generic status: "failed"; nothing marks it as a timeout.
testing.ts:505 → command-failed → :906-911 error issue → :946-951 execution 0/20. --skip-tests gets 10/20 (:953): timing out is penalised more than not running.
execSync timeout kills only the /bin/sh child; grandchildren (npx → vitest → workers) keep running. Verified locally with execSync('sh -c "sleep …" | cat', { timeout: 1000 }): both processes survive the throw.
testing.ts:587 reads coverage/coverage-summary.json from disk regardless of run outcome — after a timeout this can be a previous run's file (inferred from code; not reproduced).
What to do (cheapest first)
exec.ts: on ETIMEDOUT record status: "timeout", timedOut: true, timeoutMs on the ToolRun. Every runner benefits.
testing.ts: per-project run status timeout (label "timed out after N s"); a warning issue test-run-timeout, not an error; execution scored as for --skip-tests (10/20); do not read coverage for a root whose run timed out. The check-level status stays the normal derivation — no new check status.
Kill the whole process tree on timeout (spawn detached + process.kill(-pid), or spawnSync with an explicit tree kill).
Alternatives rejected
Exclude testing from the composite on timeout. Discards 60 static points (pyramid/pairing/quality, :933-971) and weight 13, and makes slow tests a way to raise the score.
New check-level status timeout. Consumers recognise only skipped/unavailable (github-app metric-history.ts:252, app repoMetricHistory.ts:398) and would plot it as a real score.
Acceptance criteria
Fixture with a test that sleeps past a 2 s configured timeout: ToolRun timedOut: true, project status timeout, a warning (not error), execution 10/20, no coverage read for that root.
Same fixture with a larger timeout passes.
No process from the timed-out run is alive after scan() returns.
Docs PR on vibecodeqa/vibecodeqa: --test-timeout in docs/docs/reference.md, settings.timeoutMs in docs/docs/configuration.md.
Delivered in #110 (branch fix/106-test-timeout, commits 62701d7 and 6fbb074). It covers all four steps:
timeout recorded as its own status in exec.ts
a test-run-timeout warning, with execution scored the same as --skip-tests and no coverage read for the timed-out project
--test-timeout and checks.testing.settings.timeoutMs
the whole process tree stopped through src/runners/exec-tree.ts
Verified: pnpm build is clean; pnpm lint has 0 errors (62 warnings, same as main); pnpm test passes 746/746. The 2 s timeout fixture in src/runners/test-timeout.test.ts covers every acceptance criterion, including that no pid is alive after scan() returns.
Docs: vibecodeqa/vibecodeqa#58. Merge it after this ships in a release.
Activity
serge-ivo commentedon Oct 5, 2026
Delivered in #110 (branch
fix/106-test-timeout, commits 62701d7 and 6fbb074). It covers all four steps:exec.tstest-run-timeoutwarning, with execution scored the same as--skip-testsand no coverage read for the timed-out project--test-timeoutandchecks.testing.settings.timeoutMssrc/runners/exec-tree.tsVerified:
pnpm buildis clean;pnpm linthas 0 errors (62 warnings, same as main);pnpm testpasses 746/746. The 2 s timeout fixture insrc/runners/test-timeout.test.tscovers every acceptance criterion, including that no pid is alive afterscan()returns.Docs: vibecodeqa/vibecodeqa#58. Merge it after this ships in a release.
refactor(exec): build every recorded tool run in one helper (#106)
fix(exec): tear the test process group down however the run ends (#106)
fix(testing): reject a test timeout no timer can hold (#106)
fix(testing): give timeout credit only when every project timed out (#…
fix(monitor): let the scan outlast the configured test timeout (#106)
docs: list --test-timeout in the README and CLAUDE.md flags (#106)
docs: match --test-timeout wording to the final cli#110 behaviour (vi…
fix(exec): trap SIGQUIT; record signal kills as 128 + signal (#106)
test(testing): give must-finish runs headroom under load (#106)
docs(reference): say process group, not process tree (vibecodeqa/cli#106
refactor(exec): record tool runs through the shared recorder (#106)
Merge pull request #110 from vibecodeqa/fix/106-test-timeout