Repository navigation
release: 0.7.1 — the runtime check reads verbose output; silent failures need another conversation's run - #29
Merged
Merged
Conversation
…res need another conversation's run Patch release (fixed existing behaviour → patch +1, VERSIONING.md). - Claude Code runtime check: the judge reads verbose output (a JSON array, judged on its last type "result" element) and requires num_turns >= 1 and a result that answers OK; RUNTIME_CHECK_SETTINGS turns verbose off (verbose: false, viewMode: "default"). - A run that exits non-zero with nothing on stdout past the 5 s window is RunOutcome.silent and reaches the kit as RunEnd.timedOut: reported only once another conversation's run completed. The window is measured to the process's exit. - The could-not-process log line counts hidden_count, not the capped id list. - Codex runtime check: --disable hooks; ok only on exit 0 with a turn.completed and an agent_message in its --json events (codexRuntimeCheckJudge); no forced model_reasoning_effort. - doctor says the runtime check failed only while the status file is fresh; otherwise it says the daemon is not running.
`spawnRun({ shell: true })` runs the CLI's `listen --run` command line.
A command that got going and exited 1 must be a failed run with exit 1;
one the shell cannot find (sh 127, cmd.exe 9009, nothing on stdout) is
not started. Expected to fail on Windows until spawnRun stops routing
shell runs through cross-spawn, whose Windows hook turns every exit 1
of a shell spawn into a `spawn … ENOENT` error.
Shell runs (the CLI's `listen --run` command line) went through cross-spawn. With `shell: true` cross-spawn resolves nothing, but its Windows hook still re-emits every exit 1 as `spawn <command> ENOENT`: a command that ran and failed with 1 became `not_started`, said to be missing, and was never a could-not-process failure. Shell runs now use Node's own spawn and keep the exit code; the existing rule tells a command cmd.exe cannot find (exit 1, nothing on stdout: not started) from one that ran and failed (exit 1 after output: failed). Direct runs keep cross-spawn for PATHEXT and .cmd shim resolution. The test's Windows exit code for a command not found is 1, as the previous commit's CI run on Windows showed (cmd.exe /c exits 1).
…stderr are not silent From the independent review of #29: - Claude check: `language: "english"` in RUNTIME_CHECK_SETTINGS; the judge no longer reads the answer's words (drops /\bok\b/i: "Okay" or an answer in another language is an answer) and requires subtype success, is_error false, num_turns >= 1 and, when usage is present, usage.output_tokens >= 1. RUNTIME_CHECK_ANSWER is gone (never released). - Codex check: `hooks` is no longer disabled. A hook that breaks every run must fail the check too; Codex runs print --json events, so a broken run is never silent and only the check can catch it. - spawnRun: a run whose redacted stderr tail says `API Error` is not marked silent (a refusal moved to stderr stays a plain failure). - doctor, stale status file: "the daemon has not written its status for <N> — it may be down", the words status uses (integrationStatusStaleFor replaces integrationStatusWritten). - Tests: "Okay" and a non-English answer pass, num_turns 0 and output_tokens 0 fail; the stderr guard; a shell run that exits non-zero with nothing on stdout past the window is failed + silent. - CHANGELOG and README follow.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes
Patch release 0.7.1 (fixed existing behaviour → patch +1,
VERSIONING.md). Fixes from an independent review of 0.7.0 and a second one of this PR, all to the could-not-process path, plus the Windows shell-run fix from #28:src/presets/claude-code.ts,claudeRuntimeCheckJudge,RUNTIME_CHECK_SETTINGS). With verbose on (--verbose, the user'sverbosesetting or its~/.claude.jsoncopy,viewMode: "verbose"),claude -p --output-format jsonprints a JSON array of every message, and the judge refused every array: the check never passed, so a lone failed conversation was never reported on it and kept being run again. The judge now reads the array's lasttype: "result"element; the check's--settingsturn verbose off (verbose: false,viewMode: "default") and saylanguage: "english". Ok only onsubtype: "success",is_error: false,num_turns >= 1and, whenusageis there,usage.output_tokens >= 1; the answer's words are not read ("Okay" or an answer in another language is an answer).src/integration/spawn-run.ts,src/integration/integration.ts). A run that ran past 5 s and then exited non-zero with nothing on stdout (a crash with no output) isRunOutcome.silentand reaches the kit asRunEnd.timedOut, so it is reported only once another conversation's run completed, never on the runtime check alone. A provider refusal printsAPI Error: …on stdout and stays a plain failure, and a run whose stderr saysAPI Erroris not marked silent either. The detail still says… with nothing on stdout (it ran, then ended). The 5 s window is measured to the process's exit, not to classification.hidden_count, not the at most 200 listed ids (falls back to the list on a server without it).src/presets/codex.ts): ok only on exit 0 with aturn.completedand anagent_messagewith text in the--jsonevents, and noturn.failed(codexRuntimeCheckJudge, bounded capture, never logged);model_reasoning_effort="low"dropped (Codex sends a configured effort as is, so a model withoutlowrefused every check). The user's hooks stay on, as in the runs: a hook that breaks every run must fail the check too, since a Codex run prints--jsonevents and is never silent.doctor's runtime-check warning only while the status file is fresh (written within 3 min; a running daemon rewrites it every 30 s); otherwise doctor says "the daemon has not written its status for — it may be down", the wordsstatususes. A paused conversation is still said either way.spawnRun({ shell: true })uses Node's own spawn, so a command line that ran and failed with 1 isfailed, notspawn … ENOENT/not_started.Version 0.7.1 in
package.jsonandsrc/version.ts; CHANGELOG[0.7.1] — 2026-11-20; README updated (Could not process, the runtime check bullets, doctor, run classification).Verification
pnpm type-checkclean;pnpm buildclean;vitest run39 files, 646 passed, 3 skipped (platform-only), on Node 22.23.1 / macOS; CI on ubuntu, macOS and Windows. New tests: the Claude judge on an object, a verbose array, an error result,num_turns0 or missing,usage.output_tokens0 or missing, "Okay" and non-English answers passing, and the runner with verbose output forced; the Codex judge on real-shaped events and the runner on each; spawnRun'ssilentflag, the stderrAPI Errorguard, a shell run past the window (failed + silent), and the exit-time window; a silent failure not reported on the runtime check and reported astimed_outwithother_run_completed;hidden_countin the log; doctor on a stale and a fresh status file. The silent-failure,hidden_count, exit-time and stderr-guard tests were each run against the code without their change and fail there.verbose: true,viewMode: "verbose"or~/.claude.jsonverbose, and one result object with 0.7.1's; with the user'slanguage: "japanese"the request carried the Japanese instruction without our key and the English one with it; hooks stayed off with the new keys, and an unknown key or invalid value in--settingsleft the rest in force;ClaudeRunner.checkRuntimefrom this build: ok with the user's verbose setting on, not ok on a provider refusal. Codex 0.154.0: the request carries no forced effort, andCodexRunner.checkRuntimefrom this build is ok on an answer and not ok on a refusal. A refusedclaude -p --output-format textrun printsAPI Error: …on stdout (exit 1).Risk and rollback