feat(sessions): hard max-runtime backstop for headless runs — fix the stuck-mid-turn leak (v0.266.0)#466
Merged
Merged
Conversation
… stuck-mid-turn leak (v0.266.0) A headless/unattended run (automation/task/chat/ask) is an attachable TUI torn down at turn-end by the Stop beacon. If it hangs mid-turn it never beacons, so last_activity stays NULL: the idle-straggler sweep (needs a beacon) skips it and the idle-interactive janitor (headed-only) skips it too. Reaped by NOTHING, it lingers for days holding a ~500MB claude process + a cap slot. Confirmed on instawp: unattended runs stuck at 60h+, a primary driver of the box's memory pressure and "sudden unresponsiveness". - New reaper pass reaps a headless running row purely on wall-clock age once it exceeds the ceiling (settings unattendedMaxHours, default 24h, 0=off, 1h-30d). No turn-end beacon required — that's the point. - Scoped to the headless lane ONLY: headed (interactive member) sessions are never cut mid-work; they keep the idle-based interactive janitor (which only closes a detached one). - Guards: never yank a pane a human is attached to; teardownUnattended now cancels any dangling question/approval so an abandoned blocked run is released. - Settings → Runtime gains "Force-close headless runs after (hours)"; audited session.reaped reason:'max-runtime' + settings.unattendedMax.updated. Decision logic proven with an 8-case truth table (leak reaped, short runs protected, watched panes never yanked, done-orphan unchanged, 0 disables). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WPLeBNcNJiURPKiUZ6iMmF
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Diagnosing why the instawp box goes suddenly unresponsive turned up a session leak. A headless/unattended run (automation/task/chat/ask) is now an attachable interactive TUI, torn down at turn-end by the Stop beacon. If it hangs mid-turn it never beacons →
last_activitystaysNULL:Reaped by nothing, it lingers for days holding a ~500 MB
claudeprocess + a concurrency-cap slot. Confirmed live on instawp: unattended runs stuck at 60 h+ withlast_activity=NULL— a primary driver of memory pressure (service peak 57.4 G / 62 G).What
reapIdleSessions): reaps a headlessrunningrow purely on wall-clock age once it exceeds the ceiling — no beacon required.Settings → Runtime → "Force-close headless runs after", default 24 h,0= off, clamped 1 h–30 d (unattendedMaxHours).hasClient);teardownUnattendednow cancels any dangling question/approval, so an abandoned blocked run past the ceiling is released cleanly (the idle path still keeps a legitimately-blocked run alive). Row stays Resumable.session.reapedreason:'max-runtime'+settings.unattendedMax.updated.Verification
npm run typecheck+npm run build+webbuild clean.0→ disabled/no-op.Follow-up lever (not in this PR): the headed idle timeout on instawp is set to 72 h — worth lowering operationally to shed the 14 idle member sessions faster.
🤖 Generated with Claude Code