Skip to content

feat(sessions): hard max-runtime backstop for headless runs — fix the stuck-mid-turn leak (v0.266.0)#466

Merged
vikasprogrammer merged 1 commit into
mainfrom
feat/headless-maxruntime
Jul 24, 2026
Merged

feat(sessions): hard max-runtime backstop for headless runs — fix the stuck-mid-turn leak (v0.266.0)#466
vikasprogrammer merged 1 commit into
mainfrom
feat/headless-maxruntime

Conversation

@vikasprogrammer

Copy link
Copy Markdown
Owner

Why

Diagnosing why the instawp box goes suddenly unresponsive turned up a session leak. A headless/unattended run (automation/task/chat/ask) is now an attachable interactive TUI, torn down at turn-end by the Stop beacon. If it hangs mid-turn it never beaconslast_activity stays NULL:

  • the idle-straggler sweep skips it (it requires a turn-end beacon — the "never touch a mid-first-turn long run" guard), and
  • the idle-interactive janitor skips it (headless-only exclusion).

Reaped by nothing, it lingers for days holding a ~500 MB claude process + a concurrency-cap slot. Confirmed live on instawp: unattended runs stuck at 60 h+ with last_activity=NULL — a primary driver of memory pressure (service peak 57.4 G / 62 G).

What

  • New reaper pass (extends sweep 2 in reapIdleSessions): reaps a headless running row purely on wall-clock age once it exceeds the ceiling — no beacon required. Settings → Runtime → "Force-close headless runs after", default 24 h, 0 = off, clamped 1 h–30 d (unattendedMaxHours).
  • Scoped to the headless lane only. Headed (interactive member) sessions are never selected here, so a human is never cut mid-work — they keep the existing idle-based interactive janitor (which only closes a detached one).
  • Guards preserved / added: never reaps a pane a human is attached to (hasClient); teardownUnattended now cancels any dangling question/approval, so an abandoned blocked run past the ceiling is released cleanly (the idle path still keeps a legitimately-blocked run alive). Row stays Resumable.
  • Audited session.reaped reason:'max-runtime' + settings.unattendedMax.updated.

Verification

  • npm run typecheck + npm run build + web build clean.
  • Decision logic proven with an 8-case truth table: the 60 h no-beacon leak → reaped; a legit 5 h first turn → protected; watched pane past ceiling → never yanked; classic idle straggler → unchanged; done-orphan → unchanged; 0 → disabled/no-op.

Follow-up lever (not in this PR): the headed idle timeout on instawp is set to 72 h — worth lowering operationally to shed the 14 idle member sessions faster.

🤖 Generated with Claude Code

… stuck-mid-turn leak (v0.266.0)

A headless/unattended run (automation/task/chat/ask) is an attachable TUI torn
down at turn-end by the Stop beacon. If it hangs mid-turn it never beacons, so
last_activity stays NULL: the idle-straggler sweep (needs a beacon) skips it and
the idle-interactive janitor (headed-only) skips it too. Reaped by NOTHING, it
lingers for days holding a ~500MB claude process + a cap slot. Confirmed on
instawp: unattended runs stuck at 60h+, a primary driver of the box's memory
pressure and "sudden unresponsiveness".

- New reaper pass reaps a headless running row purely on wall-clock age once it
  exceeds the ceiling (settings unattendedMaxHours, default 24h, 0=off, 1h-30d).
  No turn-end beacon required — that's the point.
- Scoped to the headless lane ONLY: headed (interactive member) sessions are
  never cut mid-work; they keep the idle-based interactive janitor (which only
  closes a detached one).
- Guards: never yank a pane a human is attached to; teardownUnattended now
  cancels any dangling question/approval so an abandoned blocked run is released.
- Settings → Runtime gains "Force-close headless runs after (hours)"; audited
  session.reaped reason:'max-runtime' + settings.unattendedMax.updated.

Decision logic proven with an 8-case truth table (leak reaped, short runs
protected, watched panes never yanked, done-orphan unchanged, 0 disables).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WPLeBNcNJiURPKiUZ6iMmF
@vikasprogrammer
vikasprogrammer merged commit 02d7c61 into main Jul 24, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant