Skip to content

Busy is not dead: the supervisor stops killing a working server - #495

Merged
gHashTag merged 1 commit into
feat/queen-supervisorfrom
fix/liveness-tolerance
Sep 21, 2026
Merged

gHashTag merged 1 commit into
feat/queen-supervisorfrom
fix/liveness-tolerance

Conversation

@gHashTag

Copy link
Copy Markdown
Owner

The liveness supervisor (#494) killed the server itself at 14 and at 20 lanes, about two minutes after the lanes filled: with every lane working the event loop is saturated and /health takes longer than ten seconds. The server was alive, and ending it destroyed every bee's turn in flight.

Probe timeout 10 s → 30 s, misses allowed 4 → 12 (about twelve minutes of silence). Both halves proven: a server answering in 3 s against a 5 s probe is still running after 30 s; a server frozen with SIGSTOP is ended (exit 143).

🤖 Generated with Claude Code

… server

The first version allowed four misses of a ten-second /health probe. Measured
2026-09-21, it killed the server itself - at fourteen lanes and at twenty, each
time about two minutes after the lanes filled:

  [liveness] /health did not answer (1 of 4)
  ...
  [liveness] /health did not answer (4 of 4)
  [liveness] the server stopped answering; ending it so the platform restarts the container
  [liveness] the server exited (137)

With every lane working the event loop is saturated and /health takes longer
than ten seconds. The server was alive and making progress, and ending it
destroyed every bee's turn in flight - the protection doing more damage than
what it protects against.

A probe now waits LIVENESS_TIMEOUT (30 s) and twelve misses in a row are
needed: about twelve minutes of silence. Both halves proven before shipping:

  a server answering in 3 s against a 5 s probe      still running after 30 s
  a server frozen with SIGSTOP                        ended, exit 143

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@gHashTag
gHashTag merged commit 143e0b5 into feat/queen-supervisor Sep 21, 2026
@github-actions

Copy link
Copy Markdown

❌ Tests failed — 11/2436 failed

Suite Passed Failed Skipped
agent 87/87 0 0
build 9/9 0 0
cdp-protocol 5/5 0 0
eval 93/93 0 0
server-agent 272/272 0 0
server-api 1252/1315 6 57
server-browser 6/6 0 0
server-integration 10/11 0 1
server-lib 279/279 0 0
server-pglive 1/3 2 0
server-root 68/68 0 0
server-skills 31/31 0 0
server-tools 240/243 3 0
shared 14/14 0 0
Failed tests
  • server-apithe note says who committed the branch, where it can be read > puts the salvage fact where the 1500-character cap cannot cut it
  • server-apia turn the provider killed does not spend the issue > is judged and sent back, but charges neither counter
  • server-apideriving work the repository already measured > carries the command that produced it
  • server-apideriving work the repository already measured > gives every candidate a path that can be a boundary
  • server-apideriving work the repository already measured > proposes nothing from code this project does not own
  • server-apisalvaging a turn that ended with its work uncommitted > refuses a worktree holding an unmerged path
  • server-pglivethe migration block, applied to a real PostgreSQL > creates every object it promises, and survives a second boot
  • server-pglivethe live gate > ran, or its absence is on the record
  • server-toolsnavigation tools > new_hidden_page opens a hidden tab
  • server-toolsnavigation tools > show_page restores a hidden page to visible
  • server-toolswindow tools > create_hidden_window creates and closes a hidden window

View workflow run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants