Repository navigation
fix(run): report an engine the run stopped as finished - #277
Merged
Merged
Conversation
berth stop and a stop signal from a supervisor end the engine on purpose, but the engine dies from the signal with a non-zero code. berth run reports only exit 0 as finished, so status keeps the engine LATCHED. The two new cases fail until the run reports the stops it carries out. Refs #275
berth run now sends finished when it ended the engine itself, whatever the exit code: a drain past the park step (berth stop, a higher priority, a park that failed), a wake that never answered, or a stop signal it forwarded from llama-swap or systemd. berthd clears the latch, and the engine shows as offline instead of unresponsive. An engine that exits non-zero on its own still latches. Refs #275
This was referenced Oct 31, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #275 (point 3).
What changes for users
When Berth stops an engine itself,
berth statusand the harbor showed that engine as LATCHED (unresponsive), as if it had crashed. It now shows as offline. An engine that exits non-zero on its own still shows as LATCHED.Cause
The engine watcher clears a latch when
berth runsendsfinished, and the run sent it only on exit 0 (#266). A stop that Berth carries out ends the engine with a signal, so the exit code is non-zero: 143 or 137 on Linux, and 1 after a kill on Windows. Aberth stop, a drain for a higher priority, a park that failed, and llama-swap unloading a model all left the engine latched.Fix
The run's lease line records when the run sets out to end the engine.
finishedis sent on exit 0 or when that flag is set. The flag is set in four places:drain, after the park step, where every remaining step ends the engine. A drain that parks the engine does not set it.drain, before it kills a checkpointed engine.stopNow, which a failed or silent wake calls. It is now a method on the lease line.forwardSignals, when a stop signal arrives from llama-swap, systemd or Ctrl-C.Proof
TestACleanExitDoesNotLatchTheEnginegains two cases,berth stopand a SIGTERM to the run. The test-only commit fails both, and the fix commit passes them, 3 runs out of 3. The exit 0 and exit 1 cases keep their results.askStopcall indrainfails theberth stopcase. Removing theasked()call inforwardSignalsfails the supervisor case.stopNowand checkpoint paths have no test of their own. They use the same flag.go test ./..., the Windows build and vet, and golangci-lint (0 issues) all pass.