Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,13 @@ has a matching GitHub release whose body comes from its section below.
gateway's own board, switch and stop with the node's API key, so `berth gateway` must run; `berth harbor
--gateway` and `--key-file` name them when they are not in the default place.

### Fixed

- An engine stopped by a SIGTERM or SIGINT sent to it directly, as `kill PID`, a stop script or Ctrl+C sends,
no longer shows as unresponsive (LATCHED) in `berth status` and the harbor. It shows as offline, as an engine
Berth or its supervisor stopped already does. One killed by SIGKILL or any other signal still shows as
unresponsive, as a crash does.

## [v0.4.1] - 2026-11-01

Each card on a machine with several now has its own gateway URL, `/cards/CARD/v1`, and one model can be served
Expand Down
11 changes: 9 additions & 2 deletions internal/runner/finished_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,9 @@ import (
// A batch engine that exits 0 stopped answering on purpose: the engines
// tab shows it down, not latched (issue 265). So does one Berth stopped,
// through berth stop or a stop signal from its supervisor, whatever code
// the signal leaves it. One that exits non-zero on its own keeps the
// latch, so a crash still reads as one.
// the signal leaves it, and one stopped by a SIGTERM or SIGINT sent to it
// directly. One that exits non-zero on its own, or dies of any other
// signal, keeps the latch, so a crash still reads as one.
func TestACleanExitDoesNotLatchTheEngine(t *testing.T) {
forever := []string{"sh", "-c", "exec sleep 30"}
for _, c := range []struct {
Expand All @@ -42,6 +43,12 @@ func TestACleanExitDoesNotLatchTheEngine(t *testing.T) {
}{
{"exit 0", []string{"sh", "-c", "exit 0"}, nil, "down"},
{"exit 1", []string{"sh", "-c", "exit 1"}, nil, "latched"},
// A stop sent to the engine itself, as a shell's strata stop or
// kill PID does, is on purpose too. A SIGKILL is what the OOM killer
// and a crash look like, so it keeps the latch.
{"SIGTERM to the engine", []string{"sh", "-c", "kill -TERM $$"}, nil, "down"},
{"SIGINT to the engine", []string{"sh", "-c", "kill -INT $$"}, nil, "down"},
{"SIGKILL to the engine", []string{"sh", "-c", "kill -KILL $$"}, nil, "latched"},
{"berth stop", forever, func(t *testing.T, b *broker.Broker) {
if _, err := b.Stop("deduper"); err != nil {
t.Fatal(err)
Expand Down
2 changes: 1 addition & 1 deletion internal/runner/runner.go
Original file line number Diff line number Diff line change
Expand Up @@ -395,7 +395,7 @@ func Run(o Options) int {
err = nil // exited cleanly; a grandchild still held the output copy
}
code := exitCode(cmd, err)
if code == 0 || line.stopWasAsked() {
if code == 0 || line.stopWasAsked() || stopSignal(code) {
_ = line.send(proto.Msg{Op: proto.OpFinished})
}
close(done)
Expand Down
7 changes: 7 additions & 0 deletions internal/runner/runner_unix.go
Original file line number Diff line number Diff line change
Expand Up @@ -105,3 +105,10 @@ func exitCode(cmd *exec.Cmd, err error) int {
}
return ee.ExitCode()
}

// stopSignal is whether the engine died of a signal that asks a process to
// stop, SIGTERM or SIGINT, as kill PID, a script's stop or Ctrl+C sends it:
// a stop on purpose, not a crash.
func stopSignal(code int) bool {
return code == 128+int(syscall.SIGTERM) || code == 128+int(syscall.SIGINT)
}
4 changes: 4 additions & 0 deletions internal/runner/runner_windows.go
Original file line number Diff line number Diff line change
Expand Up @@ -98,3 +98,7 @@ func exitCode(_ *exec.Cmd, err error) int {
}
return 1
}

// stopSignal is false on Windows: there are no stop signals, and an exit
// code says nothing about who ended the engine.
func stopSignal(int) bool { return false }
Loading