Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,11 @@ has a matching GitHub release whose body comes from its section below.

### Added

- `berth gateway --harbor URL` redirects the gateway's `/gui/` index and `/` to `URL/#switch`, so an old bookmark of the
engine list lands on the harbor's SWITCH tab. The per-engine wake pages, their state endpoints and the wake POST keep
answering, so a bookmark of one engine's web UI still opens its wake page. Without the flag, `/gui/` is the index
page.

- The harbor's SWITCH tab loads one llama-swap model: a paired device taps LOAD on a model that is not loaded, and
the model shows loading until llama-swap reports it ready. The gateway's new `POST /v1/berth/load
{"engine","model"}` does it by asking llama-swap for the model's `/upstream/MODEL/health`, since llama-swap has
Expand Down
4 changes: 4 additions & 0 deletions contrib/berth-gateway.service
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,10 @@ Wants=berthd.service
[Service]
# The address is [gateway] listen in ~/.config/berth/berth.toml (default
# 127.0.0.1:9799), so a port change is a config edit and a restart here.
# Behind tailscale serve on this box: append --gui-listen 127.0.0.1:9796 --allow-host <machine>.<tailnet>.ts.net
# for the wake pages, and --harbor https://<machine>.<tailnet>.ts.net:9798 to send the gateway's /gui/ index to the
# harbor's SWITCH tab. --harbor must name the origin the browser reaches the harbor at, and a wake page opened by that
# name is only reachable through a serve in front of --gui-listen.
ExecStart=%h/.local/bin/berth gateway
Restart=on-failure
RestartSec=2
Expand Down
15 changes: 12 additions & 3 deletions docs/gateway.md
Original file line number Diff line number Diff line change
Expand Up @@ -445,6 +445,14 @@ Point the bookmark (or a home-page tile) at `http://127.0.0.1:9796/gui/comfyui`
- On a node with several cards, each row in `/gui/` also names its card. With one engine per card for a model
([One URL per card](#one-url-per-card)), the row you launch is the card you pick.

There is one switchboard: the harbor's SWITCH tab. `berth gateway --harbor URL` sends this index to it. `GET /gui/` and
`GET /` then answer 302 to `URL/#switch`, so an old bookmark of the engine list lands on the tab. Name the origin the
browser reaches the harbor at, the address `berth pair --url` takes (`https://box.tailnet.ts.net:9798` for a harbor
behind `tailscale serve`). A redirect only hands the browser that address, so a URL it cannot reach is a page that
never loads. `/gui/<engine>`, `/gui/<engine>/state`, `POST /gui/<engine>/wake` and `/gui/state` keep answering, so an
engine's own bookmark still opens its wake page, and the Host rule below covers those URLs too. Without `--harbor`,
`/gui/` is the index page and `/` redirects to it.

<p align="center"><img src="https://raw.githubusercontent.com/T-Crypt/berth/main/assets/preview-wake.png" alt="The gateway's wake page for comfyui: comfyui is offline and the card is free, with a Launch comfyui button" width="560"></p>

A start never runs two engines on the card at once. Berth drains the holder only at its boundary: llama-swap once its
Expand All @@ -453,9 +461,10 @@ once its `/slots` show idle (`drain_slots`, below). A pinned holder (a game, a r
the page says so and the start waits until it ends.

The wake page needs no API key: it can only start an engine you gave a `start` command. It listens on
loopback, accepts loopback Host names plus the gateway's `--allow-host` names (for `tailscale serve` in
front), and takes a start only as a POST from its own page, so another site cannot make a visitor's
browser drain the card.
loopback and accepts only a loopback Host or a name in the gateway's `--allow-host` list, so a wake page you
open by name needs that name in `--allow-host` and its own forwarder to `--gui-listen` (one `tailscale serve`
target per port). Any other Host gets 403. It takes a start only as a POST from its own page, so another
site cannot make a visitor's browser drain the card.

## Run an engine under a resource-policy slice

Expand Down
7 changes: 5 additions & 2 deletions docs/harbor.md
Original file line number Diff line number Diff line change
Expand Up @@ -182,9 +182,12 @@ shows each under its card.

The board and the actions are the gateway's (`GET /v1/berth/board`, `POST /v1/berth/switch`,
`POST /v1/berth/stop` and `POST /v1/berth/load`, see [the gateway](gateway.md)). The harbor sends them with the node's API key,
so `berth gateway` must be running. `berth harbor --gateway URL` names the gateway when it is not on
so `berth gateway` must be running. The page asks the harbor and the harbor asks the gateway, so the browser only
ever opens the harbor's own origin. `berth harbor --gateway URL` names the gateway when it is not on
`[gateway] listen`, and `--key-file` names the key when it is not in the default place. The harbor
reads the key and never creates it. Without a key, the tab says so and offers no actions.
reads the key and never creates it. Without a key, the tab says so and offers no actions. On the gateway side,
`berth gateway --harbor URL` redirects the gateway's old `/gui/` index and `/` to `URL/#switch`, so a bookmark of the
engine list lands on this tab; its per-engine wake pages keep answering.

**The ENGINES tab.**
- **The engine cards** answer "is this engine up and serving?" with the state the
Expand Down
25 changes: 24 additions & 1 deletion internal/cli/gateway.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,12 @@ package cli
import (
"context"
"errors"
"fmt"
"net/http"
"net/url"
"os"
"os/signal"
"strings"
"syscall"
"time"

Expand Down Expand Up @@ -46,6 +49,7 @@ func runGateway(c *Context) int {
headroomMiB := fs.Uint64("headroom-mib", 2048, "RAM kept for the machine on a cold start; a start leaving less available is refused")
comfyListen := fs.String("comfyui-listen", "", "serve ComfyUI's own port behind Berth on this loopback address (web UI, API, websocket; a queued prompt wakes or starts ComfyUI first); empty = off")
guiListen := fs.String("gui-listen", "", "serve the gui plane (/gui/ lists every engine with its state and one button; /gui/<engine> starts an engine with a gui URL, then opens its web UI) on this loopback address; empty = off")
harborFlag := fs.String("harbor", "", "harbor base URL to redirect the gui plane's index (/gui/ and /) to, as its SWITCH tab is the one board; the wake pages and /gui/state keep answering; empty = keep the index page")
keyFile := fs.String("key-file", apikey.DefaultPath(), "the node's API key, created on first start ($BERTH_API_KEY_FILE)")
var allow hostList
fs.Var(&allow, "allow-host", "extra Host name the loopback gateway answers to, for a same-machine reverse proxy such as tailscale serve (repeatable or comma-separated)")
Expand All @@ -55,6 +59,10 @@ func runGateway(c *Context) int {
if code, done := c.noArgs(); done {
return code
}
harborURL, err := harborBase(*harborFlag)
if err != nil {
return c.fail(Failf("%v", err))
}
log := c.Log()
cfg, warns, err := suite.Merged(*cfgPath)
if err != nil {
Expand Down Expand Up @@ -83,7 +91,7 @@ func runGateway(c *Context) int {
return c.fail(Systemf("api key: %v", err))
}
g := &gateway.Gateway{Socket: *sock, Engines: cfg.Engine, Ready: *ready, Key: key, Log: log, HeadroomFloor: *headroomMiB << 20,
AllowHosts: allow, StateFile: defaultStateFile("gateway-engines.json"),
AllowHosts: allow, StateFile: defaultStateFile("gateway-engines.json"), Harbor: harborURL,
CostFile: defaultStateFile("swapcost.json"), DeferredFile: defaultStateFile("comfyui-queue.json"), ConfigPath: *cfgPath}
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
Expand Down Expand Up @@ -118,3 +126,18 @@ func runGateway(c *Context) int {
}
return ExitOK
}

// harborBase is the harbor's base URL as the browser reaches it, with no
// trailing "/". Anything but an http or https address with a host fails at
// startup: every old bookmark would follow the typo instead of the index.
func harborBase(raw string) (string, error) {
if raw == "" {
return "", nil
}
u, err := url.Parse(raw)
if err != nil || (u.Scheme != "http" && u.Scheme != "https") || u.Host == "" {
return "", fmt.Errorf("--harbor %q is not an http or https address", raw)
}
u.Fragment = "" // the SWITCH tab's own link carries #switch, which is what an operator copies
return strings.TrimRight(u.String(), "/"), nil
}
40 changes: 40 additions & 0 deletions internal/cli/gateway_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
// Copyright 2026 Trevin Tindall (T-Crypt)
// SPDX-License-Identifier: Apache-2.0

package cli

import "testing"

// --harbor is the harbor's base URL, so anything else is refused at startup
// rather than turning every old /gui/ bookmark into a dead redirect.
func TestGatewayHarborBase(t *testing.T) {
for _, c := range []struct{ in, want string }{
{"", ""},
{"https://box.tailnet.ts.net:9798", "https://box.tailnet.ts.net:9798"},
{"https://box.tailnet.ts.net:9798/", "https://box.tailnet.ts.net:9798"},
{"http://127.0.0.1:9798/#switch", "http://127.0.0.1:9798"},
{"https://box.tailnet.ts.net/harbor/", "https://box.tailnet.ts.net/harbor"},
} {
got, err := harborBase(c.in)
if err != nil || got != c.want {
t.Errorf("harborBase(%q) = %q, %v; want %q", c.in, got, err, c.want)
}
}
for _, bad := range []string{"ftp://x", "notaurl", "https://", "//box"} {
if got, err := harborBase(bad); err == nil {
t.Errorf("harborBase(%q) = %q, accepted", bad, got)
}
}
}

// The gateway command takes the flag and refuses a value that is not an http
// or https address.
func TestGatewayHarborFlag(t *testing.T) {
isolate(t)
if code, _, errOut := runCLI(t, "gateway", "--harbor", "ftp://x"); code != ExitFail {
t.Fatalf("--harbor ftp://x = exit %d, want %d:\n%s", code, ExitFail, errOut)
}
if code, _, errOut := runCLI(t, "gateway", "--harbor", "notaurl"); code != ExitFail {
t.Fatalf("--harbor notaurl = exit %d, want %d:\n%s", code, ExitFail, errOut)
}
}
5 changes: 5 additions & 0 deletions internal/gateway/gateway.go
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,11 @@ type Gateway struct {
ConfigPath string
// LogMax caps an engine's log file before it starts over; 0 = 64 MiB.
LogMax int64
// Harbor is the harbor's base URL as a browser reaches it. The gui
// plane's index (/gui/ and /) then redirects to its SWITCH tab, which is
// the board; empty keeps the index page. The per-engine wake pages and
// /gui/state answer either way.
Harbor string

mu sync.Mutex
deferred *deferredBook // loaded on first use from DeferredFile
Expand Down
15 changes: 13 additions & 2 deletions internal/gateway/gui.go
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,18 @@ func (g *Gateway) GUIHandler() http.Handler {
assets, _ := fs.Sub(guiFS, "gui")
mux := http.NewServeMux()
mux.Handle("GET /_wake/", http.StripPrefix("/_wake/", http.FileServerFS(assets)))
mux.HandleFunc("GET /gui/{$}", g.guiIndex)
// The harbor's SWITCH tab is the switchboard. When the operator names the
// harbor, that tab is the index, so an old bookmark to /gui/ or / lands
// there. Empty keeps the plane's own index page. The wake pages and
// /gui/state answer either way: they are bookmarks to an engine's web UI.
var index http.Handler = http.HandlerFunc(g.guiIndex)
root := "/gui/"
if g.Harbor != "" {
to := g.Harbor + "/#switch"
index = http.RedirectHandler(to, http.StatusFound)
root = to
}
mux.Handle("GET /gui/{$}", index)
// The static route beats GET /gui/{name} in Go's ServeMux, so an engine
// named "state" loses that one URL to the plane's state endpoint.
mux.HandleFunc("GET /gui/state", func(w http.ResponseWriter, r *http.Request) {
Expand All @@ -65,7 +76,7 @@ func (g *Gateway) GUIHandler() http.Handler {
mux.HandleFunc("GET /gui/{name}/state", g.guiState)
mux.HandleFunc("POST /gui/{name}/wake", g.guiWake)
mux.HandleFunc("GET /{$}", func(w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, "/gui/", http.StatusFound)
http.Redirect(w, r, root, http.StatusFound)
})
return g.guiGuard(mux)
}
Expand Down
66 changes: 66 additions & 0 deletions internal/gateway/gui_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -419,6 +419,72 @@ func TestGUIPlaneLaunchAPIOnly(t *testing.T) {
}
}

// The harbor's SWITCH tab is the one board: with --harbor the plane's index
// redirects an old bookmark there, and the wake page and the plane's state
// answer as before. Without it both index URLs are the index page.
func TestGUIPlaneIndexToSwitch(t *testing.T) {
const harbor = "https://box.tailnet.ts.net:9798"
// / with no harbor keeps the relative Location it has always sent.
for _, c := range []struct {
harbor string
path string
want int
loc string // the Location header, empty for the index page
}{{harbor, "/gui/", http.StatusFound, harbor + "/#switch"},
{harbor, "/", http.StatusFound, harbor + "/#switch"},
{"", "/gui/", http.StatusOK, ""},
{"", "/", http.StatusFound, "/gui/"},
} {
sock, b := daemon(t)
g := &Gateway{Socket: sock, Ready: time.Second, Key: testKey, Harbor: c.harbor,
Log: slog.New(slog.NewTextHandler(io.Discard, nil)),
Engines: []config.Engine{{Name: "comfyui", Type: "comfyui", URL: "http://127.0.0.1:1",
Start: []string{"false"}, GUI: "http://127.0.0.1:1/"}}}
srv := httptest.NewServer(g.GUIHandler())
resp := guiReq(t, "GET", srv.URL+c.path, nil)
page, _ := io.ReadAll(resp.Body)
resp.Body.Close()
srv.Close()
loc := resp.Header.Get("Location")
if c.loc == "" {
if resp.StatusCode != c.want || !strings.Contains(string(page), `data-name="comfyui"`) {
t.Errorf("%s with no harbor = %d: %s", c.path, resp.StatusCode, page)
}
} else if resp.StatusCode != c.want || loc != c.loc {
t.Errorf("%s with harbor %q = %d to %q, want %d to %q", c.path, c.harbor, resp.StatusCode, loc, c.want, c.loc)
}
if st := b.Status(); len(st.Leases) != 0 {
t.Fatalf("opening the index took a lease: %+v", st.Leases)
}
}
}

// The harbor does not take the wake page or the plane's state away: an
// engine's own bookmark still lands on its page, and the state endpoint
// still answers JSON.
func TestGUIPlaneWakeAndStateWithHarbor(t *testing.T) {
sock, b := daemon(t)
g := &Gateway{Socket: sock, Ready: time.Second, Key: testKey, Harbor: "https://box.tailnet.ts.net:9798",
Log: slog.New(slog.NewTextHandler(io.Discard, nil)),
Engines: []config.Engine{{Name: "comfyui", Type: "comfyui", URL: "http://127.0.0.1:1",
Start: []string{"false"}, GUI: "http://127.0.0.1:1/"}}}
srv := httptest.NewServer(g.GUIHandler())
defer srv.Close()

resp := guiReq(t, "GET", srv.URL+"/gui/comfyui", nil)
page, _ := io.ReadAll(resp.Body)
resp.Body.Close()
if resp.StatusCode != http.StatusOK || !strings.Contains(string(page), `data-engine="comfyui"`) {
t.Fatalf("wake page with a harbor = %d: %s", resp.StatusCode, page)
}
if rows := guiPlaneRows(t, srv.URL); len(rows) != 1 || rows[0].Name != "comfyui" {
t.Fatalf("/gui/state with a harbor = %+v", rows)
}
if st := b.Status(); len(st.Leases) != 0 {
t.Fatalf("reading the plane took a lease: %+v", st.Leases)
}
}

// A rebinding name cannot read the plane's state.
func TestGUIPlaneGuard(t *testing.T) {
sock, b := daemon(t)
Expand Down
Loading