diff --git a/CHANGELOG.md b/CHANGELOG.md index 4124154..dd9cc32 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,11 @@ has a matching GitHub release whose body comes from its section below. ### Added +- `berth gateway --harbor URL` redirects the gateway's `/gui/` index and `/` to `URL/#switch`, so an old bookmark of the + engine list lands on the harbor's SWITCH tab. The per-engine wake pages, their state endpoints and the wake POST keep + answering, so a bookmark of one engine's web UI still opens its wake page. Without the flag, `/gui/` is the index + page. + - The harbor's SWITCH tab loads one llama-swap model: a paired device taps LOAD on a model that is not loaded, and the model shows loading until llama-swap reports it ready. The gateway's new `POST /v1/berth/load {"engine","model"}` does it by asking llama-swap for the model's `/upstream/MODEL/health`, since llama-swap has diff --git a/contrib/berth-gateway.service b/contrib/berth-gateway.service index e310cd1..38f805b 100644 --- a/contrib/berth-gateway.service +++ b/contrib/berth-gateway.service @@ -15,6 +15,10 @@ Wants=berthd.service [Service] # The address is [gateway] listen in ~/.config/berth/berth.toml (default # 127.0.0.1:9799), so a port change is a config edit and a restart here. +# Behind tailscale serve on this box: append --gui-listen 127.0.0.1:9796 --allow-host ..ts.net +# for the wake pages, and --harbor https://..ts.net:9798 to send the gateway's /gui/ index to the +# harbor's SWITCH tab. --harbor must name the origin the browser reaches the harbor at, and a wake page opened by that +# name is only reachable through a serve in front of --gui-listen. ExecStart=%h/.local/bin/berth gateway Restart=on-failure RestartSec=2 diff --git a/docs/gateway.md b/docs/gateway.md index dcc005c..f414dca 100644 --- a/docs/gateway.md +++ b/docs/gateway.md @@ -445,6 +445,14 @@ Point the bookmark (or a home-page tile) at `http://127.0.0.1:9796/gui/comfyui` - On a node with several cards, each row in `/gui/` also names its card. With one engine per card for a model ([One URL per card](#one-url-per-card)), the row you launch is the card you pick. +There is one switchboard: the harbor's SWITCH tab. `berth gateway --harbor URL` sends this index to it. `GET /gui/` and +`GET /` then answer 302 to `URL/#switch`, so an old bookmark of the engine list lands on the tab. Name the origin the +browser reaches the harbor at, the address `berth pair --url` takes (`https://box.tailnet.ts.net:9798` for a harbor +behind `tailscale serve`). A redirect only hands the browser that address, so a URL it cannot reach is a page that +never loads. `/gui/`, `/gui//state`, `POST /gui//wake` and `/gui/state` keep answering, so an +engine's own bookmark still opens its wake page, and the Host rule below covers those URLs too. Without `--harbor`, +`/gui/` is the index page and `/` redirects to it. +

The gateway's wake page for comfyui: comfyui is offline and the card is free, with a Launch comfyui button

A start never runs two engines on the card at once. Berth drains the holder only at its boundary: llama-swap once its @@ -453,9 +461,10 @@ once its `/slots` show idle (`drain_slots`, below). A pinned holder (a game, a r the page says so and the start waits until it ends. The wake page needs no API key: it can only start an engine you gave a `start` command. It listens on -loopback, accepts loopback Host names plus the gateway's `--allow-host` names (for `tailscale serve` in -front), and takes a start only as a POST from its own page, so another site cannot make a visitor's -browser drain the card. +loopback and accepts only a loopback Host or a name in the gateway's `--allow-host` list, so a wake page you +open by name needs that name in `--allow-host` and its own forwarder to `--gui-listen` (one `tailscale serve` +target per port). Any other Host gets 403. It takes a start only as a POST from its own page, so another +site cannot make a visitor's browser drain the card. ## Run an engine under a resource-policy slice diff --git a/docs/harbor.md b/docs/harbor.md index 7063734..7e25dd0 100644 --- a/docs/harbor.md +++ b/docs/harbor.md @@ -182,9 +182,12 @@ shows each under its card. The board and the actions are the gateway's (`GET /v1/berth/board`, `POST /v1/berth/switch`, `POST /v1/berth/stop` and `POST /v1/berth/load`, see [the gateway](gateway.md)). The harbor sends them with the node's API key, -so `berth gateway` must be running. `berth harbor --gateway URL` names the gateway when it is not on +so `berth gateway` must be running. The page asks the harbor and the harbor asks the gateway, so the browser only +ever opens the harbor's own origin. `berth harbor --gateway URL` names the gateway when it is not on `[gateway] listen`, and `--key-file` names the key when it is not in the default place. The harbor -reads the key and never creates it. Without a key, the tab says so and offers no actions. +reads the key and never creates it. Without a key, the tab says so and offers no actions. On the gateway side, +`berth gateway --harbor URL` redirects the gateway's old `/gui/` index and `/` to `URL/#switch`, so a bookmark of the +engine list lands on this tab; its per-engine wake pages keep answering. **The ENGINES tab.** - **The engine cards** answer "is this engine up and serving?" with the state the diff --git a/internal/cli/gateway.go b/internal/cli/gateway.go index 5654e6c..a4454f0 100644 --- a/internal/cli/gateway.go +++ b/internal/cli/gateway.go @@ -6,9 +6,12 @@ package cli import ( "context" "errors" + "fmt" "net/http" + "net/url" "os" "os/signal" + "strings" "syscall" "time" @@ -46,6 +49,7 @@ func runGateway(c *Context) int { headroomMiB := fs.Uint64("headroom-mib", 2048, "RAM kept for the machine on a cold start; a start leaving less available is refused") comfyListen := fs.String("comfyui-listen", "", "serve ComfyUI's own port behind Berth on this loopback address (web UI, API, websocket; a queued prompt wakes or starts ComfyUI first); empty = off") guiListen := fs.String("gui-listen", "", "serve the gui plane (/gui/ lists every engine with its state and one button; /gui/ starts an engine with a gui URL, then opens its web UI) on this loopback address; empty = off") + harborFlag := fs.String("harbor", "", "harbor base URL to redirect the gui plane's index (/gui/ and /) to, as its SWITCH tab is the one board; the wake pages and /gui/state keep answering; empty = keep the index page") keyFile := fs.String("key-file", apikey.DefaultPath(), "the node's API key, created on first start ($BERTH_API_KEY_FILE)") var allow hostList fs.Var(&allow, "allow-host", "extra Host name the loopback gateway answers to, for a same-machine reverse proxy such as tailscale serve (repeatable or comma-separated)") @@ -55,6 +59,10 @@ func runGateway(c *Context) int { if code, done := c.noArgs(); done { return code } + harborURL, err := harborBase(*harborFlag) + if err != nil { + return c.fail(Failf("%v", err)) + } log := c.Log() cfg, warns, err := suite.Merged(*cfgPath) if err != nil { @@ -83,7 +91,7 @@ func runGateway(c *Context) int { return c.fail(Systemf("api key: %v", err)) } g := &gateway.Gateway{Socket: *sock, Engines: cfg.Engine, Ready: *ready, Key: key, Log: log, HeadroomFloor: *headroomMiB << 20, - AllowHosts: allow, StateFile: defaultStateFile("gateway-engines.json"), + AllowHosts: allow, StateFile: defaultStateFile("gateway-engines.json"), Harbor: harborURL, CostFile: defaultStateFile("swapcost.json"), DeferredFile: defaultStateFile("comfyui-queue.json"), ConfigPath: *cfgPath} ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM) defer stop() @@ -118,3 +126,18 @@ func runGateway(c *Context) int { } return ExitOK } + +// harborBase is the harbor's base URL as the browser reaches it, with no +// trailing "/". Anything but an http or https address with a host fails at +// startup: every old bookmark would follow the typo instead of the index. +func harborBase(raw string) (string, error) { + if raw == "" { + return "", nil + } + u, err := url.Parse(raw) + if err != nil || (u.Scheme != "http" && u.Scheme != "https") || u.Host == "" { + return "", fmt.Errorf("--harbor %q is not an http or https address", raw) + } + u.Fragment = "" // the SWITCH tab's own link carries #switch, which is what an operator copies + return strings.TrimRight(u.String(), "/"), nil +} diff --git a/internal/cli/gateway_test.go b/internal/cli/gateway_test.go new file mode 100644 index 0000000..5ebdc33 --- /dev/null +++ b/internal/cli/gateway_test.go @@ -0,0 +1,40 @@ +// Copyright 2026 Trevin Tindall (T-Crypt) +// SPDX-License-Identifier: Apache-2.0 + +package cli + +import "testing" + +// --harbor is the harbor's base URL, so anything else is refused at startup +// rather than turning every old /gui/ bookmark into a dead redirect. +func TestGatewayHarborBase(t *testing.T) { + for _, c := range []struct{ in, want string }{ + {"", ""}, + {"https://box.tailnet.ts.net:9798", "https://box.tailnet.ts.net:9798"}, + {"https://box.tailnet.ts.net:9798/", "https://box.tailnet.ts.net:9798"}, + {"http://127.0.0.1:9798/#switch", "http://127.0.0.1:9798"}, + {"https://box.tailnet.ts.net/harbor/", "https://box.tailnet.ts.net/harbor"}, + } { + got, err := harborBase(c.in) + if err != nil || got != c.want { + t.Errorf("harborBase(%q) = %q, %v; want %q", c.in, got, err, c.want) + } + } + for _, bad := range []string{"ftp://x", "notaurl", "https://", "//box"} { + if got, err := harborBase(bad); err == nil { + t.Errorf("harborBase(%q) = %q, accepted", bad, got) + } + } +} + +// The gateway command takes the flag and refuses a value that is not an http +// or https address. +func TestGatewayHarborFlag(t *testing.T) { + isolate(t) + if code, _, errOut := runCLI(t, "gateway", "--harbor", "ftp://x"); code != ExitFail { + t.Fatalf("--harbor ftp://x = exit %d, want %d:\n%s", code, ExitFail, errOut) + } + if code, _, errOut := runCLI(t, "gateway", "--harbor", "notaurl"); code != ExitFail { + t.Fatalf("--harbor notaurl = exit %d, want %d:\n%s", code, ExitFail, errOut) + } +} diff --git a/internal/gateway/gateway.go b/internal/gateway/gateway.go index f232717..6084776 100644 --- a/internal/gateway/gateway.go +++ b/internal/gateway/gateway.go @@ -83,6 +83,11 @@ type Gateway struct { ConfigPath string // LogMax caps an engine's log file before it starts over; 0 = 64 MiB. LogMax int64 + // Harbor is the harbor's base URL as a browser reaches it. The gui + // plane's index (/gui/ and /) then redirects to its SWITCH tab, which is + // the board; empty keeps the index page. The per-engine wake pages and + // /gui/state answer either way. + Harbor string mu sync.Mutex deferred *deferredBook // loaded on first use from DeferredFile diff --git a/internal/gateway/gui.go b/internal/gateway/gui.go index 88668c3..cbfe59f 100644 --- a/internal/gateway/gui.go +++ b/internal/gateway/gui.go @@ -52,7 +52,18 @@ func (g *Gateway) GUIHandler() http.Handler { assets, _ := fs.Sub(guiFS, "gui") mux := http.NewServeMux() mux.Handle("GET /_wake/", http.StripPrefix("/_wake/", http.FileServerFS(assets))) - mux.HandleFunc("GET /gui/{$}", g.guiIndex) + // The harbor's SWITCH tab is the switchboard. When the operator names the + // harbor, that tab is the index, so an old bookmark to /gui/ or / lands + // there. Empty keeps the plane's own index page. The wake pages and + // /gui/state answer either way: they are bookmarks to an engine's web UI. + var index http.Handler = http.HandlerFunc(g.guiIndex) + root := "/gui/" + if g.Harbor != "" { + to := g.Harbor + "/#switch" + index = http.RedirectHandler(to, http.StatusFound) + root = to + } + mux.Handle("GET /gui/{$}", index) // The static route beats GET /gui/{name} in Go's ServeMux, so an engine // named "state" loses that one URL to the plane's state endpoint. mux.HandleFunc("GET /gui/state", func(w http.ResponseWriter, r *http.Request) { @@ -65,7 +76,7 @@ func (g *Gateway) GUIHandler() http.Handler { mux.HandleFunc("GET /gui/{name}/state", g.guiState) mux.HandleFunc("POST /gui/{name}/wake", g.guiWake) mux.HandleFunc("GET /{$}", func(w http.ResponseWriter, r *http.Request) { - http.Redirect(w, r, "/gui/", http.StatusFound) + http.Redirect(w, r, root, http.StatusFound) }) return g.guiGuard(mux) } diff --git a/internal/gateway/gui_test.go b/internal/gateway/gui_test.go index 43877c3..691ba0a 100644 --- a/internal/gateway/gui_test.go +++ b/internal/gateway/gui_test.go @@ -419,6 +419,72 @@ func TestGUIPlaneLaunchAPIOnly(t *testing.T) { } } +// The harbor's SWITCH tab is the one board: with --harbor the plane's index +// redirects an old bookmark there, and the wake page and the plane's state +// answer as before. Without it both index URLs are the index page. +func TestGUIPlaneIndexToSwitch(t *testing.T) { + const harbor = "https://box.tailnet.ts.net:9798" + // / with no harbor keeps the relative Location it has always sent. + for _, c := range []struct { + harbor string + path string + want int + loc string // the Location header, empty for the index page + }{{harbor, "/gui/", http.StatusFound, harbor + "/#switch"}, + {harbor, "/", http.StatusFound, harbor + "/#switch"}, + {"", "/gui/", http.StatusOK, ""}, + {"", "/", http.StatusFound, "/gui/"}, + } { + sock, b := daemon(t) + g := &Gateway{Socket: sock, Ready: time.Second, Key: testKey, Harbor: c.harbor, + Log: slog.New(slog.NewTextHandler(io.Discard, nil)), + Engines: []config.Engine{{Name: "comfyui", Type: "comfyui", URL: "http://127.0.0.1:1", + Start: []string{"false"}, GUI: "http://127.0.0.1:1/"}}} + srv := httptest.NewServer(g.GUIHandler()) + resp := guiReq(t, "GET", srv.URL+c.path, nil) + page, _ := io.ReadAll(resp.Body) + resp.Body.Close() + srv.Close() + loc := resp.Header.Get("Location") + if c.loc == "" { + if resp.StatusCode != c.want || !strings.Contains(string(page), `data-name="comfyui"`) { + t.Errorf("%s with no harbor = %d: %s", c.path, resp.StatusCode, page) + } + } else if resp.StatusCode != c.want || loc != c.loc { + t.Errorf("%s with harbor %q = %d to %q, want %d to %q", c.path, c.harbor, resp.StatusCode, loc, c.want, c.loc) + } + if st := b.Status(); len(st.Leases) != 0 { + t.Fatalf("opening the index took a lease: %+v", st.Leases) + } + } +} + +// The harbor does not take the wake page or the plane's state away: an +// engine's own bookmark still lands on its page, and the state endpoint +// still answers JSON. +func TestGUIPlaneWakeAndStateWithHarbor(t *testing.T) { + sock, b := daemon(t) + g := &Gateway{Socket: sock, Ready: time.Second, Key: testKey, Harbor: "https://box.tailnet.ts.net:9798", + Log: slog.New(slog.NewTextHandler(io.Discard, nil)), + Engines: []config.Engine{{Name: "comfyui", Type: "comfyui", URL: "http://127.0.0.1:1", + Start: []string{"false"}, GUI: "http://127.0.0.1:1/"}}} + srv := httptest.NewServer(g.GUIHandler()) + defer srv.Close() + + resp := guiReq(t, "GET", srv.URL+"/gui/comfyui", nil) + page, _ := io.ReadAll(resp.Body) + resp.Body.Close() + if resp.StatusCode != http.StatusOK || !strings.Contains(string(page), `data-engine="comfyui"`) { + t.Fatalf("wake page with a harbor = %d: %s", resp.StatusCode, page) + } + if rows := guiPlaneRows(t, srv.URL); len(rows) != 1 || rows[0].Name != "comfyui" { + t.Fatalf("/gui/state with a harbor = %+v", rows) + } + if st := b.Status(); len(st.Leases) != 0 { + t.Fatalf("reading the plane took a lease: %+v", st.Leases) + } +} + // A rebinding name cannot read the plane's state. func TestGUIPlaneGuard(t *testing.T) { sock, b := daemon(t)