Repository navigation
chore(release): v0.4.1 - #283
Merged
Merged
Conversation
Merge main into next: v0.4.0 and the CI fixes from #269
Goal: a wrapper unit's memory and swap limits never reach the engine, because berth run starts it in a sibling transient scope; the model- resident process pages freely under a MemorySwapMax=0 wrapper. How: slice = "NAME.slice" on the engine entry (berth run --slice wins, like every other flag) passes --slice to systemd-run, placing the engine's scope under the slice unit, whose CPU, memory and swap policy covers the engine and everything it forks. Slice names are validated by config check and by run, before berthd is dialed. Gateway launches carry the entry's slice on the run line. Where no scope is created, a Type=notify service or no user manager, a set slice is dropped with a Warn naming the engine, the slice and the reason. An entry with no slice starts its scope exactly as before, on every platform.
feat(run): start the engine scope under a configured systemd slice
On a node with several cards, berth hold and berth vfio run always took the first one. Both now take --gpu, the same card names and PCI addresses berth run takes. hold accepts a comma list for one hold on several cards. vfio run takes one card, refuses a list, and reads the named card's nvidia-smi index from berthd's status narrowed to that card. Without --gpu, both take the first card as before.
feat(cli): name the card for berth hold and berth vfio run
A hold claim held every card on the node, so a game pinned to one card drained the engines on all of them. A [[claim]] now takes gpu, the same card names and PCI addresses berth run --gpu takes. berthd resolves it to the node's cards at start and asks for the hold on those cards only; a name the node lacks is logged and the claim holds every card. Each card reports the claim active only where it holds, and node.View marks a claim inactive in a view of cards it does not touch, so the gateway starts background engines on the free cards. A claim without gpu holds every card as before.
The About tab now leads with the node's version and links to the docs, changelog, issues and source, covers machines with several cards, and drops planned features that shipped or changed. Its sections are cards in two columns on a wide screen, with the same section rail as the other tabs. The header no longer clips the ADVANCED switch or cuts its tagline off, and a phone alert wraps instead of ending in an ellipsis. A section link such as #sec-library opened the card tab on a reload. It now opens the tab that holds the section.
The scroll ran before the first status rendered its alerts and panels, so a linked section landed under the header or far below it.
The harbor never showed claims: a game's hold appeared only as a claim: lease, and an app a track claim names, such as OBS, did not appear at all. A CLAIMS panel now lists the rules matching now, as berth claims does. A hold names its lease or says it is waiting, a track gives the card memory it names, and on a machine with several cards each row names its card. Idle rules are counted. The demo fixture carries an active OBS track and idle built-in rules.
feat(claims): hold only the cards a claim's gpu names
feat(harbor): rewrite the About tab and fix header clipping
feat(harbor): list the claims matching now on the card tab
berth stop and a stop signal from a supervisor end the engine on purpose, but the engine dies from the signal with a non-zero code. berth run reports only exit 0 as finished, so status keeps the engine LATCHED. The two new cases fail until the run reports the stops it carries out. Refs #275
berth run now sends finished when it ended the engine itself, whatever the exit code: a drain past the park step (berth stop, a higher priority, a park that failed), a wake that never answered, or a stop signal it forwarded from llama-swap or systemd. berthd clears the latch, and the engine shows as offline instead of unresponsive. An engine that exits non-zero on its own still latches. Refs #275
fix(run): report an engine the run stopped as finished
One model served by an engine on gpu0 and another on gpu1. A request to /cards/gpu1/v1 must reach gpu1's engine, a card named by its PCI address must work, an unknown card is a 404 listing the cards, and the card's model list names only its engines' models. berth config check must accept the two entries. Both tests fail until the routes and the config rule land.
/cards/CARD/v1 takes chat, completions, embeddings, models and berth/wait. CARD is the card's name or PCI address. The gateway resolves the model among the engines whose entry runs on that card and strips the prefix before it proxies. berth config check now lets one model be served by one engine per card, and still refuses two on one card or where one names no card. A request without the prefix is routed as before. First slice of v0.5, per-card gateway endpoints.
feat(gateway): one base URL per card
The roadmap lists what is merged to next for v0.4.1 and the four v0.5 slices for per-card gateway endpoints, with the first one linked. The design page puts joined gateways in v0.6, as the roadmap does. The pre-release page names v0.4.1-rc.N tags. The README covers hold, vfio run and claim cards, and the per-card base URL. The agent skill explains the UP column (LATCHED versus NO) and what berth stop does with no lease.
docs: v0.4.1 contents, the v0.5 plan, and stale version references
One model served on gpu0 and gpu1. A plain /v1 request must go to a parked copy before a free card, a free card before one a lease holds, and a held card before one a game pins. With nothing to tell the cards apart, the first entry wins. Three cases fail until the gateway ranks the cards.
When several entries serve the model on different cards and no card is named, resolve ranks each by what reaching it takes: a resume from RAM, a free card, a drain of the holder, then a card a VM or a pinned lease keeps. An engine that is up still answers first, and a tie keeps berth.toml's order. Second slice of v0.5, per-card gateway endpoints.
The broker must keep a new batch or foreground lease waiting with a reason that names the cordon, still grant a hold, and grant the waiter after an uncordon. berth cordon and uncordon must work against a live daemon and show in berth status. The gateway must treat the card as out of reach and start no background run for it. These fail until the cordon lands.
berth cordon CARD stops new grants on a card, holds aside, while the node's other cards serve; berth uncordon CARD lifts it and schedules the card's waiters at once. A cordon drains nothing. Waiters say the card is cordoned, berth status marks it CORDONED, and berth events records card.cordoned and card.uncordoned. The gateway treats the card as out of reach: no background engine starts there and a ComfyUI job for it queues. The cordon lives in berthd's memory, so a restart clears it. Third slice of v0.5, per-card gateway endpoints.
feat(gateway): place a plain request on the cheapest card
feat: berth cordon and uncordon a card
On a two-card node with gpu1 cordoned, GET /v1/berth/board must give each card its base URL and cordon, and each engine the cards its entry runs on, a PCI address and a missing gpu key included. /gui/state must carry the same cards per engine. Fails until the board and the rows carry them.
guiRow carries the cards an engine's entry runs on, from berthd's status, so GET /v1/berth/board and /gui/state both name them. Each board card gets its base URL on the gateway and its cordon. The /gui/ page shows each row's card as a chip when the rows span more than one card, so launching the copy of a model on gpu1 picks that card from a phone. One card, or berthd not answering, shows no chip. Fourth slice of v0.5, per-card gateway endpoints.
feat(gateway): a card picker on the board and the /gui/ page
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brings
nextintomainfor v0.4.1. The release commit dates[Unreleased]as[v0.4.1] - 2026-11-01with a summary, sets the README status to v0.4.1, and moves the per-card gateway work on the roadmap from "Next: v0.5" to the shipped v0.4.1 section. v0.5 keeps the harbor SWITCH tab. The CHANGELOG section lists every change.Proof
v0.4.1-rc.1is published as a pre-release fromnextat bd0d8bc, the same tree as this branch apart from the release commit.releases/latestis still v0.4.0./cards/gpu0/v1/modelslists the card's models,/cards/gpu9/v1/modelsanswers 404 withno card gpu9 on this node (cards: gpu0), andGET /v1/berth/boardgives gpu0 itsbaseof/cards/gpu0/v1. A chat through/cards/gpu0/v1forrouter-stratadrained the UD Strata and started SC117, and a plain/v1chat for the UD model switched back in 78 s./gui/statenamesgpu0on every row.go build,gofmt,go vet,go test ./..., the Windows build and vet, and golangci-lint (0 issues) pass locally on bd0d8bc.After merge, the operator tags
v0.4.1on the merge commit.