Repository navigation
fix: settled issues and the maintainer's interview decisions (redirects, UTF-8, caps, admission, prompt cache, panel) - #203
Merged
Conversation
The self-test's and the context probe's unload read the pin through Pool.Pinned and then stopped the model through Pool.Unload, each taking the pool's lock on its own. A pin saved between the two was overridden: the idle job unloaded a model a person had just pinned, leaving it pinned and not loaded. Pool.UnloadUnpinned reads the pin and stops the model under one hold of p.mu, so a pin saved through SetPinned lands either before the check and is honoured, or after the stop. Both idle jobs route through it and map runtime.ErrPinned to their own ErrPinned; the operator's Unload keeps its rule and still stops a pinned model. The pin state stays the pool's one set; nothing copies it. Resolves: iss-2610042033419572 Assisted-by: Claude
The update check's content fetch dropped the access token only on the hop that left the Hub's origin. net/http rebuilds each hop's headers from the first request, so a chain that left and came back sent the token to the Hub again, letting a host off the origin choose one authenticated Hub request. It also followed an https-to-http hop. The model download set no redirect policy at all and relied on net/http's stripping, which keeps the token on a hop to the Hub's host at another port or to a subdomain. Both now go through doContent and its one policy, followContent: keep the 10-hop limit, refuse a non-https hop when the base URL is https, and drop Authorization from every hop once any hop of the chain is off the Hub's origin. refuseOffOrigin, the API client's policy, is unchanged. SmallFileAt's non-200 error now names the content CDN and gives no token advice when the refusal came from off the Hub's origin, since the CDN is never sent the token. Resolves: iss-2610071200187548 Resolves: iss-2610071200187920 Resolves: iss-2610071200183905 Assisted-by: Claude
Go's json.Unmarshal into RawMessage does not validate UTF-8 inside strings, and json.Marshal forwards those bytes unchanged. mlx-lm 0.32.0's handler then fails raw_body.decode() on its handler thread, drops the connection, and the client gets a 502 it cannot act on. The completion routes (/v1/chat/completions and /v1/completions share one handler) now run utf8.Valid on the body bytes they already hold, right after the size-limited read and before the JSON parse, and refuse with a 400 in the usual error shape: "request body is not valid UTF-8". Nothing is resolved, acquired or forwarded for such a request. The prompt-reading ADRs govern any further reading of prompt content: "We will treat any further reading of prompt content by the gateway as a new decision that supersedes this record, as the records before it said." (adr-2610042021365934) This check is not such a reading. It is a check of the whole body's wire encoding, the same class as the JSON syntax check the gateway already makes over every byte: RFC 8259 section 8.1 requires JSON exchanged between systems to be UTF-8. It names no field, learns nothing about what any prompt says, keeps nothing, and the refusal carries none of the body; the archtest readers' scan is untouched. TestMergingRefusesInvalidUTF8InSystemContent asserted the old relay of invalid bytes; on the network path the body is now refused before merging, so it asserts the 400 and that nothing reached the model server. The merge's own UTF-8 guard stays, since the bridge's Ask path merges a body it did not read from the network, and is now held by a case in TestMergeSystemMessagesRefusesShapesItCannotRebuild. Resolves: iss-2610042036094209 Assisted-by: Claude
The idle threshold stopped at one hour, so an operator could not keep the idle jobs to a long quiet spell such as the night: their threshold was already at the cap. The maintainer confirmed 24 hours on 2026-10-07. MaxIdleThresholdSec is now 86400. The config validation, the panel's Idle threshold field and docs/context-probe.md move together, and a test holds the field to exactly the server's bounds beside the three-surfaces check that it is never narrower. The minimum and the default are unchanged, so this is additive. Resolves: iss-2610041948272032 Assisted-by: Claude
A resident-only Acquire could return ErrGone while another caller's load was finishing; when that load's modelLoaded report reached Enqueue before drain reached dequeue, Enqueue skipped the model as already queued and dequeue then removed it, so the model went unprobed until its next load. The queue now keeps one flag for its head: Enqueue sets it when the model it is asked for is the head, serve clears it before each look at the model, and dequeue moves a flagged head to the back instead of dropping it. A load reported before the probe's last look is one that look sees, so it asks for nothing more; only one reported after it survives. A single flag is enough because only the head is ever being served, which makes it simpler than a generation counter. Resolves: iss-2610032231045098 Assisted-by: Claude
The second update removes its own aside copy just after the download mark is cleared, outside the lock, so the test sometimes counted two copies straight after the update settled. That failed it in about 1 run in 10 under load. The test now waits for the one held copy. The behaviour it guards is unchanged. It passes 40 runs in a row under -race. Refs: iss-2610042101430192 Assisted-by: Claude
An interview settled each open issue whose remedy needed a choice. The remedies now name the chosen fix. A moot issue moves to wontfix. The probe's usability seed is promoted to a draft intent, and a decision-model self-test is seeded for later. One decision line lists every answer. The self-test docs now say models that cannot chat are passed over, which has been the behaviour since a026423 and resolves the first issue. Resolves: iss-2610041945034263 Assisted-by: Claude
…ps on length Every bridged request asked for max_tokens = 1024 whatever window the model was served at, so a long answer stopped mid-sentence without a word, and a reasoning model could spend all of it thinking and be reported as having answered with nothing. buildRequest already fell back to half the window for the answer when the window was too small for the fixed figure; that half becomes the rule, so the answer grows with the window while the history keeps the other half and the request still fits the window the gateway judges. The notice comes from the gateway's finish_reason, not a guess: a stream ending on "length" is followed by a message of its own saying the answer reached its length limit (kept out of the history), and one that wrote nothing says the model reached its length limit before it wrote an answer. Resolves: iss-2610041945020793 Assisted-by: Claude
Saving an unrelated setting was never refused over api_unload_off only because the field has no validation rule. No test named it, so a later rule could have refused every other save without a failure. The form helper now posts it, the unedited-save table has a row with the route turned off, and a new test changes another setting while the switch is off and checks it stays off. Both fail when Validate is made to refuse the switch. Resolves: iss-2610042100409263 Assisted-by: Claude
…'s encoding The security review of the UTF-8 refusal found that the system-message merge decoded a lone surrogate escape (\ud800) to U+FFFD, rewriting content into a character the client never sent. The merge now declines such content, as it declines invalid bytes. Ask, the bridge's in-process route, now makes the same encoding check as the network route, as its contract says. The docs say an escape is checked as the bytes it is, and a decision line records why the check fits the prompt-reading ADR. Refs: iss-2610042036094209 Assisted-by: Claude
The security review of the shared redirect rule found that only the update check's read named the content CDN on a refusal. A download the CDN refused, such as one with an expired signed URL, still read as the Hub needing an access token, though the CDN is never sent one. Both content paths now build their refusal through one helper. Refs: iss-2610071200183905 Assisted-by: Claude
The security review of the idle-unload pin race fix found a smaller window left: a settings save writes the pin under cfgMu and tells the pool after releasing it. Closing it needs a lock-order check first, so it is filed with that remedy. Assisted-by: Claude
…he reference The bridge reads a message only when it is created, so a mention or a question added by editing goes unanswered, and the page did not say so. It now says so under Limits. docs/transcript.md mixed a how-to with reference material. The recording field and the icons drawn from it now live in the models-list reference under Recording, and the how-to keeps the marking steps, the bridge and debug-logging rules, and a link. Resolves: iss-2610041945030524 Resolves: iss-2610032216257762 Assisted-by: Claude
The update check's press release for itd-2610030857275099 says a decision model is never marked, but criterion 9 and the panel gave an unreviewed newer version the 'newer version awaiting review' mark. The maintainer decided on 2026-10-07 that the press release stands. An unreviewed newer version of a decision model is now recorded as the check records a model with nothing newer: no mark, no Update. A reviewed version is still the only one a decision model is offered or fetched at, so removing the mark does not open an update the review gate would have refused. The awaiting_review status is gone from the check, the registry (a stored record carrying it is dropped on load, pre-1.0) and the panel; the intent's audit notes record the withdrawn mark. Resolves: iss-2610042101436891 Assisted-by: Claude
The update card's test that HuggingFace names are shown safely searched the panel source for escapeHtml calls, and the card showed no file names at all, so the file-name half of itd-2610030857275099 criterion 11 was never exercised. The maintainer decided on 2026-10-07 to show and test file names rather than drop that half. The check now records the files a newer version changes, adds or removes (documentation never), bounded to the first ten by name plus a count of all, so a repository that changes a thousand shards costs the index a dozen names. The registry holds a planted list to plain names, the bound and a count no smaller than the list. The card lists them on a line of its own through escapeHtml, and a UI test renders the panel's own renderModels with markup in the repository and a file name, built from the registry's own JSON, and asserts both appear as text. Resolves: iss-2610042101436222 Assisted-by: Claude
When a program unloads a model through /api/models/unload the recorder writes a removal with reason released and the kind of caller, but the per-model counters the Statistics tab draws counted evictions only, so the panel never showed it. AGENTS.md treats a Go capability with no panel equivalent as a gap. ModelCounters now carries released, counts by caller kind (an unknown kind under ""), kept apart from evictions and copied out of Summary. The panel's model card says "unloaded by a program N× (...)" beside "evicted N×", in its own words for each kind and never echoing a kind it does not know. A UI test draws the card from stats.ModelCounters as Go marshals it and walks stats.CallerKinds, so a renamed field or a new kind without words fails. Resolves: iss-2610042100405045 Assisted-by: Claude
The read bounds turned one connection held forever into one held for the bound, then reconnect; nothing capped how many a peer could hold at once, so a machine reconnecting in a loop on /pair or the API could hold a socket and a goroutine per attempt. The maintainer decided on a cap per client address plus a global cap per listener. One wrapper, capConns, now sits under every listener: the plain ones in runServer before they serve, and the TLS ones beneath the TLS layer so a refused connection is closed before any handshake byte is read and net/http still sees the *tls.Conn. A connection over 32 from one address or 512 on the listener is closed on accept, counted, and logged at debug at most once a second without the peer's address. The slot is released once, however many times the connection is closed. Loopback is held to the 512 only: every account on this Mac is one address, and a per-address cap there would let one local client lock the panel out. IPv6 peers are keyed by the full address; the figures and what would show them wrong are recorded in .abcd/work/DECISIONS.md. Resolves: iss-2609190254516275 Assisted-by: Claude
The self-test shared idle_threshold_sec with the context probe, whose default is five minutes, so out of the box it loaded and benchmarked a model five minutes after the last request: an ordinary pause in use. On 2026-10-04 it started loading models straight after a peer's run. The maintainer wants the two jobs separated, the self-test waiting four hours by default while the context probe keeps its own threshold. self_test_idle_threshold_sec sits beside idle_threshold_sec, defaults to 14400 seconds and takes the same 60-86400 bounds, on all three surfaces: config validation and load-time repair, a field in the panel's Self-test box with the panel's self-test prose reading it, and the self-test pages. The idle loop keeps one idea of what counts as use, including every admitted client request, and takes the quiet period per run. The self-test's default wait moves from 300 s to 4 hours, which the repository records as breaking on the decode_concurrency precedent. Resolves: iss-2610041956214381 Assisted-by: Claude
…ng folder The restore removes the version that failed to load just after it writes the record, so the test sometimes found it still in the staging folder when the record had already changed. Lanes 3 and 11 saw it fail under full-suite load. The test now waits for the folder to empty. It passes 40 runs in a row under -race. Refs: iss-2610042101430192 Assisted-by: Claude
…6-10-07b # Conflicts: # internal/gateway/control_untouched_test.go
The security review of the connection caps found that with a bind to :: the panel's localhost reaches the dual-stack wildcard listener as ::1. There it shared the 512 with every remote peer, so a machine on the LAN that filled the listener locked the operator out of the panel, with far fewer connections than descriptor exhaustion needed before. Loopback now counts against a 512 of its own, and remote peers against theirs. The docs say so, and that the per-address cap guards against accidents, not against a peer with many addresses. Refs: iss-2609190254516275 Assisted-by: Claude
…lake The connection caps' security review assessed how much memory whole request bodies can hold on a keyless install. The cap now bounds it, but the exposure predates it and is filed with a remedy. A Discord bridge test that failed once under full-suite load, on a base older than today's bridge changes, is filed to be reproduced before it is called a flake. Assisted-by: Claude
chargeLocked charges each model one served window of KV per decode slot (DecodeConcurrency sequences), but startLocked sized the model's semaphore at 2 x DecodeConcurrency. At the default of 1 the server is launched without --decode-concurrency, so mlx-lm's own batch limit let both admitted requests decode together, and two long requests on one model could build about twice the cache the budget reserved. The maintainer decided on 2026-10-07 to admit what is charged: the semaphore is now sized to DecodeConcurrency. At the default a second request to the same model waits for the first to finish. The overload ceiling (cap(sem) + MaxQueueDepth) falls by the same amount, so MaxQueueDepth keeps meaning "requests waiting beyond the batch". TestTheChargeCoversEveryRequestThePoolAdmits holds the charge to every request the pool runs at once, at 1, 2 and 3 batched requests, and checks the next request waits until a running one is released. Three tests that assumed the 2x capacity are updated; the resident-only acquire test now releases the loading request first, as the tool-call probe's always is. impact: breaking (a behaviour change, recorded in CHANGELOG and docs). Resolves: iss-2610071035138788 Assisted-by: Claude
With each model admitting only the requests its charge covers, one by default, a client arriving while the tool-call probe held the model waited for the probe's whole answer, up to two minutes. The security review of that change found it. The probe now watches the model while its request runs, as the self-test does. When a request is in flight beside its own hold, it abandons the request, lets the model go, and is asked again once the model falls quiet. Refs: iss-2610071035138788 Assisted-by: Claude
…al KV serve.py built server.LRUPromptCache(cli.prompt_cache_size) with the entry count alone, so mlx-lm 0.32.0 kept up to ten prompt caches with max_bytes at its default of 2^63. Each entry can be a whole served window of KV, so a model serving several long conversations held several windows of cache that its charge does not count. The maintainer decided on 2026-10-07 to bound the cache within the existing charge: max_bytes is one served window of real (uninflated) KV for one sequence, and the charge is not raised. The real figure is stored, not derived: kv_bytes_per_token sits beside kv_charge_per_token in the registry, computed from the same shape by KVShape.BytesPerToken, bounded at the scan and again when read back (the charge's bound, and never above the charge). It travels through ModelFacts, the rescan (which fills it for existing models at the next start), the download and update records, app's ResolvedModel, runtime.ResolvedModel and the launcher's Spec. The launcher passes Dessau's own --dessau-prompt-cache-bytes N (N = real bytes per token x served window) straight after the socket, and --prompt-cache-size 10 explicitly. serve.py takes the flag by position before mlx-lm's parser sees the rest, refuses the launch on a missing, negative, non-integer or out-of-range value, and passes it as max_bytes= (the 0.32.0 signature, confirmed against the wheel: __init__(self, max_size: int = 10, max_bytes: int = 1 << 63)). A model whose real figure or window is unknown follows the precedent for a missing KVChargePerToken: it is charged the flat figure (capability.LoadCostOf), a documented fallback rather than a refusal. That charge reserves no cache, so its bound is 0 and the server keeps no prompt cache between requests; checked against the real 0.32.0 LRUPromptCache, max_bytes=0 holds no entries. Upstream's own --prompt-cache-bytes is not unused, as the issue first said: in 0.32.0 it drives a trim on the batched path (server.py around line 761) but never reaches the constructor. It is not passed. Still owed: a real-Mac footprint check (launcher footprint sampling) that a model holding several long conversations stays inside its charge. impact: fix. Resolves: iss-2610071035130302 Assisted-by: Claude
…named The security review of the prompt-cache bound found that a config claiming a compressed-latent cache lowers its own memory charge. It predates the bound and is filed with a remedy. Assisted-by: Claude
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every open issue that could be settled without a design decision, and every issue the maintainer settled in an interview on 2026-10-07. Each lane was built test-first on its own branch. Every lane that touches a trust boundary had an adversarial security review, and every review finding is applied or filed.
Fixes and changes
Pool.UnloadUnpinned)api_unload_offimpact: additive)impact: breaking: the self-test's default wait moves from 5 minutes)::bind) fixedimpact: breaking: a second request to one model waits at the default). The tool-call probe now gives way to a clientkv_bytes_per_tokenand passed to serve.pyTest-timing races in two staged-update tests that came with #202 are fixed (each passes 40 runs in a row).
Records
.abcd/work/DECISIONS.md. One issue is moot (wontfix). The context probe's usability check is promoted to draft itd-2610071208598233, and a decision-model self-test is seeded.Still owed
Gates (on the head pushed)
gofmt -l .empty;go build ./...;go vet ./...(darwin and linux)make test: exit 0, all packages under-race