Fix completion, run settings and concurrent cache state

CACHE_PROTOCOL.md

@@ -0,0 +1,77 @@
+# Relevance cache write coordination
+
+The committed cache remains UTF-8 JSON in the existing shape:
+
+```json
+{"version": 1, "entries": {"cache-key": {"decision": "yes"}}}
+```
+
+No cache-key, prompt-version, or record-format migration is required. Each
+instance tracks the records it has changed. A flush rereads the latest file,
+merges **only those dirty records**, and atomically replaces the JSON file.
+For conflicting keys, the last serialized flush wins. An old clean snapshot
+never overwrites a newer record or resurrects records after the file is cleared.
+A failed flush retains unsaved records so the same instance can retry.
+
+## Shared Node/Python protocol
+
+Updated Node and Python writers coordinate through `<absolute-cache-path>.lock/`
+on a local filesystem. It is a persistent directory, usually empty when idle.
+It must not be deleted while writers are active. Memory-only/browser caches do
+not access the filesystem or acquire claims.
+
+Each contender uses a new random 32-character lowercase hexadecimal token and
+atomically publishes `<token>.json` with these fields:
+
+```json
+{"version": 1, "hostname": "host", "pid_scope": "linux:pid:[123]", "pid": 1234, "token": "0123456789abcdef0123456789abcdef", "ticket": 0}
+```
+
+`ticket: 0` announces that the contender is choosing. It then publishes one more
+than the greatest visible ticket and waits until no contender is still choosing
+and no `(ticket, token)` pair sorts before its own. This is the bakery-lock
+algorithm; tokens break ties identically in both languages. JSON is published
+through `.<token>.tmp` followed by atomic rename, so readers never observe a
+half-written claim. Ticket/PID values must be positive safe integers (ticket 0
+is the sole choosing exception), tokens must match filenames, and every owner
+field must be well formed.
+
+A writer removes only its own unique claim on completion or error. It never
+removes the shared directory. A claim left by a crashed process can be removed
+only on POSIX, when its hostname and PID scope match the current process and a
+process probe proves its PID no longer exists. Windows and other unsupported
+platforms never signal an owner or automatically reclaim its claim; an
+abandoned claim there requires manual inspection/removal after owners stop. Linux scope identifies `/proc/self/ns/pid`
+so another container's live PID cannot be mistaken for a dead local process.
+Other platforms use their platform identifier. Missing namespace information,
+permission-denied probes, foreign owners and malformed records are not evidence
+of death and are never automatically stolen. Unique filenames avoid the race
+where two stale-lock reapers accidentally remove a replacement owner's lock.
+
+Lock acquisition waits at most two seconds (apart from filesystem call latency).
+Contention or an unidentifiable abandoned claim then fails the cache flush:
+Node rejects `flush()`; Python logs a warning, retaining its existing API contract.
+`filterRelevant`/`filter_relevant` already log cache failures and continue the
+classification result. Unknown/foreign abandoned records require an operator to
+inspect them; remove them only after establishing that their owners have stopped.
+Temporary files left by a killed process do not block later claims.
+
+## Compatibility limits
+
+This coordination protocol requires all concurrent writers to be upgraded.
+Older clients can still read the version-1 file, but an older writer ignores the
+claims and can still overwrite newer entries. Network/distributed filesystems,
+cache-path aliases, and external deletion or modification of active claims are
+outside this local-filesystem coordination contract. A loaded instance is an
+in-memory snapshot; reopen it (or successfully flush local changes) to see other
+writers' new records.
+
+Node also serializes flush calls on the same instance. A `put` during async I/O
+stays dirty until a subsequent flush, so an older pending flush cannot erase it.
+Python keeps its existing per-instance thread lock around each flush.
+
+The Node regression suite can run an actual mixed-language process test with
+`PATCHLING_PY_SOURCE` pointing to a Python source checkout and
+`PATCHLING_PYTHON` selecting its Python interpreter. Without that optional
+checkout, independent-process coverage still runs and the mixed-language check
+is explicitly skipped; tests never fetch a repository or call an inference API.

README.md

@@ -63,6 +63,22 @@ on 2026-10-13 (re-checked 2026-10-21, when the default `deepseek/deepseek-v4.1-f
 The decision demo requests MiMo's supported `none` reasoning effort.
 Library environment defaults below are independent of the demo choices.
 
+The homepage snapshots its mode and models for each run; changes made while it
+is working apply to the next run. Model catalogs retry after temporary failures
+and reload when the saved credential changes.
+
+OpenAI-compatible streaming completions must include `[DONE]` or a successful terminal finish
+reason. A truncated stream, `finish_reason: length`, or `content_filter` rejects
+with an `IncompleteCompletionError`; length/filter failures also reject on the
+non-streaming path. Tokens may already have reached `onToken`, so publish file
+changes only after the completion promise resolves. Partial streams are not
+automatically retried, avoiding replay of a possibly billed request.
+
+Disk-backed relevance caches retain version-1 JSON and coordinate updated
+Node/Python writers. See [cache coordination and compatibility](CACHE_PROTOCOL.md)
+for lock recovery, timeout behavior, and the requirement to upgrade concurrent
+writers together. Browser and memory-only caches keep their existing behavior.
+
 ## Configuration
 
 Configuration comes from environment variables (Node) or `setEnv(...)` overrides (browser). OAuth sign-in sets the API key override for you.

landing/index.html

@@ -728,11 +728,12 @@ <h3>Python</h3>
 </div>
 
 <script type="module">
-import { buildEnvironment, generateDiff, smartapply, parseDiffPerFile }
+import { buildEnvironment, generateDiff, smartapply, parseDiffPerFile, callLlmForApply }
   from "https://esm.sh/patchling";
 import { registerClient, beginSignIn, completeSignIn }
   from "https://esm.sh/patchling/oauth";
 import { filterRelevant, DEFAULT_FILTER_MODEL } from "./relevance.js";
+import { callLlm } from "./lib/llm.js";
 
 const $ = (id) => document.getElementById(id);
 const DEFAULT_MODEL = "xiaomi/mimo-v2.6-flash";
@@ -981,14 +982,18 @@ <h1>${esc(title)}</h1>${cards}`;
 
 /* ---------- key + model storage ---------- */
 const getKey = () => localStorage.getItem("pl_key");
-const setKey = (k) => (k ? localStorage.setItem("pl_key", k) : localStorage.removeItem("pl_key"));
+const setKey = (k) => {
+  k ? localStorage.setItem("pl_key", k) : localStorage.removeItem("pl_key");
+  invalidateModels();
+};
 // Replace previous factory picks; preserve visitors' custom model choices.
 const savedModel = localStorage.getItem("pl_model");
 let MODEL = savedModel && !["zai-org/glm-5.2:thinking", "xiaomi/mimo-v2.5-pro-ultraspeed"].includes(savedModel)
   ? savedModel : DEFAULT_MODEL;
 
 /* ---------- model picker (same pattern as the live examples) ---------- */
 let allModels = null, topOrder = new Map();
+let modelsLoaded = false, modelsKey = null, modelsRequest = null;
 const LABS = [
   ["OpenAI", /^openai\/(gpt-[0-9]|o[1-9])/i],
   ["Anthropic", /claude-(opus|sonnet)/i],
@@ -1028,19 +1033,55 @@ <h1>${esc(title)}</h1>${cards}`;
   $("modelprice").textContent = m.prompt != null ? `(${px(m.prompt)} / ${px(m.completion)} per 1M)` : "";
 }
 async function loadModels() {
-  if (allModels) return allModels;
-  let arr = [];
-  try {
-    const headers = {}; const k = getKey(); if (k) headers.Authorization = `Bearer ${k}`;
-    arr = (await (await fetch("https://nano-gpt.com/api/v1/models?detailed=true", { headers })).json()).data || [];
-  } catch { /* offline / blocked — the exact-id path still works */ }
-  allModels = arr.map((m) => ({ id: m.id, prompt: m.pricing?.prompt, completion: m.pricing?.completion, created: m.created })).filter((m) => m.id);
-  if (!allModels.length) allModels = [{ id: DEFAULT_MODEL }];
-  computeTop();
-  allModels.sort((a, b) => (rank(a.id) - rank(b.id)) || a.id.localeCompare(b.id));
+  const key = getKey() || "";
+  if (modelsLoaded && modelsKey === key) return allModels;
+  if (modelsRequest && modelsRequest.key === key) return modelsRequest.promise;
+  const request = { key, promise: null };
+  modelsRequest = request;
+  request.promise = (async () => {
+    let next, loaded = false;
+    try {
+      const headers = {}; if (key) headers.Authorization = `Bearer ${key}`;
+      const response = await fetch("https://nano-gpt.com/api/v1/models?detailed=true", { headers });
+      if (!response.ok) throw new Error("Model catalog request failed");
+      const data = (await response.json()).data;
+      if (!Array.isArray(data)) throw new Error("Invalid model catalog");
+      next = data.filter((m) => m && typeof m.id === "string" && m.id.trim()).map((m) => ({
+        id: m.id, prompt: m.pricing?.prompt, completion: m.pricing?.completion, created: m.created,
+      }));
+      if (!next.length) throw new Error("Empty model catalog");
+      loaded = true;
+    } catch { next = [{ id: DEFAULT_MODEL }]; } // exact-id entry remains available; the next open retries
+    try {
+      // A sign-in/out or newer request must not inherit an older account's catalog/prices.
+      if (modelsRequest !== request || (getKey() || "") !== key) return allModels || [{ id: DEFAULT_MODEL }];
+      allModels = next;
+      modelsLoaded = loaded;
+      modelsKey = key;
+      computeTop();
+      allModels.sort((a, b) => (rank(a.id) - rank(b.id)) || a.id.localeCompare(b.id));
+      renderModelRow();
+      renderFilterModelRow();
+      return allModels;
+    } finally {
+      if (modelsRequest === request) modelsRequest = null;
+    }
+  })();
+  return request.promise;
+}
+function invalidateModels() {
+  allModels = null;
+  modelsLoaded = false;
+  modelsKey = null;
+  modelsRequest = null;
+  topOrder = new Map();
+  for (const prefix of ["model", "filtermodel"]) {
+    $(prefix + "picker").style.display = "none";
+    $(prefix + "list").innerHTML = "";
+    $(prefix + "count").textContent = "";
+  }
   renderModelRow();
   renderFilterModelRow();
-  return allModels;
 }
 function modelButton(m) {
   const b = document.createElement("button");
@@ -1430,7 +1471,7 @@ <h1>${esc(title)}</h1>${cards}`;
 // The punchline, told only after it happened: which files git apply would
 // have bounced, and what smartapply did about it. Built from the actual run
 // outcome, so it's just as true in real mode as in the canned demo.
-function renderVerdict(perFile, inputFiles, updated, llmResolved, decisions, price) {
+function renderVerdict(perFile, inputFiles, updated, llmResolved, decisions, price, runMode = mode, scenario = SCENARIOS[scenarioKey]) {
   const v = $("verdict");
   v.innerHTML = "";
   const repaired = [...llmResolved].filter((p) => p in updated);
@@ -1462,10 +1503,10 @@ <h1>${esc(title)}</h1>${cards}`;
 
   if (decisions) line("ok", formatClassifierCost(decisions, price));
 
-  if (mode === "demo" && SCENARIOS[scenarioKey].reveal) {
+  if (runMode === "demo" && scenario.reveal) {
     const why = document.createElement("p");
     why.className = "vwhy";
-    why.textContent = SCENARIOS[scenarioKey].reveal;
+    why.textContent = scenario.reveal;
     v.appendChild(why);
   }
   v.style.display = "block";
@@ -1481,11 +1522,15 @@ <h1>${esc(title)}</h1>${cards}`;
 /* ---------- the run ---------- */
 async function run() {
   const scn = SCENARIOS[scenarioKey];
+  // Settings changed during a run apply to the next run, never halfway through this one.
+  const runMode = mode;
+  const runDiffModel = MODEL;
+  const runApplyModel = APPLY_MODEL;
 
   // Demo mode replays a canned response, so it must run against the canned
   // files and goal. If the visitor edited either, reset (and say so) rather
   // than lie about what produced the output.
-  if (mode === "demo") {
+  if (runMode === "demo") {
     const pristine = JSON.stringify(files) === JSON.stringify(scn.files)
       && $("goal").value.trim() === scn.goal;
     files = { ...scn.files };
@@ -1528,18 +1573,18 @@ <h1>${esc(title)}</h1>${cards}`;
     let decisions = null;
     if (filtering) {
       $("decisionwrap").style.display = "block";
-      $("filtersrc").textContent = mode === "demo" ? "· pre-recorded decisions" : "· live, cheap model";
+      $("filtersrc").textContent = runMode === "demo" ? "· pre-recorded decisions" : "· live, cheap model";
       decisions = await collectDecisions(inputFiles, goal, runId, runModel);
       if (!isCurrentRun(runId)) return;
     }
 
     if (!isCurrentRun(runId)) return;
     $("phase").textContent = "generating diff…";
-    $("diffsrc").textContent = mode === "demo" ? "· pre-recorded reply, real engine" : "· live from NanoGPT";
+    $("diffsrc").textContent = runMode === "demo" ? "· pre-recorded reply, real engine" : "· live from NanoGPT";
     $("diffwrap").style.display = "block";
-    const genOpts = mode === "demo"
+    const genOpts = runMode === "demo"
       ? { callLlm: makeFakeCallLlm(scn), onToken: onDiffToken }
-      : { apiKey: key, model: MODEL, onToken: onDiffToken };
+      : { apiKey: key, model: runDiffModel, callLlm, onToken: onDiffToken };
 
     const diff = await generateDiff(buildEnvironment(filesForDiff(inputFiles, decisions)), goal, genOpts);
     if (!isCurrentRun(runId)) return;
@@ -1579,9 +1624,13 @@ <h1>${esc(title)}</h1>${cards}`;
       pre.scrollTop = pre.scrollHeight;
     };
 
-    const applyOpts = mode === "demo"
+    const applyOpts = runMode === "demo"
       ? { callLlmForApply: makeFakeApply(scn), onToken: onApplyToken }
-      : { apiKey: key, model: APPLY_MODEL, onToken: onApplyToken };
+      : {
+        apiKey: key, model: runApplyModel, onToken: onApplyToken,
+        callLlmForApply: (path, original, patch, model, opts) =>
+          callLlmForApply(path, original, patch, model, { ...opts, callLlm }),
+      };
 
     const updated = await smartapply(diff, inputFiles, applyOpts);
     if (!isCurrentRun(runId)) return;
@@ -1611,14 +1660,14 @@ <h1>${esc(title)}</h1>${cards}`;
     renderEditors();
     renderPreview();
 
-    renderVerdict(perFile, inputFiles, updated, llmResolved, decisions, runPrice);
+    renderVerdict(perFile, inputFiles, updated, llmResolved, decisions, runPrice, runMode, scn);
 
     const cost = decisions ? formatClassifierCost(decisions, runPrice) : "";
     $("phase").textContent = cost
       ? "done ✓ — " + cost + " — the preview and files above are the new map"
       : "done ✓ — the preview and files above are the new map";
     document.querySelector("#status .spin").style.display = "none";
-    if (mode === "demo") $("upsell").style.display = "block";
+    if (runMode === "demo") $("upsell").style.display = "block";
   } catch (e) {
     if (!isCurrentRun(runId)) return;
     $("phase").textContent = "error";