feat: per-Turn cache usage accumulators, and sign-in to a Bridgic cloud account - #37
Draft
NiceCode666 wants to merge 12 commits into
Draft
feat: per-Turn cache usage accumulators, and sign-in to a Bridgic cloud account#37NiceCode666 wants to merge 12 commits into
NiceCode666 wants to merge 12 commits into
Conversation
…er Turn `cached_input_tokens` only ever held the latest round: `previous` was read for input and output totals but never for cache, so the last call overwrote the whole history. With per-round usage kept nowhere else, a Turn's cache hit rate could not be reconstructed from `session_turns.context_usage` at all. Accumulate both cache legs across the Turn, and split cache writes out of the input fold so Anthropic's separately-priced write is visible on its own. Cache reporting is three-state, not two. Providers differ: some omit the cache fields entirely on a cold call rather than reporting zero, and some report only intermittently. Treating "not reported" as zero underestimates the hit rate; skipping those rounds overestimates it. `cached_reported_rounds` against `cached_total_rounds` keeps the two apart, so a hit rate can also state how much of its sample it actually observed. New fields carry defaults, so stored rows written before this change deserialize unchanged — no column backfill needed.
Adds an Account tab to Settings. Signing in exchanges email and password for a cloud access token, then stores it as an ordinary provider channel pointed at the gateway. That last part is the whole design. The platform plan is not a new protocol or a new code path: the gateway speaks the OpenAI wire, `provider_id` has accepted any user-chosen slug since the catalog stopped being a validation gate, and `first_enabled_other` already treats a row with an api_key as usable. So the model picker, `build_llm` dispatch, and the chat path need no changes at all — sign-in only changes where the key comes from. Model context windows arrive with the model list, so occupancy still renders and a window can change server-side without a client release. Signing out deletes the channel rather than just forgetting it in memory: the token IS the credential, so leaving the row behind would keep the account spendable from this machine after the user asked to be signed out.
Runs the real `addProviderAtom` against a mocked fetch that answers both the gateway and the daemon, so the assertion is on the credential request the daemon actually receives rather than on a stubbed atom. Also pins the case where a model reports no context window: it must be left out of model_limits entirely, since a zero would render as "0 tokens of context".
The sign-in form showed "Failed to fetch" — fetch's own message, in English, in a Chinese UI, naming nothing the user could do. fetch rejects with that one opaque string for every transport failure, and the browser withholds the real reason by design, so a refused connection, a DNS miss and a blocked CORS preflight are indistinguishable at that point. Passing it through tells the user nothing; it is replaced with the one thing they can act on. Also adds a reveal toggle to the password field. Masking hides typos as well as it hides the password: a mis-pasted character stays invisible until sign-in fails with a message that cannot say which character was wrong.
The Account tab previously showed whatever the balance was at sign-in and never moved. Opening it now loads the current figure, and a button next to it refreshes on demand. Opening the tab also recovers the session, which is the part that was actually broken. The token lives in memory, so restarting the app lost it — while the credential itself was still on the User row and platform models kept working. The tab therefore showed a sign-in form to someone who was signed in, and a refresh would have had no token to refresh with. It now falls back to reading the stored provider credential, whose api_key IS the token. A rejected token drops the session rather than leaving a stale balance on screen: it means the account was suspended or the password changed elsewhere, and that balance is no longer ours to display. The automatic load is quiet and the manual one is not, so opening the tab does not flash a spinner over what is usually an instant local read.
The Account tab showed a bare credit figure. Credits are an invented unit, so that number is unreadable without the rate — and the rate appeared nowhere in the app. The yuan equivalent now sits under the balance. The rate is read from the gateway rather than assumed, so it stays correct after operations changes it. A zero rate renders nothing at all: showing no conversion beats showing a wrong one.
… rejects /me/models was read once, at sign-in, and written into the provider row. What an account may buy is the operator's to change and nothing tells this machine when they do, so the local list drifted in both directions: it kept offering models that had been withdrawn — they 404 with a reason the user never sees — and hid ones put on sale since. cloudRefreshAtom now re-reads it, in its own try, because the balance is what that atom exists to load. A refresh writes only the list. The backend upsert overwrites just the fields it is handed, so there is no reason to move the token again. On 401 the credential row is deleted too. Clearing the session in memory left a dead channel in the model picker with its stale list attached; a rejected token is what being signed out means, so it now reaches the same end state. Which failures count as 401 is read off the status rather than matched in the message. The message is the gateway's own `detail`, written for a human and rewritten whenever the wording is, and deleting a credential on a substring match is not a thing to be one sentence away from.
The balance rendered as 9969.1555, and the call meant to format it was not broken so much as inert: `creditsBalance` is typed `number` and holds a string. The gateway sends credits as an exact decimal string on purpose — a JSON number is a double by the time this parses it, and the figure is money — so `toLocaleString` was String's, which returns its own value untouched. The fixtures agreed with the wrong type, which is why no test caught it. They now send what the gateway sends, and the conversion happens once, at the boundary, so the declared `number` is true. Nothing on this side does arithmetic that has to be exact; carrying the string further would only mean every consumer converting it. `wholeCredits` rounds down, never up. This is display and the stored balance keeps all of its places, so the one thing a shown balance must not do is claim more money than the account has.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft. Two related but separable changes share this branch; see Scope below
before reviewing.
What is in it
1. Cache usage accumulators (
src/, ~36 lines + tests)_usage_valuesnow also returns cache-write tokens, andContextUsageSnapshotcarries four new running totals per Turn:
cached_input_tokens_total,cache_write_tokens_total,cached_reported_rounds,cached_total_rounds.The last two exist because cache reporting is three-state, not two: some
providers omit the fields entirely on a cold call rather than reporting zero.
Counting reported rounds against total rounds keeps "no cache" apart from "no
report", so a hit rate computed later can say how much of its sample it
actually saw. Folding the two together biases the rate in whichever direction
the reader guessed.
2. Cloud account sign-in (
desktop/, ~900 lines)Sign in or register against the gateway from Settings → Account, and spend
platform credits from the desktop app.
The platform is deliberately not a new kind of provider. Signing in writes
an ordinary
provider_credentialsrow whose api_key happens to be the cloudaccess token and whose base_url points at the gateway, which speaks the OpenAI
wire — so
build_llm, the model picker and the whole chat path work on ituntouched. The only thing sign-in adds is where the key comes from.
Also here:
/me/modelsis re-read on every refresh rather than frozen atsign-in (what an account may buy is the operator's to change, and nothing tells
this machine when they do); a 401 drops the credential row as well as the
in-memory session, keyed off the status rather than a substring of the
gateway's human-readable
detail; and the balance is converted from thegateway's exact decimal string exactly once, at the boundary.
Verification
Run locally against this branch, all green:
pytest tests/agent/cognitive/test_usage_accumulation.py tests/agent/cognitive/test_thinking.py→ 10 passedbun run typecheck(shared · ui · electron · scripts) → cleanbun run lint→ cleanbun test→ 1352 passed, 6 skipped, 0 failed (1358 across 161 files)Known blockers — why this is a draft
CLOUD_BASE_URLis hardcoded tohttp://127.0.0.1:8787. Shipping asis points every user's sign-in at their own machine. Needs the production
host, and HTTPS, before this can merge.
filing_numberis fetched and dropped./me/modelsreturns it andModelPayloadparses it, but nothing renders it. If the filing number hasto be published client-side for compliance, that surface is still missing.
CloudModel(atoms/cloud.ts:65) is unused.syncModelsreadsModelPayloadinstead. Delete it, or use it.Scope
The branch name describes change 1; change 2 is ~95% of the diff. Splitting
them is straightforward if reviewers would rather take them separately —
say so and I will.