Repository navigation
translate: c0mpute first, Anthropic fallback, off the request path - #273
Merged
Merged
Conversation
…uest path Translations go through @profullstack/ai-providers 0.2.0: c0mpute streams the answer (provider.stream) in a tagged format a 9B model copies reliably; the existing Anthropic structured-output call stays as the paid tier. c0mpute takes minutes under load, so ensureTranslation runs the call in the background, waits up to TRANSLATE_WAIT_MS (8 s) and otherwise answers pending; the page says so and the next view is a cache hit. A repeat ask while running is neither a second call nor a second charge. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fleet rollout (ai-providers ROLLOUT.md wave 3: rssamplifier translate needs streaming).
packages/translateroutes both calls through@profullstack/ai-providers@0.2.0(pinned exact) withgetAi().run: c0mpute first, streamed throughprovider.stream(the 0.2.0 streaming path); the existing Anthropicclaude-haiku-4-5structured-output calls are kept verbatim as the paid tier. OPENAI, if ever set, gets the c0mpute prompt.<title>/<summary>/<source_language>/<body>tags rather than JSON (a 9B model escapes a page of HTML into JSON badly).readTaggedparses it; anything unusable falls to the next provider, then to the original as today.truncated). The paid path keeps 60k.ensureTranslationstarts the job in the background, waitsTRANSLATE_WAIT_MS(default 8 s), else returnspending: true. The read page says the translation is running;/api/translateJSON carriespending. Asking again while it runs is not a second call or charge. Max 24 in flight per process; past that the original shows, uncharged.modelis nowc0mpute:<model>for peer answers.Dead-key behaviour unchanged: Anthropic billing failure -> null -> original shown.
Tests: 5 new (quick answer shown + cached with c0mpute model, background pending + no double charge + later cache hit, untagged answer shows original, tag parser). All workspace suites pass locally against Postgres 17.
🤖 Generated with Claude Code