Skip to content

Keep scripted usage and copy FakeModel responses - #375

Open
anbuzin wants to merge 2 commits into
mainfrom
fix-fake-model-responses
Open

anbuzin wants to merge 2 commits into
mainfrom
fix-fake-model-responses

Conversation

@anbuzin

@anbuzin anbuzin commented Nov 20, 2026 •

Copy link
Copy Markdown
Collaborator

FakeModel dropped scripted token counts from stream.usage, and editing a generated response could change the script. Completed streams now report the scripted usage, and responses are deep copies. The testing reference explains what the fake does and doesn't simulate.

from ai.types.usage import Usage
prompt = ai.user_message("Say hello")
scripted = ai.assistant_message("Hello")
scripted.usage = Usage(input_tokens=3, output_tokens=5)
model = ai.testing.FakeModel([prompt, scripted])
async with ai.stream(model, [prompt]) as stream:
    async for _ in stream: pass
usage = stream.usage  # before: None; after: 3 input tokens, 5 output tokens
response = await ai.generate(model, [prompt])
print(response is scripted)  # before: True; after: False

Carry scripted usage through stream completion and return deep copies of
scripted responses so callers cannot mutate the fake's script through
them.

Keep direct provider stream selection eager. Add regression tests and
document usage accounting and the limits of scripted model tests.
@anbuzin anbuzin added the fix Bug fix label Nov 20, 2026
@vercel

vercel Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
ai-python Ready Ready Preview, v0 Nov 20, 2026 12:09am UTC

This branch was successfully deployed

1 active deployment
Preview — 1f46a900 Deployed Nov 20, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fix Bug fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant