Replaying a TanStack AI stream with the model server switched off

[xizhuomengcontin]

Verified against @tanstack/ai 0.54.0 today: a streaming chat() run can be recorded from outside the process and replayed with no model server present.

Disclosure: I work on the recording tool (OrcaReplay, Apache-2.0, no paid tier).

The run

const stream = chat({
  adapter: openaiText('qwen3.5:0.8b'),   // origin comes from OPENAI_BASE_URL, not from code
  messages: [{ role: 'user', content: 'Reply with exactly: TANSTACK' }],
});

$ orca record generic-openai -- node agent.mjs
tanstack said: …
info recorded run=run_db791673cfbf events=6 blobs=1 exit=0

$ taskkill /F /IM ollama.exe          # model server gone
$ orca replay last
tanstack said: …
info replay.done reused=1/1 exact=1 divergences=0 unmatched=0 exit=0

The streaming part is the point

This is why I bothered posting rather than filing it away: the recording keeps the raw SSE frames, and replay hands them back frame by frame rather than as one finished blob. So chat()'s own stream parsing, delta accumulation and event sequencing run for real on the replay.

That matters for a streaming-first library. A fixture that returns a completed string would exercise none of that — and stream handling is exactly where this kind of code breaks.

What it is useful for

Re-running a chat while you change the adapter, the message shaping, or your UI's consumption of the stream — the model's tokens stay byte-identical, so any difference is yours. Also CI without provider keys or spend.

Scope

egress=blocked on replay means model-provider egress. The rest of your app still runs for real; it is not a sandbox. And it records the HTTP going to the model, not TanStack AI's internals.

Happy to answer anything about the capture.