Anthropic prompt cache: turn after a 5–60 min user pause rewrites the whole prefix (idle gap never selects the 1h TTL) #1898

graff: 0.302.74.65 (latest: v0.302.74.65)
Harness: 0.2.190
Surface: Harness (ACP)
Platform: macOS arm64

Symptom

In an interactive chat on an Anthropic model (claude-opus-5-5 via the Codegraff gateway), the first request of a user turn reads nothing from the prompt cache whenever the user replied more than ~5 minutes after the previous turn ended. The trace reports cache_miss: "first_request" with prefix_changed: false, so this is a TTL expiry of an unchanged prefix, not a prefix bust. The prompt and toolset hashes (prompt_sha, toolset_sha) are identical across the turns.

From one long chat (11 traced runs, ~180 model calls, context growing to ~270k tokens):

The 4 cold openers wrote 582k tokens, 31% of all cache writes in the chat. In conversational use, where people reply a few minutes to an hour later, this is the common case.

Repro

1. Start a chat on an Anthropic model and build up 100k+ tokens of context.
2. Wait 10 minutes, then send a short follow-up.
3. In .graff/traces/<run>.jsonl, the first usage event has cache_read_tokens: 0, cache_write_tokens ≈ the whole context, and cache_miss: "first_request".

Root cause

This is the documented policy, not a bug. v0.302.74.65:src/cache_ttl.zig:1-14 asks for the 1-hour TTL only after a slow call inside a turn (#1320). ADR 0220 (docs/adr/0220-claude-prompt-cache-forks-and-pricing.md; src/cache_ttl.zig:10-14, test at :138) has the turn-opening request inherit the last in-turn gap and ignore the user's idle time. So a turn made of fast calls ends on 5-minute breakpoints, and any user pause longer than 5 minutes forces a full prefix rewrite at 1.25× (src/pricing_cache.zig:6).

Fix (proposal)

Use the user's idle gaps as a signal too. For example: if the previous user idle gap fell between 5 and 60 minutes, or the session is an interactive ACP chat above some context size (say 64k), mark the requests in this turn with the 1-hour TTL, so the prefix survives the next pause.

Rough economics for a prefix of P tokens, gap between 5 and 60 minutes:

• Today: about 1.25·P per pause (a full rewrite).
• 1-hour TTL: about 0.1·P per pause (a read), plus the extra cost of writing new tokens at 2× instead of 1.25× (src/pricing_cache.zig:8), and possibly a one-time 2× write when the prefix first moves from a 5-minute to a 1-hour entry.

So the change pays for itself after the first pause longer than 5 minutes. A user who never comes back costs only the 2× premium on the last turn's writes, or a one-time prefix upgrade.

Open question: whether reading a 5-minute entry with a 1-hour breakpoint refreshes it or re-writes it. That decides whether the one-time 2·P upgrade happens.