Skip to content

fix(agent): trigger checkpoints on the full context size - #134

Merged
wolfy-j merged 5 commits into
masterfrom
fix/checkpoint-context-tokens
Nov 4, 2026
Merged

wolfy-j merged 5 commits into
masterfrom
fix/checkpoint-context-tokens

Conversation

@wolfy-j

@wolfy-j wolfy-j commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Purpose

The agent node's checkpoint threshold compared the last action's prompt_tokens. With prompt caching (the history_tail marker from #90), prompt_tokens counts only the uncached suffix of the prompt, so a long conversation whose prefix is cached stays far below the threshold and never checkpoints, until the model call fails on the context window. The session host moved to context_tokens for the same reason.

Change

  • The threshold compares tokens.context_tokens (uncached input plus cache reads and writes), which the llm library sets on every normalized response since 0.5.1 (the module already requires >= 0.5.3).
  • The checkpoint binding refs, the function fallback payload, and the checkpoint marker metadata carry context_tokens / checkpoint_at_context_tokens next to the unchanged prompt_tokens / checkpoint_at_prompt_tokens, so custom summarizers keep their inputs.
  • The default summarizer prompt states context_tokens next to prompt_tokens.
  • The recovery stub model can report cache_read_tokens (scenario input) to exercise cached prompts; the stub summarizer records the context_tokens it receives.

Tests

  • New threshold counts cached prompt tokens: 200 uncached + 5000 cached tokens against a 1000-token threshold must checkpoint and record 5200 context / 200 prompt tokens. It fails on master (no checkpoint).
  • The function-fallback payload's context_tokens is asserted (5200 in the cached case, 500 in the uncached threshold case). The binding path builds refs from the same values; the suite has no successful checkpoint binding stub to observe it.
  • Full suite (make test-sqlite) passes, lint clean.

The token threshold compared prompt_tokens, which under prompt caching
counts only the uncached suffix of the prompt, so long cached
conversations crossed the threshold without checkpointing. The trigger
uses context_tokens (uncached input plus cache reads and writes), as the
session host does; checkpoint payloads and markers carry both values.
@wolfy-j
wolfy-j requested a review from skhaz November 3, 2026 18:01
The stub summarizer records the context_tokens it receives and the
checkpoint tests assert it next to prompt_tokens. The default summarizer
prompt states the context size, and the stub model takes cached prompt
tokens as an argument.
@wolfy-j

wolfy-j commented Nov 4, 2026

Copy link
Copy Markdown
Contributor Author

Hold until #136 is in: this makes checkpoints fire on cached agents, and on master a checkpoint leaves tool results without their calls. #136 fixes that.

@wolfy-j
wolfy-j merged commit 86a9da5 into master Nov 4, 2026
8 checks passed
@wolfy-j
wolfy-j deleted the fix/checkpoint-context-tokens branch November 4, 2026 20:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants