Cache stability: static memory tools, guardrail tail placement, Anthropic cache audit, tokenizer adapter, skill load warnings - #49
Open
MyPrototypeWhat wants to merge 1 commit into
Conversation
…cement, Anthropic cache audit, tokenizer adapter, skill load warnings
Keeps the prompt prefix byte-stable across compiles:
- modify_memory no longer embeds a live-key enum and is always emitted — key
mutations previously rewrote the tool schema and invalidated the ENTIRE
provider prompt cache (tools are the topmost prefix segment); payload.tools
is now deep-equal and reference-stable across compiles
- withGuardrails({ placement: 'last_user' }): enforce-XML rides the last-user
tail injection — on Anthropic all system-role messages hoist into the top
system prefix, so the 'system' placement busts downstream breakpoints
- auditAnthropicCachePlacement(payload) + ChefConfig.cacheAudit: deterministic
Anthropic-only audit for volatile content inside the cached prefix, with a
per-issue suggested fix (warn-once per instance)
- createTokenizerAdapter(encode): zero-dependency glue encoding the
fields-counted convention (incl. tool-call arguments) for any external
encoder; 648 bytes in dist, no new dependencies
- SkillLoadResult.warnings: mechanical quality checks on dir loads (thin
description, oversized instructions, malformed allowedTools, dangling
relative links) — never block loading
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
First follow-up PR from the section-④ discussion (scope agreed item-by-item in session). Theme: keep the prompt prefix byte-stable across compiles — prompted by the observation that tool-list changes and system-prompt injections were invalidating provider prompt caches.
Changes (
@context-chef/coreminor → 4.1.0)modify_memorypreviously embedded anenumof live memory keys and only appeared once keys existed: every key mutation rewrote the tool schema, and since tools are the topmost prefix segment, that invalidated the entire prompt cache. Now both tools are always emitted with static schemas (same object references every compile); current keys are still surfaced via the injected memory block, and unknown-key dispatches keep failing safely atupdateMemory/deleteMemory.withGuardrails({ placement: 'last_user' })— on Anthropic, all system-role messages hoist into the top-levelsystemprefix, so the default guardrail delivery busts downstream cache breakpoints on every options change. The new placement routes the enforce-XML text through the last-user-message tail injection (the<EPHEMERAL_MESSAGE>wording was originally designed for user-stream delivery).prefillunchanged.auditAnthropicCachePlacement(payload)+ChefConfig.cacheAudit— deterministic, Anthropic-only audit (the one provider with explicit breakpoints; zero heuristics): flags volatile content (memory data / dynamic state / implicit context / guardrail instructions) at or before the lastcache_controlbreakpoint, with a per-issue suggested fix.cacheAudit: truewarns each distinct issue once per instance.createTokenizerAdapter(encode)— zero-dependency glue turning any(text) => tokensencoder into a correctJanitorConfig.tokenizer, encoding the fields-counted convention once (content + thinking + redacted data + tool-call names/arguments). Footprint: 648 bytes in dist, zero new dependencies (core still depends only onzod).SkillLoadResult.warnings: mechanical checks on dir loads (thin description, oversized instructions, malformedallowedTools, dangling relative links); never blocks a load. (Replaces the earlier standalone skill-lint idea per discussion — creation-time quality is covered by skill-creator tools; the npmskill-lintpackage is a security scanner, different niche.)Verification
pnpm lint✓ · build ✓ · all typechecks +typecheck:compat+ examples ✓ · 1112 tests (core 866 [+34], ai-sdk-middleware 132, tanstack-ai 114) ✓ · README updated (en + zh, five sections each).Not in this PR (next up per discussion): B — handoff (mechanical + LLM tiers) + branch/fold (outcome-oriented fold prompt as the core deliverable); C — FileMemory (zero-injection,
memory_20250818-compatible commands over VFS); backlog —prepareStepdurable in-loop compaction + per-stepactiveToolsnarrowing.