Skip to content

fix(anthropic): keep full multi-message context in input.value - #247

Open
jjcav84 wants to merge 1 commit into
future-agi:devfrom
jjcav84:fix/anthropic-input-value-context
Open

jjcav84 wants to merge 1 commit into
future-agi:devfrom
jjcav84:fix/anthropic-input-value-context

Conversation

@jjcav84

@jjcav84 jjcav84 commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do?

Third instance of the INPUT_VALUE first-message-only defect family from #151 (openai → #245, litellm → #246).

All five anthropic wrapper chains (Completions ×2, Messages ×2, TokenCount) built their attribute dict ending with _get_query(query) where query = eval_input[0] — the first message's text only. It overwrote the joined eval_input produced just earlier in the same chain(...), so multi-turn conversations lost every message after the first in input.value.

Fix, matching the sibling handlers (openai/litellm/dspy/groq all end on the joined text):

  • Drop _get_query(query) from the five chains, delete the helper, and remove the query key from _extract_image_data (no other consumers).
  • Swap _get_eval_input to last in each chain so the joined text wins INPUT_VALUE — previously _get_raw_input (full-kwargs JSON) would have won, which preserves context but diverges from the flat-text convention evaluators read.

Why?

Same reported bug as #151: trace correctness for multi-message inputs.

How was it tested?

  • Unit tests added — TestInputValueMultiMessage in python/tests/test_framework_anthropic.py: joined text preserves every message, image placeholders embed correctly, and a direct dict(chain(raw_input, eval_input)) ordering assertion pins the winning write
  • pytest tests/test_framework_anthropic.py::TestInputValueMultiMessage — 3/3 pass
  • Baseline compared on clean main: 7 pre-existing failures (anthropic.resources.completions removed in newer SDKs — instrumentor version drift, unrelated); same 7 fail before and after, zero regressions

Checklist

  • Branch is off main
  • Conventional Commits (fix(anthropic): …)
  • No TODOs or commented-out code
  • No secrets in the diff

Added in second commit: the instrumentor imported anthropic.resources.completions unconditionally, but the current SDK removed that module — every test failed at _instrument. The import is now guarded (nothing to instrument when the resource is absent) and the two legacy-resource tests are skipif-gated. pytest tests/test_framework_anthropic.py: 10 pass, 1 skip (was 7 failed / 4 passed on clean main).

@jjcav84

jjcav84 commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

Bump — fixes multi-message context truncation in input.value. Mergeable.

Third instance of the future-agi#151 defect family (openai future-agi#245, litellm future-agi#246):
each wrapped call built its attribute dict with _get_query last, where
query = eval_input[0] — the first message's text — overwriting the
joined eval_input that carried every message.

Remove the query write, its dead _get_query helper, and the "query" key
from _extract_image_data; move _get_eval_input to the end of the chain
so the joined multi-message text wins INPUT_VALUE, matching the openai,
litellm, dspy, and groq handlers.

Generated with Devin
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@jjcav84
jjcav84 force-pushed the fix/anthropic-input-value-context branch from a803a72 to 9660687 Compare November 6, 2026 00:30
@jjcav84
jjcav84 changed the base branch from main to dev November 6, 2026 00:30

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant