Skip to content

fix(openai): keep full multi-message context in input.value - #245

Closed
jjcav84 wants to merge 1 commit into
future-agi:devfrom
jjcav84:fix/openai-input-value-context
Closed

jjcav84 wants to merge 1 commit into
future-agi:devfrom
jjcav84:fix/openai-input-value-context

Conversation

@jjcav84

@jjcav84 jjcav84 commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Fixes INPUT_VALUE silently dropping all but the first message for conversational inputs (#151).

_process_input_data wrote INPUT_VALUE three times:

  1. JSON dump of the filtered structured messages
  2. Joined text of every message's content (eval_input)
  3. eval_input[0] — the first message's text only ← the bug

Write 3 is dead logic (its if is implied by write 2's) and destroyed the multi-message context write 2 produced. Removed so the joined string survives; image parts still surface as {{input_images.N}} placeholders with INPUT_IMAGES set alongside.

process([{"role": "user", "content": "a"},
         {"role": "assistant", "content": "b"},
         {"role": "user", "content": "c"}], span)
# before: INPUT_VALUE == "a"
# after:  INPUT_VALUE == "a \n b \n c"

Sibling note for reviewers: traceai_litellm carries the identical pattern — _process_messages returns query = eval_input[0] and _instrument_func_type_completion writes it last, overwriting the joined eval_input. Same first-message-only loss. I kept this PR scoped to the reported path; happy to follow up there if you'd like it fixed identically.

Why?

Closes #151 — trace correctness for multi-message inputs: every message after the first was invisible in input.value.

How was it tested?

  • Unit tests added — TestProcessInputData in python/tests/test_framework_openai.py: multi-message preservation, multimodal text+image placeholders, single message, non-list input
  • pytest tests/test_framework_openai.py::TestProcessInputData — 4/4 pass
  • Pre-existing suite failures unchanged: TestOpenAIFramework fails identically on clean main — its httpx.Client.send mock no longer intercepts under openai 3.x (real API call → 401); unrelated to this diff

Checklist

  • Branch is off main
  • Conventional Commits (fix(openai): …)
  • No TODOs or commented-out code
  • No secrets in the diff

@jjcav84

jjcav84 commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

Bump — fixes multi-message context truncation in input.value. Mergeable.

_process_input_data ended with a redundant third write that reduced
INPUT_VALUE to eval_input[0] — the first message's text only — after
already writing the joined text of every message. Multi-turn and
multi-part inputs lost all but the first content piece (future-agi#151).

Drop the overwrite so the joined eval_input string survives; image
parts still surface as {{input_images.N}} placeholders alongside
INPUT_IMAGES.

Generated with Devin
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@jjcav84
jjcav84 force-pushed the fix/openai-input-value-context branch from b935b46 to 6648eac Compare November 6, 2026 00:28
@jjcav84
jjcav84 changed the base branch from main to dev November 6, 2026 00:28
jjcav84 added a commit to jjcav84/traceAI that referenced this pull request Nov 6, 2026
Third instance of the future-agi#151 defect family (openai future-agi#245, litellm future-agi#246):
each wrapped call built its attribute dict with _get_query last, where
query = eval_input[0] — the first message's text — overwriting the
joined eval_input that carried every message.

Remove the query write, its dead _get_query helper, and the "query" key
from _extract_image_data; move _get_eval_input to the end of the chain
so the joined multi-message text wins INPUT_VALUE, matching the openai,
litellm, dspy, and groq handlers.

Generated with Devin
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
jjcav84 added a commit to jjcav84/traceAI that referenced this pull request Nov 6, 2026
Remaining instances of the future-agi#151 defect family (openai future-agi#245, litellm
eval_input with a first-message-only "query" as the last INPUT_VALUE
write.

- mistralai: drop the trailing eval_input[0] yield so the joined text
  is the last write
- vertexai: remove the query set_attribute and the "query" key from
  _extract_image_data
- langchain: remove _as_query from the span attribute chain and move
  _as_eval_input last so the joined text wins INPUT_VALUE; drop the
  query element from _filter_images and the now-unused _as_query helper

Generated with Devin
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
jjcav84 added a commit to jjcav84/traceAI that referenced this pull request Nov 6, 2026
Same defect as future-agi#151 (fixed for openai in future-agi#245): the completion
instrumentation wrote INPUT_VALUE three times, ending with
"query" = eval_input[0] — the first message's text only — after
already writing the joined text of every message. Multi-turn inputs
lost all but the first content piece.

Drop the query write and the now-unused "query" key from
_process_messages so the joined eval_input survives; image parts still
surface as {{input_images.N}} placeholders alongside INPUT_IMAGES.

Also repairs three stale attribute assertions in the litellm test file
that referenced pre-migration names (FI_SPAN_KIND -> GEN_AI_SPAN_KIND,
LLM_MODEL_NAME -> GEN_AI_REQUEST_MODEL) so the file actually runs green.

Generated with Devin
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
nik13 pushed a commit that referenced this pull request Nov 9, 2026
Fixes `INPUT_VALUE` silently dropping all but the first message for conversational inputs (#151).

`_process_input_data` wrote `INPUT_VALUE` three times:

1. JSON dump of the filtered structured messages
2. Joined text of every message's content (`eval_input`)
3. **`eval_input[0]` — the first message's text only** ← the bug

Write 3 is dead logic (its `if` is implied by write 2's) and destroyed the multi-message context write 2 produced. Removed so the joined string survives; image parts still surface as `{{input_images.N}}` placeholders with `INPUT_IMAGES` set alongside.

```python
process([{"role": "user", "content": "a"},
         {"role": "assistant", "content": "b"},
         {"role": "user", "content": "c"}], span)
# before: INPUT_VALUE == "a"
# after:  INPUT_VALUE == "a \n b \n c"
```

**Sibling note for reviewers:** `traceai_litellm` carries the identical pattern — `_process_messages` returns `query = eval_input[0]` and `_instrument_func_type_completion` writes it last, overwriting the joined `eval_input`. Same first-message-only loss. I kept this PR scoped to the reported path; happy to follow up there if you'd like it fixed identically.
nik13 added a commit that referenced this pull request Nov 9, 2026
Completes #245 (issue #151): input.value now keeps every message, so derive
the MIME type from that exported value instead of the first message. A JSON
first message followed by text is now text/plain. Update the bare-prompt test
to the current unquoted behaviour and add JSON/text MIME regressions.
@nik13

nik13 commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

Thanks! This change is now on dev in 2979dc2 as part of the traceAI release train (squashed into one commit).

@nik13 nik13 closed this Nov 9, 2026
nik13 added a commit that referenced this pull request Nov 10, 2026
…14 and Java 1.0.3

Core 1.3.1 also ships the PII-redaction scoping fixes (#195/#244) that sat under
Unreleased, and the 1.3.0 heading gets its release date. OpenAI 0.1.14 also
ships the Azure metadata (#154), Baseten (#541), Azure Government (#502),
OpenRouter host (#547), full multi-message input (#245) and exhausted-stream
changes. Java 1.0.3 also ships #543 and #605. Notes the remaining blank
OTEL-env Authorization limitation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nik13 added a commit that referenced this pull request Nov 10, 2026
… traceAI-langchain 0.2.2, Java 1.0.3 (#636)

* chore(release): bump core, OpenAI, LangChain and Java patch versions

Synchronize runtime versions, reactor parent references and release changelogs for #155, #348, #397, #393 and #425. Dependency floors are unchanged.

Co-Authored-By: Claude Code <noreply@anthropic.com>

* chore(release): list every change shipping in core 1.3.1, OpenAI 0.1.14 and Java 1.0.3

Core 1.3.1 also ships the PII-redaction scoping fixes (#195/#244) that sat under
Unreleased, and the 1.3.0 heading gets its release date. OpenAI 0.1.14 also
ships the Azure metadata (#154), Baseten (#541), Azure Government (#502),
OpenRouter host (#547), full multi-message input (#245) and exhausted-stream
changes. Java 1.0.3 also ships #543 and #605. Notes the remaining blank
OTEL-env Authorization limitation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Code <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

INPUT_VALUE overwritten in _process_input_data leading to loss of multi-message context

2 participants