Skip to content

buffer_streamed_tool_calls=True crashes when a Chat Completions provider omits the tool call index #4823

Description

@rioyu123

Summary

The buffered tool-call path assumes every streamed tool_calls[] delta carries an index. The OpenAI SDK's lenient construct_type parsing leaves index=None when a provider omits it; the buffered replay then raises pydantic_core.ValidationError (ChoiceDeltaToolCall.index must be an int) and, when indexed and index-less calls coexist, TypeError: '<' not supported between instances of 'NoneType' and 'int'. The unbuffered path handles the same stream correctly, and the option's documented audience (src/agents/models/openai_provider.py:85-88) is exactly OpenAI-compatible providers whose chunk semantics are not reliable.

Affected code (main @ 89c02c8), src/agents/models/chatcmpl_stream_handler.py

  • _accumulate_tool_call_delta (:309-316) keys buffered calls on tool_call_delta.index, which may be None.
  • _buffered_tool_call_delta (:355-356): ChoiceDeltaToolCall(index=buffered_call.index, ...) raises ValidationError for None.
  • _buffered_tool_calls_chunk (:379-383): sorted(buffered_calls.items()) raises TypeError with mixed None/int keys.

Reproduction

from openai._models import construct_type
from openai.types.chat.chat_completion_chunk import ChatCompletionChunk
from agents.models.chatcmpl_stream_handler import ChatCmplStreamHandler

def chunk(delta, finish_reason=None):
    return construct_type(
        type_=ChatCompletionChunk,
        value={"id": "c", "object": "chat.completion.chunk", "created": 1, "model": "m",
               "choices": [{"index": 0, "delta": delta, "finish_reason": finish_reason}]},
    )

chunks = [
    chunk({"role": "assistant", "tool_calls": [{"id": "call_1", "type": "function",
                                                "function": {"name": "f", "arguments": '{"a":'}}]}),
    chunk({"tool_calls": [{"function": {"arguments": "1}"}}]}),
    chunk({}, "tool_calls"),
]

async def gen():
    for c in chunks:
        yield c

events = [e async for e in ChatCmplStreamHandler.handle_stream(
    response, ChatCmplStreamHandler.buffer_tool_call_stream(gen()))]
# pydantic_core.ValidationError: 1 validation error for ChoiceDeltaToolCall ... index ... int_type

Observed: ValidationError, and TypeError when one delta has index: 0 and another has none.
Expected: the same response.completed output as the unbuffered path, [('function_call', 'call_1', '{"a":1}')], and for the mixed stream both calls in a deterministic order.

Root cause

The buffered replay hard-codes buffered_call.index and sorts on it without accounting for None.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions