Skip to content

fix(openai): migrate vLLM and attribute Ollama clients - #424

Merged
nik13 merged 3 commits into
devfrom
fix/openai-ollama-client-provider
Nov 1, 2026
Merged

nik13 merged 3 commits into
devfrom
fix/openai-ollama-client-provider

Conversation

@nik13

@nik13 nik13 commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Keep Ollama's and vLLM's OpenAI-compatible routes in the existing traceAI-openai package. Correct provider attribution and complete vLLM's non-breaking legacy migration. Do not create an ollama-openai package or remove either native traceai-ollama or the legacy traceAI-vllm package.

  • Recognize the documented ollama.com API host as ollama; look-alike hosts and unconfigured local servers retain their existing behavior.

  • Add set_client_provider(client, provider="ollama") for explicit per-client attribution on local/self-hosted endpoints. provider=None restores host detection. Overrides take precedence, are isolated to the exact client instance, and never enter SDK requests or headers.

  • Preserve OpenAIInstrumentor().instrument(...) and all existing SDK entry points. This changes one span attribute, not the request/stream lifecycle, and creates no additional spans.

  • Keep configuration on the client itself, without a global registry or a hashability requirement. Clients returned by SDK copy() / with_options() require their own explicit override.

  • Deprecate VLLMInstrumentor without removing its public entry point or raising OpenAI's legacy 1.0.0 floor. When the shared instrumentor is active, it owns client spans and privacy/export configuration. The legacy wrapper resumes chat tracing after shared teardown; setup and teardown orders are independent.

  • Replace the overbroad vLLM README with executable shared-client migration guidance and a clear chat-only legacy scope. Embeddings/Responses use the shared route; server/engine telemetry remains native vLLM OpenTelemetry.

  • Fix the referenced OpenAI quickstart to use OpenAI().chat.completions.create() and attribute-based response access rather than the removed ChatCompletion API.

Why this route

Ollama documents https://ollama.com/v1 and http://localhost:11434/v1 for OpenAI-compatible clients. Langfuse's Ollama guide likewise reuses its OpenAI integration. That reuse is appropriate; it does not establish better automatic attribution for unknown local hosts. This fix deliberately does not guess from a port, model name or loopback address.

Before changing implementation, the real SDK and a localhost OTLP/HTTP receiver reproduced gen_ai.provider.name=openai on the official Ollama endpoint. The final regression suite against unchanged dev d6a9966dd4d5d5fb344f7f5ab11b7483c2c2a6bc fails 65 tests and passes 39 at both OpenAI 1.69.0 and 3.42.0; failures pin the incorrect official-host label and absent per-client override API.

Verification

All local tests ran with the existing macOS sandbox profile denying outbound networking except localhost. Final regression runs additionally use pytest-socket to permit only localhost connections and Unix sockets. Requests use the real SDK with its matching MockTransport; no vendor/model API or Future AGI ingestion API was called and no real credentials were used.

  • OpenAI 1.69.0 (declared floor), source core: 166 package tests passed, including 104 new contracts.
  • OpenAI 1.93.1, published core 1.1.0, OTel 1.34.1 (the existing CI dependency generation): 166 package tests passed.
  • OpenAI 3.42.0 (current), source core: 104 new contracts passed. The older package-wide fixtures use httpx rather than the SDK's httpx2 transport; this is not a claim that the entire legacy suite was run at 3.x.
  • Wire assertions decode actual OTLP protobuf received over HTTP: exactly one span, provider/model/status, stream completion timing, and absence of hidden input/output text and the synthetic API key.
  • Covers sync/async chat, streaming chat, embeddings, Responses, streaming Responses, HTTP errors for all three APIs, exact-host matching, override precedence/reset, multiple concurrent clients, copied clients, identifier validation, garbage collection, and unhashable SDK subclasses.
  • The workflow now repeats the 104 provider contracts plus 91 migration/example contracts at floor and current SDK versions, and 38 legacy tests at all three boundaries. Tests run in a loopback-only network namespace, with pytest-socket restrictions as well.
  • Ruff E/F/I and formatting checks on new Python files, git diff --check, and wheel build passed.
  • Held docs Python snippets executed offline against all three SDK environments with mocked registration and SDK transport.

Independent review fixes (final head b4054cbf2a58e5f376abf43e4dd4e0a71529faa7)

Both P2 findings were accepted and fixed here; PR #367 remains unchanged.

  • New migration/example contracts against the prior fix(openai): migrate vLLM and attribute Ollama clients #424 head: 50 failed / 41 passed at OpenAI 1.69.0 and 3.42.0. These reproduce double spans, missing migration warning, and the removed quickstart API.
  • Fixed head, OpenAI 1.69.0 and 3.42.0: 195 provider/migration contracts passed at each boundary. Covers both setup orders, both teardown orders, reenabling, separate tracer providers, shared privacy ownership, sync/async chat/streams/embeddings/Responses/errors, real SDK README execution, and actual localhost OTLP.
  • Standalone legacy package: 38 tests passed each at OpenAI 1.0.0, 1.69.0, 3.42.0, including six new real-SDK chat/stream/error contracts. No shared import is introduced at the legacy floor. Existing mock tests now restore their singleton and SDK-method state instead of contaminating real-client tests.
  • Final head after merging dev: entire OpenAI package 335 passed at both 1.69.0 and 3.42.0. The earlier pre-merge floor run passed 257 tests; dev added provider/README regressions that are included in the final count.
  • Final-head local composition with fix(fireworks): prefer a single OpenAI-compatible trace #367's existing shared-accumulator/Fireworks changes: 441 full OpenAI package tests passed at each of 1.69.0 and 3.42.0. Separate independent protobuf/JSON OTLP and duplicate-span probes: 74 passed at each boundary. Composition is local evidence, not a claim that fix(fireworks): prefer a single OpenAI-compatible trace #367 is merged.
  • release_plan.py --check, bootstrap_audit.py --check, all 54 release unittests, new-test Ruff E/F/I/format checks and git diff --check passed. OpenAI 3.42.0 was independently rechecked as the current PyPI version.
  • All execution was offline in the required outbound-denied macOS sandbox, using synthetic credentials and real SDK mock transports; OTLP stayed on localhost. The first combined-current run lacked the optional httpx test dependency; rerunning in the existing Fireworks environment with both transport libraries installed passed. No vendor/model API or live Future AGI call was made.

CI on final head b4054cbf2a58e5f376abf43e4dd4e0a71529faa7: 43 checks passed, 12 skipped, none failed or pending. Includes all three loopback-only migration jobs, Python package tests, integration tests, release validation and provider recipe checks.

The first push exposed a README merge conflict with newly merged dev work, which prevented PR test dispatch. Merged dev c8118613 using chore: merge dev, preserving the Vercel guide, environment instructions and force-flush addition alongside the context-managed modern SDK snippet. Repeated package, composition, independent, legacy and release checks on the resulting final head.

Release sequencing

Release a shared traceAI-openai build containing this PR and #367's streamed-tool normalization before directing vLLM customers to the replacement. Release the legacy traceAI-vllm deprecation patch alongside/after that shared release. Preserve both legacy release inventory entries and their existing floors. The package README now contains the migration, but the separate docs-site page remains held until the corresponding shared package is published. No docs PR was opened or published.

Review and rollout

Self-reviewed against future-agi/internal-docs at 7360c146183c684a96e75e57069b2661445135ef: additive API, bounded identifier validation, minimal implementation, compatible failure behavior, per-client isolation, no retained client registry, and regression coverage. No migrations, new runtime dependencies, tenant/auth changes or OSS/EE boundary changes. No ticket identifier was supplied for this audit item; none was invented.

E2E coverage: not applicable (no e2e/ product harness in this repository); SDK-to-OTLP contracts cover the affected boundary.

Open PRs were checked before implementation. Other OpenAI provider fixes do not implement this Ollama/client override. PR #329 also changes _get_attributes_from_instance; when integrating both, preserve its gateway endpoint attributes and use the explicit client override before automatic host/route resolution. No other branch was modified.

Release-please owns versions and changelogs; neither is manually bumped. The corrected Ollama and vLLM docs-site pages are held locally for the maintainer until a package containing this API is on the registry. No docs PR advertises the unreleased helper. The guide was not rendered in the docs site or verified against live Ollama/Future AGI services. Merge and registry publication remain with the maintainer.

Generated with Claude Code

Recognize only the official Ollama host automatically. Add a per-client provider override for local and self-hosted compatible servers without changing requests or installing another instrumentor.

Co-Authored-By: Claude Code <noreply@anthropic.com>
nik13 and others added 2 commits October 31, 2026 19:15
Preserve the standalone OpenAI 1.0 entry point, yield to the shared instrumentor in either setup order, document the chat-only legacy boundary, and execute modern SDK README examples. Add offline floor/current migration and standalone legacy CI contracts.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Preserve both README updates: modern context-managed OpenAI usage, flushing, and the Vercel section.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@nik13 nik13 changed the title fix(openai): attribute Ollama and explicitly configured clients fix(openai): migrate vLLM and attribute Ollama clients Nov 1, 2026
@nik13
nik13 merged commit cd4ea17 into dev Nov 1, 2026
66 checks passed
@nik13
nik13 deleted the fix/openai-ollama-client-provider branch November 1, 2026 03:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant