Repository navigation
fix(openai): migrate vLLM and attribute Ollama clients - #424
Merged
Merged
Conversation
Recognize only the official Ollama host automatically. Add a per-client provider override for local and self-hosted compatible servers without changing requests or installing another instrumentor. Co-Authored-By: Claude Code <noreply@anthropic.com>
This was referenced Oct 31, 2026
Preserve the standalone OpenAI 1.0 entry point, yield to the shared instrumentor in either setup order, document the chat-only legacy boundary, and execute modern SDK README examples. Add offline floor/current migration and standalone legacy CI contracts. Co-Authored-By: Claude Code <noreply@anthropic.com>
Preserve both README updates: modern context-managed OpenAI usage, flushing, and the Vercel section. Co-Authored-By: Claude Code <noreply@anthropic.com>
This was referenced Nov 1, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Keep Ollama's and vLLM's OpenAI-compatible routes in the existing
traceAI-openaipackage. Correct provider attribution and complete vLLM's non-breaking legacy migration. Do not create anollama-openaipackage or remove either nativetraceai-ollamaor the legacytraceAI-vllmpackage.Recognize the documented
ollama.comAPI host asollama; look-alike hosts and unconfigured local servers retain their existing behavior.Add
set_client_provider(client, provider="ollama")for explicit per-client attribution on local/self-hosted endpoints.provider=Nonerestores host detection. Overrides take precedence, are isolated to the exact client instance, and never enter SDK requests or headers.Preserve
OpenAIInstrumentor().instrument(...)and all existing SDK entry points. This changes one span attribute, not the request/stream lifecycle, and creates no additional spans.Keep configuration on the client itself, without a global registry or a hashability requirement. Clients returned by SDK
copy()/with_options()require their own explicit override.Deprecate
VLLMInstrumentorwithout removing its public entry point or raising OpenAI's legacy 1.0.0 floor. When the shared instrumentor is active, it owns client spans and privacy/export configuration. The legacy wrapper resumes chat tracing after shared teardown; setup and teardown orders are independent.Replace the overbroad vLLM README with executable shared-client migration guidance and a clear chat-only legacy scope. Embeddings/Responses use the shared route; server/engine telemetry remains native vLLM OpenTelemetry.
Fix the referenced OpenAI quickstart to use
OpenAI().chat.completions.create()and attribute-based response access rather than the removedChatCompletionAPI.Why this route
Ollama documents
https://ollama.com/v1andhttp://localhost:11434/v1for OpenAI-compatible clients. Langfuse's Ollama guide likewise reuses its OpenAI integration. That reuse is appropriate; it does not establish better automatic attribution for unknown local hosts. This fix deliberately does not guess from a port, model name or loopback address.Before changing implementation, the real SDK and a localhost OTLP/HTTP receiver reproduced
gen_ai.provider.name=openaion the official Ollama endpoint. The final regression suite against unchanged devd6a9966dd4d5d5fb344f7f5ab11b7483c2c2a6bcfails 65 tests and passes 39 at both OpenAI 1.69.0 and 3.42.0; failures pin the incorrect official-host label and absent per-client override API.Verification
All local tests ran with the existing macOS sandbox profile denying outbound networking except localhost. Final regression runs additionally use pytest-socket to permit only localhost connections and Unix sockets. Requests use the real SDK with its matching MockTransport; no vendor/model API or Future AGI ingestion API was called and no real credentials were used.
git diff --check, and wheel build passed.Independent review fixes (final head
b4054cbf2a58e5f376abf43e4dd4e0a71529faa7)Both P2 findings were accepted and fixed here; PR #367 remains unchanged.
release_plan.py --check,bootstrap_audit.py --check, all 54 release unittests, new-test Ruff E/F/I/format checks andgit diff --checkpassed. OpenAI 3.42.0 was independently rechecked as the current PyPI version.httpxtest dependency; rerunning in the existing Fireworks environment with both transport libraries installed passed. No vendor/model API or live Future AGI call was made.CI on final head
b4054cbf2a58e5f376abf43e4dd4e0a71529faa7: 43 checks passed, 12 skipped, none failed or pending. Includes all three loopback-only migration jobs, Python package tests, integration tests, release validation and provider recipe checks.The first push exposed a README merge conflict with newly merged dev work, which prevented PR test dispatch. Merged dev
c8118613usingchore: merge dev, preserving the Vercel guide, environment instructions and force-flush addition alongside the context-managed modern SDK snippet. Repeated package, composition, independent, legacy and release checks on the resulting final head.Release sequencing
Release a shared
traceAI-openaibuild containing this PR and #367's streamed-tool normalization before directing vLLM customers to the replacement. Release the legacytraceAI-vllmdeprecation patch alongside/after that shared release. Preserve both legacy release inventory entries and their existing floors. The package README now contains the migration, but the separate docs-site page remains held until the corresponding shared package is published. No docs PR was opened or published.Review and rollout
Self-reviewed against
future-agi/internal-docsat7360c146183c684a96e75e57069b2661445135ef: additive API, bounded identifier validation, minimal implementation, compatible failure behavior, per-client isolation, no retained client registry, and regression coverage. No migrations, new runtime dependencies, tenant/auth changes or OSS/EE boundary changes. No ticket identifier was supplied for this audit item; none was invented.E2E coverage: not applicable (no
e2e/product harness in this repository); SDK-to-OTLP contracts cover the affected boundary.Open PRs were checked before implementation. Other OpenAI provider fixes do not implement this Ollama/client override. PR #329 also changes
_get_attributes_from_instance; when integrating both, preserve its gateway endpoint attributes and use the explicit client override before automatic host/route resolution. No other branch was modified.Release-please owns versions and changelogs; neither is manually bumped. The corrected Ollama and vLLM docs-site pages are held locally for the maintainer until a package containing this API is on the registry. No docs PR advertises the unreleased helper. The guide was not rendered in the docs site or verified against live Ollama/Future AGI services. Merge and registry publication remain with the maintainer.
Generated with Claude Code