Skip to content

fix(openai): attribute Vercel AI Gateway client spans - #460

Merged
nik13 merged 3 commits into
devfrom
fix/openai-vercel-gateway-provider
Nov 1, 2026
Merged

nik13 merged 3 commits into
devfrom
fix/openai-vercel-gateway-provider

Conversation

@nik13

@nik13 nik13 commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Keep Vercel AI Gateway on the existing Python traceAI-openai integration. Attribute the documented ai-gateway.vercel.sh endpoint to vercel, not the SDK's default openai label. No package is created or deleted, and OpenAIInstrumentor().instrument(...) remains unchanged.

  • Extend the existing exact-host map by one entry; do not infer an upstream/fallback provider from a model name, host suffix or response model.
  • Preserve content-bearing client spans for sync, async, streaming and failed Chat Completions. No additional spans or token accounting are introduced.
  • Explain the client-versus-drain distinction in the package README.
  • Add floor/current SDK contracts with an actual localhost OTLP/HTTP receiver and a dedicated CI matrix.

Why this is the right route

Langfuse's gateway guide also reuses its OpenAI client integration. Vercel's native Trace Drains (reference last updated 2026-09-08) expose routing, fallback/provider attempts, tokens and cost, and list Braintrust as a native destination. Those drains omit prompts/completions and require a Pro/Enterprise team. They complement rather than replace client instrumentation.

The vercel label identifies the service known to the client, not the actual provider serving a fallback. This follows the OTel provider-name caveat: instrumentation identifies what it knows, which may differ from the actual model provider.

No drain-to-Future-AGI recipe is advertised. A direct native drain still needs separate validation of FI authentication, project resources, GenAI mapping and deduplicated accounting. register(project_name=...) configures application spans, not Vercel-generated traces; do not sum client and drain usage for the same requests.

Reproduction and tests

On unchanged dev d6a9966dd4d5d5fb344f7f5ab11b7483c2c2a6bc, the new regression suite has 26 failures and 8 passes with both OpenAI 1.69.0 and 3.42.0: the exact gateway host and every emitted gateway span report openai rather than vercel.

With this fix, using published fi-instrumentation-otel==1.2.2 and Python 3.13:

  • OpenAI 1.69.0 (declared SDK floor): 96 package tests passed, including all 34 new contracts.
  • OpenAI 3.42.0 (current registry version): 34 new contracts passed. This is not a claim that the older package-wide httpx fixtures were run against the 3.x httpx2 transport.
  • OpenAI 1.93.1, tracing core 1.1.0, OpenTelemetry 1.34.1, Python 3.11 (the package-lock generation): 96 package tests passed. The initial CI run exposed a test-only use of a newer TraceConfig keyword; the final suite uses the environment flag and tests attribution on the legacy core too. Stronger error/tool-definition privacy assertions remain mandatory with the guide's core 1.2.2, not falsely claimed for 1.1.0.
  • OTLP checks cover exact-host normalization/look-alikes, one span per client call, sync/async streaming and errors, requested/response model differences, usage present/absent, content and tool-definition masking, error privacy, and preservation of error status.
  • The wire assertions verify /tracer/v1/traces, FI auth headers versus gateway Bearer auth, and project_name/project_type resources. These are local collector contracts, not a claim of live FI ingestion validation.
  • The held docs' exact Python example ran offline on both SDK versions with the real registration/export pipeline and a synthetic SDK transport.
  • The held guide rendered in the real Astro docs site: HTTP 200, both internal integration links 200, code blocks visible, no browser page errors, screenshot inspected.
  • Ruff E/F/I, formatting, git diff --check, and a wheel build passed.

All local execution tests ran under the existing macOS sandbox profile denying outbound networking except localhost; pytest also blocked non-local sockets. No vendor/model API, production ingestion API, or real credentials were used. The new CI matrix applies the same pytest socket restriction.

Review and release

Self-reviewed against the internal code-review playbook: minimal shared-map extension, exact matching, no new runtime dependency, public API/lifecycle preservation, no guessed fallback attribution, regression and failure-path coverage. No migrations, tenant/auth changes or OSS/EE boundary changes. No product e2e/ harness applies; SDK-to-OTLP tests cover this boundary. No ticket identifier was supplied for this audit item, so none is invented.

Open PRs were checked before implementation. Other OpenAI provider fixes (including #424's explicit per-client override) and #369's TypeScript Vercel AI SDK/Eve changes do not implement this exact gateway attribution. No other branch was modified.

Release-please owns the version and changelog; neither is manually changed. The corrected customer guide is held locally until a package containing this mapping is published. Its install line must then be pinned to that release before a docs PR; no docs PR currently advertises the unreleased fix. No merge, tag or publication was performed.

🤖 Generated with Claude Code

Keep shared client instrumentation and identify the documented gateway host without guessing its upstream provider. Exercise SDK floor/current contracts through OTLP/HTTP.

Co-Authored-By: Claude Code <noreply@anthropic.com>
nik13 and others added 2 commits October 31, 2026 16:52
Keep gateway attribution contracts running with core 1.1.0 without claiming the stronger privacy behavior documented for core 1.2.2.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
@nik13
nik13 merged commit 2990414 into dev Nov 1, 2026
40 checks passed
@nik13
nik13 deleted the fix/openai-vercel-gateway-provider branch November 1, 2026 01:27
@nik13 nik13 mentioned this pull request Nov 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant