Skip to content

fix(openai): attribute Vercel gateway TypeScript requests - #463

Merged
nik13 merged 1 commit into
devfrom
fix/openai-vercel-gateway-typescript
Nov 1, 2026
Merged

nik13 merged 1 commit into
devfrom
fix/openai-vercel-gateway-typescript

Conversation

@nik13

@nik13 nik13 commented Oct 31, 2026

Copy link
Copy Markdown
Contributor

Summary

Keep Vercel AI Gateway on the existing @traceai/openai integration. Correct the receiving-service attribution and provide a version-qualified TypeScript client recipe; do not introduce a duplicate gateway package.

  • Resolve the SDK resource's owner on each request (client at the 4.0 floor, _client on newer SDKs). An exact, normalized ai-gateway.vercel.sh hostname records gen_ai.provider.name=vercel before the request, including failures. Chat Completions, Completions, Responses and Embeddings share the resolver. Other endpoints retain the OpenAI compatibility label; Embeddings now also carry provider attribution.
  • Do not infer the upstream provider from a model prefix or response model. Do not export the endpoint URL, credentials, path or query. Malformed SDK metadata fails open with a sanitized diagnostic.
  • Preserve the public instrumentor entry point and runtime module paths. Select the matching ESM/CJS declaration in package exports: the exact ESM TypeScript recipe otherwise fails on OpenAI 5.23.2 because its ESM and CJS APIError classes have distinct private types.
  • Add a short, privacy-conscious recipe pinned to openai@5.23.2, real-SDK/local-OTLP regressions, module type fixtures and focused network-isolated CI. Declare the Zod 3 test dependency already imported by the existing suite. No runtime dependency, version or changelog bump.

Reproduction and validation

Base: d6a9966dd4d5d5fb344f7f5ab11b7483c2c2a6bc (dev). The initial attribution regression using real SDK clients and synthetic fetch responses produced 10 failures / 6 passes on OpenAI 4.0.0 and 13 failures / 6 passes on 5.23.2 and 7.47.0. Gateway spans were openai (or missing provider for Embeddings). The unrelated-host controls passed.

With outbound networking denied except loopback, the final committed contract passes:

OpenAI SDK Content enabled Content/error hiding enabled
4.0.0 20 passed 20 passed
5.23.2 23 passed 23 passed
7.47.0 23 passed 23 passed

The 7.x row is a manual attribution compatibility probe, not expanded peer or automatic-hook support. The declared 4.x/5.x ranges are unchanged. The floor does not run stream tests because its pre-existing missing tee() lifecycle issue is handled by the overlapping repair PRs below.

Checks include all four instrumented operations where the SDK exposes them, HTTP failures, exact normalized hosts, lookalike hosts, independent/concurrent clients, changed base URLs, malformed/getter-throwing metadata, completed chat streams on 5/7, unchanged returned responses, exactly one client span, reported non-streaming usage, FI project routing, and exclusion of URL/credential markers from the OTLP/HTTP JSON wire. LLM content/error hiding is checked; this does not claim to repair the existing shared-core embedding-content privacy gap.

  • Existing locked-SDK Jest suite: 111 passed, 2 existing skipped; 15 snapshots passed.
  • CJS/ESM/ESNext builds passed; frozen install and git diff --check passed.
  • The initial packed ESM recipe reproduced TS2345 from separate private APIError.makeMessage declarations. Matching conditional declarations fix it without casts or weakening the public signature. Packed CJS and ESM consumer fixtures typecheck.
  • The held MDX compiles. Its exact TypeScript snippet typechecks and executes with the packed repair, published @traceai/fi-core@1.1.0, OpenAI 5.23.2, synthetic model transport and a localhost collector: one correctly attributed, content-hidden span, expected project/headers and token counts.
  • GitHub CI repeats floor/current-supported/manual-current contracts with outbound networking disabled, plus the existing unit suite and packed module type checks.

No vendor/model API, real credential or secret file was used. No live gateway/cloud acceptance or full docs-site browser render is claimed.

Integrate with the existing shared-package repairs

I listed and inspected the open package/page PRs before making changes. #336 does not recognize Vercel's host; #332 is Cloudflare-specific; #410 repairs APIPromise behavior. None fixes this gateway attribution or the qualified ESM type recipe. No existing branch was modified.

The concurrently opened #460 fixes the Python gateway path and is separate from this TypeScript repair; combine their held documentation additions after both packages are released.

These TypeScript PRs touch the same shared instrumentor. When integrating, move the ai-gateway.vercel.sh -> vercel case into #336's common host resolver rather than keeping two resolvers; retain #332's explicit Cloudflare policy and #410's lazy promise observer. Do not let a mechanical conflict resolution drop either provider mapping or restore the old stream/promise handling. This PR deliberately does not duplicate those lifecycle repairs or widen the supported SDK range.

Documentation and boundaries

The package README ships with the repair. The customer gateway page is staged locally for publication after the corrected package is on npm; no docs PR is opened against an unreleased dependency. The existing Python recipe is retained in that draft.

Vercel Trace Drains expose a separate server-side surface (routing, provider attempts and billing; no prompts/completions). This change does not offer an unvalidated direct-to-Future-AGI drain route. The recipe warns against layering instrumentors or summing client and gateway usage. No package deletion, merge, tag or publication.

Review

Self-reviewed against the internal minimal-code, typing, failure-mode, testing/scenario and PR-review standards. The runtime change is one small host resolver and four call sites; module declaration selection fixes the demonstrated onboarding contract. No application routes, migrations, frontend, tenant authorization or OSS/EE boundary changes. Failure metadata is not logged verbatim because it can contain URL credentials. Human review and reconciliation with the other shared-package repairs are still required.

E2E coverage

Not applicable (no application e2e/ harness). The committed real-SDK/OTLP and packed-declaration contracts are the integration regressions.

Sources

Generated with Claude Code

Keep shared instrumentation, qualify the TypeScript client route, and select matching ESM and CJS declarations.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@nik13
nik13 merged commit 0160ab6 into dev Nov 1, 2026
27 checks passed
@nik13
nik13 deleted the fix/openai-vercel-gateway-typescript branch November 1, 2026 01:27
@nik13 nik13 mentioned this pull request Nov 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant