Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
80 changes: 80 additions & 0 deletions .github/workflows/openai-vercel-gateway-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
name: OpenAI Vercel gateway tests

on:
pull_request:
branches: [dev, main]
paths:
- 'typescript/packages/traceai_openai/**'
- 'typescript/packages/fi-core/**'
- 'typescript/packages/fi-semantic-conventions/**'
- 'typescript/pnpm-lock.yaml'
- '.github/workflows/openai-vercel-gateway-tests.yml'
workflow_dispatch:

permissions:
contents: read

jobs:
contracts:
name: Vercel gateway OpenAI ${{ matrix.sdk }}
runs-on: ubuntu-24.04
timeout-minutes: 15
strategy:
fail-fast: false
matrix:
# 7.x is a manual attribution probe, not expanded automatic-hook support.
sdk: ['4.0.0', '5.23.2', '7.47.0']
defaults:
run:
shell: bash
working-directory: typescript
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
persist-credentials: false
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version: '22.23.4'
- name: Install pinned dependencies
run: |
mkdir -p "$RUNNER_TEMP/corepack-bin"
corepack enable --install-directory "$RUNNER_TEMP/corepack-bin" pnpm
echo "$RUNNER_TEMP/corepack-bin" >> "$GITHUB_PATH"
corepack pnpm --filter @traceai/openai... install --frozen-lockfile --ignore-scripts
npm install --prefix "$RUNNER_TEMP/openai-sdk" --ignore-scripts --no-audit --no-fund "openai@${{ matrix.sdk }}"
- name: Build package
run: |
pnpm run build:deps
pnpm --filter @traceai/openai run build
- name: Install packed package for the qualified client types
if: matrix.sdk == '5.23.2'
run: |
mkdir -p "$RUNNER_TEMP/gateway-pack"
pnpm --dir packages/traceai_openai pack --pack-destination "$RUNNER_TEMP/gateway-pack"
npm install --prefix "$RUNNER_TEMP/openai-sdk" --ignore-scripts --no-audit --no-fund "$RUNNER_TEMP"/gateway-pack/*.tgz
cp packages/traceai_openai/test/moduleTypes.mts packages/traceai_openai/test/moduleTypes.cts "$RUNNER_TEMP/openai-sdk/"
- name: Network-isolated ESM and CommonJS type contracts
if: matrix.sdk == '5.23.2'
run: |
sudo unshare --net -- bash -c 'ip link set lo up; exec "$@"' bash \
env PATH="$PATH" node node_modules/typescript/bin/tsc --noEmit \
--module NodeNext --moduleResolution NodeNext --target ES2022 --strict --skipLibCheck \
"$RUNNER_TEMP/openai-sdk/moduleTypes.mts" "$RUNNER_TEMP/openai-sdk/moduleTypes.cts"
- name: Network-isolated SDK and OTLP contracts
env:
OPENAI_TEST_SDK: ${{ runner.temp }}/openai-sdk/node_modules/openai
run: |
for hide in false true; do
sudo unshare --net -- bash -c 'ip link set lo up; exec "$@"' bash \
env PATH="$PATH" OPENAI_TEST_SDK="$OPENAI_TEST_SDK" \
FI_HIDE_INPUTS="$hide" FI_HIDE_OUTPUTS="$hide" \
FI_HIDE_LLM_INVOCATION_PARAMETERS="$hide" FI_HIDE_ERROR_DETAILS="$hide" \
node --test packages/traceai_openai/test/vercelGateway.cjs
done
- name: Network-isolated locked SDK unit suite
if: matrix.sdk == '5.23.2'
run: |
sudo unshare --net -- bash -c 'ip link set lo up; exec "$@"' bash \
env PATH="$PATH" node packages/traceai_openai/node_modules/jest/bin/jest.js \
--config packages/traceai_openai/jest.config.js --runInBand
56 changes: 56 additions & 0 deletions typescript/packages/traceai_openai/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,62 @@ try {
}
```

## Vercel AI Gateway

Use the same instrumentor for the gateway's OpenAI-compatible client; do not add a second gateway instrumentor. This non-streaming Chat Completions recipe uses `openai@5.23.2`. The declared peer and automatic-hook range remains OpenAI 4.x/5.x, not 6.x/7.x. For applications using `ai` instead of `openai`, use the [Vercel AI SDK integration](https://docs.futureagi.com/docs/integrations/traceai/vercel).

```bash
npm install @traceai/openai @traceai/fi-core@1.1.0 openai@5.23.2
export AI_GATEWAY_API_KEY="your-vercel-gateway-key"
export AI_GATEWAY_MODEL="provider/model"
export FI_API_KEY="your-futureagi-api-key"
export FI_SECRET_KEY="your-futureagi-secret-key"
# Keep prompt, answer, invocation and error content out of telemetry.
export FI_HIDE_INPUTS=true
export FI_HIDE_OUTPUTS=true
export FI_HIDE_LLM_INVOCATION_PARAMETERS=true
export FI_HIDE_ERROR_DETAILS=true
```

Choose `AI_GATEWAY_MODEL` from [Vercel's model list](https://vercel.com/docs/ai-gateway/models-and-providers). With Node.js 22.23.4 or later, save as `gateway.mts` and run `node gateway.mts`:

```typescript
import OpenAI from "openai";
import { register, ProjectType } from "@traceai/fi-core";
import { OpenAIInstrumentation } from "@traceai/openai";

const model = process.env.AI_GATEWAY_MODEL;
if (!model) throw new Error("Set AI_GATEWAY_MODEL to a Vercel model ID.");

const provider = register({
projectName: "my-gateway-app",
projectType: ProjectType.OBSERVE,
});
const instrumentation = new OpenAIInstrumentation();
instrumentation.setTracerProvider(provider);
instrumentation.manuallyInstrument(OpenAI);

const client = new OpenAI({
baseURL: "https://ai-gateway.vercel.sh/v1",
apiKey: process.env.AI_GATEWAY_API_KEY,
});
try {
const response = await client.chat.completions.create({
model,
messages: [{ role: "user", content: "Say hello in five words." }],
});
console.log(response.choices[0].message.content);
} finally {
await provider.shutdown();
}
```

`gen_ai.provider.name=vercel` identifies the receiving gateway at `ai-gateway.vercel.sh`, not a model's author or the gateway's eventual upstream provider. Attribution is set before the request, including for failed calls, and is shared by Chat Completions, Completions, Responses and Embeddings. This is instrumentor coverage, not a claim that Vercel implements every OpenAI endpoint. Other endpoints retain their existing attribution behavior.

Open **Observe → my-gateway-app** to see the client span. Successful non-streaming Chat Completions include the response model and reported usage; absent usage is not zero. Content hiding does not remove model names or token counts and does not hide requests from Vercel. Tool definitions and custom attributes need their own privacy review.

This route does not expose gateway routing, upstream retries or billing. [Vercel Trace Drains](https://vercel.com/docs/ai-gateway/observability-and-spend/trace-drains) are a separate server-side source without prompts or completions. Do not sum their usage with application spans or layer multiple instrumentors on the same client call. This recipe does not configure a drain destination.

## Environment Variables

The following environment variables are required:
Expand Down
16 changes: 11 additions & 5 deletions typescript/packages/traceai_openai/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,14 @@
"types": "./dist/src/index.d.ts",
"exports": {
".": {
"types": "./dist/src/index.d.ts",
"import": "./dist/esm/index.js",
"require": "./dist/src/index.js"
"import": {
"types": "./dist/esm/index.d.ts",
"default": "./dist/esm/index.js"
},
"require": {
"types": "./dist/src/index.d.ts",
"default": "./dist/src/index.js"
}
}
},
"scripts": {
Expand Down Expand Up @@ -53,10 +58,11 @@
"@opentelemetry/semantic-conventions": "^1.34.0",
"@types/jest": "^29.5.12",
"jest": "^29.7.0",
"ts-jest": "^29.1.4",
"openai": "^4.52.7",
"rimraf": "^6.0.1",
"tsc-alias": "^1.8.16"
"ts-jest": "^29.1.4",
"tsc-alias": "^1.8.16",
"zod": "^3.25.76"
},
"keywords": [
"opentelemetry",
Expand Down
34 changes: 31 additions & 3 deletions typescript/packages/traceai_openai/src/instrumentation.ts
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,33 @@ import {

const MODULE_NAME = "openai";

/** Attribute the receiving service, not a model's author or a guessed fallback route. */
function providerForResource(resource: unknown): string {
try {
// OpenAI 4.0 uses `client`; later clients use `_client`. Resolve per request
// because different clients (or a changed baseURL) can share patched methods.
const owner = resource as {
_client?: { baseURL?: unknown };
client?: { baseURL?: unknown };
} | null;
const baseURL = (owner?._client ?? owner?.client)?.baseURL;
if (typeof baseURL === "string") {
const url = new URL(baseURL);
if (
(url.protocol === "https:" || url.protocol === "http:") &&
url.hostname.toLowerCase().replace(/\.$/, "") === "ai-gateway.vercel.sh"
) {
return "vercel";
}
}
} catch {
// Do not log the URL or the exception: either can contain credentials.
diag.debug("@traceai/openai: unable to resolve endpoint provider; retaining compatibility attribution");
}
// Preserve historical attribution for all other endpoints. Never export the URL.
return LLMProvider.OPENAI;
}

/**
* Flag to check if the openai module has been patched
* Note: This is a fallback in case the module is made immutable (e.x. Deno, webpack, etc.)
Expand Down Expand Up @@ -216,7 +243,7 @@ export class OpenAIInstrumentation extends InstrumentationBase{
[SemanticConventions.INPUT_MIME_TYPE]: MimeType.JSON,
[SemanticConventions.LLM_INVOCATION_PARAMETERS]:
JSON.stringify(invocationParameters),
[SemanticConventions.LLM_PROVIDER]: LLMProvider.OPENAI,
[SemanticConventions.LLM_PROVIDER]: providerForResource(this),
[SemanticConventions.GEN_AI_OPERATION_NAME]: "chat",
...getLLMInputMessagesAttributes(body),
...getLLMToolsJSONSchema(body),
Expand Down Expand Up @@ -310,7 +337,7 @@ export class OpenAIInstrumentation extends InstrumentationBase{
[SemanticConventions.LLM_MODEL_NAME]: body.model,
[SemanticConventions.LLM_INVOCATION_PARAMETERS]:
JSON.stringify(invocationParameters),
[SemanticConventions.LLM_PROVIDER]: LLMProvider.OPENAI,
[SemanticConventions.LLM_PROVIDER]: providerForResource(this),
[SemanticConventions.GEN_AI_OPERATION_NAME]: "text_completion",
...getCompletionInputValueAndMimeType(body),
[SemanticConventions.RAW_INPUT]: safelyJSONStringify(body) ?? "",
Expand Down Expand Up @@ -396,6 +423,7 @@ export class OpenAIInstrumentation extends InstrumentationBase{
[SemanticConventions.INPUT_MIME_TYPE]: isStringInput
? MimeType.TEXT
: MimeType.JSON,
[SemanticConventions.LLM_PROVIDER]: providerForResource(this),
[SemanticConventions.GEN_AI_OPERATION_NAME]: "embeddings",
...getEmbeddingTextAttributes(body),
[SemanticConventions.RAW_INPUT]: safelyJSONStringify(body) ?? "",
Expand Down Expand Up @@ -473,7 +501,7 @@ export class OpenAIInstrumentation extends InstrumentationBase{
[SemanticConventions.INPUT_MIME_TYPE]: MimeType.JSON,
[SemanticConventions.LLM_INVOCATION_PARAMETERS]:
JSON.stringify(invocationParameters),
[SemanticConventions.LLM_PROVIDER]: LLMProvider.OPENAI,
[SemanticConventions.LLM_PROVIDER]: providerForResource(this),
[SemanticConventions.GEN_AI_OPERATION_NAME]: "chat",
...getResponsesInputMessagesAttributes(body),
...getLLMToolsJSONSchema(body),
Expand Down
5 changes: 5 additions & 0 deletions typescript/packages/traceai_openai/test/moduleTypes.cts
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
import OpenAI from "openai";
import { OpenAIInstrumentation } from "@traceai/openai";

// Keep the existing CommonJS entry point and declaration working.
new OpenAIInstrumentation().manuallyInstrument(OpenAI);
5 changes: 5 additions & 0 deletions typescript/packages/traceai_openai/test/moduleTypes.mts
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
import OpenAI from "openai";
import { OpenAIInstrumentation } from "@traceai/openai";

// ESM must resolve the ESM OpenAI declaration, not its nominally distinct CJS type.
new OpenAIInstrumentation().manuallyInstrument(OpenAI);
Loading
Loading