Repository navigation
feat(examples): Baseten Model APIs recipe on traceai-openai (TH-8310) - #234
Merged
Merged
Conversation
Add python/examples/baseten: trace Baseten Model APIs Chat Completions made with the official openai SDK (base_url https://inference.baseten.co/v1) by using the existing traceai-openai instrumentor. No Baseten package. - src/app.py: register() + OpenAIInstrumentor before the client; the Baseten key goes only to the OpenAI client, Future AGI keys only to the tracer. check_base_url() refuses the Anthropic-beta root (no /v1) and dedicated-deployment hosts (model-*.api.baseten.co); it never rewrites. - tests: loopback contract on the shared harness Receiver: request URL and bearer key at the documented host (MockTransport), one LLM span, model, usage (omitted when absent), provider label openai (D2), project resource and FI headers, no vendor key in any exported field, streaming, 401 error span, FI_HIDE_INPUTS/OUTPUTS with control, using_session, app subprocess under a socket guard, refused URLs exit 2 before tracing. - Pins current traceai-openai behaviour: streamed and failed calls have no model attribute; the default stream has no usage attributes. - README: install, configure, run, code, what you see, Baseten specifics, privacy, limits, tests and tested versions. approved: Nikhil 2026-10-03 blanket Refs: TH-8310
…EADME) From pr-reviewer t_6453b6cd (APPROVED, P3 follow-ups): - R1: check_base_url compares the host without an absolute-FQDN trailing dot, so https://inference.baseten.co. and model-*.api.baseten.co. are refused too (the URL is still returned/refused unchanged, never rewritten). Tests pin trailing-dot, upper-case, userinfo and port variants; the two trailing-dot cases failed before this change. - R2: test_hide_outputs gains a non-hidden control. - R3: README names gen_ai.request.model and notes the requested model stays in gen_ai.request.parameters on streamed and failed spans; tests pin it. - R4: requirements.txt comment no longer mentions a worktree. - R6: test_main_rejects_url_before_tracing replaces a vacuous fake-server check with a client-constructor spy. approved: Nikhil 2026-10-03 blanket Refs: TH-8310
The chat fixture echoed the request model, so a green span could not show that traceai-openai stores completion.model. One installed-wheel case now returns a synthetic response id, asserts the OpenAI chat.completion shape, and checks the OTLP span stores that id while the request body keeps the request id. No live vendor call. Host rules are unchanged. RED: response.model equalled the request id. GREEN: the new case passes on the installed traceAI-openai 0.1.10 wheel. approved: Nikhil 2026-10-03 blanket Refs: TH-8310
Contributor
Author
|
Test pin only, not a merge. Head 0cd35c7 adds one installed-wheel case where the chat fixture returns a synthetic response model different from the request slug and the OTLP span stores that id. App behavior is unchanged. The existing video is at 34ed9de and is not exact-head for this commit. No live Baseten call. Review requested from NVJKKartik; this comment is not an approval. |
…ovider/model fix The recipe pinned traceai-openai 0.1.10 behaviour: gen_ai.provider.name is always "openai" and streamed/failed spans carry no gen_ai.request.model. #249 labels known vendor hosts and records the requested model, so these pins would break whichever PR merged second. Accept both behaviours (and the related Responses/embedding output changes where this recipe checks them), and say in the README which version does what. Passes against published traceAI-openai 0.1.10 and against #249. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
setup_tracing() passed project_name or "baseten-openai" to register(), so the default was always truthy and FI_PROJECT_NAME was never read: `python src/app.py` always exported to baseten-openai. The Anannas recipe reads FI_PROJECT_NAME with its name as the default; do the same here. An explicit setup_tracing() argument still wins, and an unset or empty FI_PROJECT_NAME keeps baseten-openai (never register()'s generic default). - README: FI_PROJECT_NAME in the Configure table and the Run section; the Code snippet reads it the same way as src/app.py. - Tests: setup_tracing() default / empty / env / explicit cases; the guarded app.py subprocess runs with FI_PROJECT_NAME unset and set; the README contract checks the new row and snippet. Passes against repository source, published traceAI-openai 0.1.10 and #249. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Reject malformed and non-base Baseten URLs before tracing, reject empty configuration, close streams, document token pricing and System One scope, and add Baseten to the three-cell recipe CI matrix. Co-Authored-By: Claude Code <noreply@anthropic.com>
Preserve Baseten, Perplexity, Qwen, and all existing recipe CI cells. Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
jabrailkhalil
pushed a commit
to jabrailkhalil/traceAI
that referenced
this pull request
Nov 1, 2026
Adds Baseten Model APIs (python/examples/baseten, merged in future-agi#234) to the recipe tables in README.md and python/README.md, next to the Perplexity and Qwen rows added in future-agi#291. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nik13
added a commit
to jjcav84/traceAI
that referenced
this pull request
Nov 2, 2026
…ses and embedding spans (TH-8402) Live E2E of the OpenAI-compatible provider recipes (future-agi#234-future-agi#243) showed: - streamed and failed Chat Completions spans had no gen_ai.request.model; - gen_ai.provider.name was always "openai" (the request-level label even overwrote the existing Azure/Google host detection); - non-streamed Responses output.value was Python str(dict), a string input was JSON-quoted, and the Observe preview showed `{}` as the answer because assistant text was only emitted as message.contents.N.message_content.text; - embedding spans put the full vector into output.value as str(dict), had no span model, ignored FI_HIDE_EMBEDDING_VECTORS for the raw embedding.embeddings JSON, and skipped items without `index` (Gemini). Fixes, test-first (python/frameworks/openai/tests, real openai SDK through httpx.MockTransport and a real OTel SDK exporter; 41 tests, openai 1.69.0 and 3.24.0): - record the requested model at span start; served model still wins on a successful non-streamed response; - map the client's base_url host to a provider via a small explicit table (_providers.py); unknown hosts, proxies and loopback stay "openai"; - Responses: JSON output.value + application/json mime, led by output_text; plain-string input recorded as text; text-only assistant messages emitted as a flat message.content; - embeddings: gen_ai.request.model = embedding model, compact JSON summary in output.value, hide-vectors strips the raw JSON too, index falls back to position; - mime types follow the recorded value; async request path also reads Responses `input`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a Baseten Model APIs Chat Completions recipe using the official OpenAI Python SDK and the shared
traceai-openaiinstrumentor. Dedicated deployments, the Anthropic Messages beta, and System One decision models remain outside this recipe.https://inference.baseten.co/v1with a separateBASETEN_API_KEY; Future AGI credentials stay with the tracer.--model/BASETEN_MODEL, andFI_PROJECT_NAMEwith the application'sbaseten-openaidefault.Exact head and local contracts
Head:
05c51dbfee45c70ab6c1718d5ffdc4d5bb20f936.Normally merged development commit
536ec49947472fb2a68d8b253e0148b4b69af34b; no history rewrite.traceAI-openai==0.1.10,fi-instrumentation-otel==1.1.0The workflow plan contains eight recipes and 24 cells, including Baseten, Perplexity, and Qwen.
Live Baseten exercise — 2026-10-28
A fresh project installed the README's exact package pins:
openai==3.24.0,traceAI-openai==0.1.10, andfi-instrumentation-otel==1.1.0. Recipe files were copied byte-for-byte from the head above.The authenticated catalog listed
openai/gpt-oss-120b. Six Chat Completions requests total were sent: synchronous, streaming, deliberately invalid model,FI_HIDE_INPUTS,FI_HIDE_OUTPUTS, and one additional synchronous request. The harness capped every request atmax_tokens=32, setreasoning_effort=low, and disabled SDK retries. Five requests returned HTTP 200; the invalid-model request returned HTTP 404 with{"error":"please check the model you provided"}. No billing or credit error occurred.Usage across the five successful requests was 449 input tokens + 106 output tokens = 555 total tokens. Streaming totals use the final cumulative usage event rather than summing repeated events. At the catalog prices, the arithmetic estimate is $0.0000979, not an account billing statement.
Future AGI Cloud readback
Observe project:
th8103-baseten-r6-20261028.33c2dfc8-c936-4daa-b281-4032993a294036c868e0-8c37-4621-80b5-52a092241c6bc04644e8-c383-4f0f-aaa7-cd0906a22f585f534945-17ae-4759-b52f-fb4a7e3adc163138858f-9224-465c-89e9-c18fc3bff5f6The initial test observer replaced fi-core's default processor. The additional synchronous request verified direct export after correcting the observer attachment. Four earlier, genuine captured spans were then exported through the pinned HTTP exporter without further inference; their captured fields round-tripped exactly, with the microsecond timestamp precision retained by
Span.to_json. They are not claimed as automatic exports from the original invocation.All five Cloud traces have the original span IDs and no parent span. Their
openaiprovider label is the documented 0.1.10 behavior. The streamed model remains in request parameters rather than a dedicated model attribute. This live Baseten default stream included usage, and the instrumentor exported the final totals. The Cloud payload scans found no populated credential values or absolute local filesystem paths.Remaining gate — keep held
The real invalid-model response produced a local
ERRORspan and exception event, but the pinnedfi-instrumentation-otel==1.1.0exception stacktrace includes two absolute local dependency filenames. That span was withheld from Cloud rather than exporting those filenames. The path-safe CloudERROR-readback gate therefore remains open. The six-request budget is exhausted; no additional inference was sent. Resolve path-safe exception export before completing that gate.Exact-head CI
All three runs succeeded at
05c51dbfee45c70ab6c1718d5ffdc4d5bb20f936(31 successful checks, four skipped):🤖 Generated with Claude Code