Skip to content

feat(examples): Baseten Model APIs recipe on traceai-openai (TH-8310) - #234

Merged
nik13 merged 10 commits into
devfrom
feat/th-8310-baseten
Oct 28, 2026
Merged

nik13 merged 10 commits into
devfrom
feat/th-8310-baseten

Conversation

@nik13

@nik13 nik13 commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds a Baseten Model APIs Chat Completions recipe using the official OpenAI Python SDK and the shared traceai-openai instrumentor. Dedicated deployments, the Anthropic Messages beta, and System One decision models remain outside this recipe.

  • Configures https://inference.baseten.co/v1 with a separate BASETEN_API_KEY; Future AGI credentials stay with the tracer.
  • Supports synchronous and streamed responses, --model / BASETEN_MODEL, and FI_PROJECT_NAME with the application's baseten-openai default.
  • Validates vendor base URLs before tracing or network activity; rejects endpoint paths, malformed host spellings, URL credentials, and unsupported Baseten deployment endpoints.
  • Documents privacy flags, session context, model catalog selection, per-token pricing, and version-specific provider/model attributes.
  • Adds offline contracts and all three Baseten CI cells. The development merge preserves every existing recipe, including Perplexity and Qwen.
  • Labels the usage-absent default-stream example explicitly as an offline fixture.

Exact head and local contracts

Head: 05c51dbfee45c70ab6c1718d5ffdc4d5bb20f936.

Normally merged development commit 536ec49947472fb2a68d8b253e0148b4b69af34b; no history rewrite.

Python OpenAI SDK traceAI packages Result
3.11 3.24.0 Repository source 58 passed
3.11 1.69.0 Repository source 58 passed
3.11 3.24.0 traceAI-openai==0.1.10, fi-instrumentation-otel==1.1.0 58 passed

The workflow plan contains eight recipes and 24 cells, including Baseten, Perplexity, and Qwen.

Live Baseten exercise — 2026-10-28

A fresh project installed the README's exact package pins: openai==3.24.0, traceAI-openai==0.1.10, and fi-instrumentation-otel==1.1.0. Recipe files were copied byte-for-byte from the head above.

The authenticated catalog listed openai/gpt-oss-120b. Six Chat Completions requests total were sent: synchronous, streaming, deliberately invalid model, FI_HIDE_INPUTS, FI_HIDE_OUTPUTS, and one additional synchronous request. The harness capped every request at max_tokens=32, set reasoning_effort=low, and disabled SDK retries. Five requests returned HTTP 200; the invalid-model request returned HTTP 404 with {"error":"please check the model you provided"}. No billing or credit error occurred.

Usage across the five successful requests was 449 input tokens + 106 output tokens = 555 total tokens. Streaming totals use the final cumulative usage event rather than summing repeated events. At the catalog prices, the arithmetic estimate is $0.0000979, not an account billing statement.

Future AGI Cloud readback

Observe project: th8103-baseten-r6-20261028.

Case Trace ID Result
Synchronous 33c2dfc8-c936-4daa-b281-4032993a2940 One root LLM span, model and 86/18/104 usage
Streaming 36c868e0-8c37-4621-80b5-52a092241c6b Accumulated output, final 86/18/104 usage
Hide inputs c04644e8-c383-4f0f-aaa7-cd0906a22f58 Prompt marker absent from the complete Cloud payload; input messages removed
Hide outputs 5f534945-17ae-4759-b52f-fb4a7e3adc16 Actual response content absent from output fields; output messages removed
Additional synchronous 3138858f-9224-465c-89e9-c18fc3bff5f6 Direct default-exporter path; model and 86/18/104 usage

The initial test observer replaced fi-core's default processor. The additional synchronous request verified direct export after correcting the observer attachment. Four earlier, genuine captured spans were then exported through the pinned HTTP exporter without further inference; their captured fields round-tripped exactly, with the microsecond timestamp precision retained by Span.to_json. They are not claimed as automatic exports from the original invocation.

All five Cloud traces have the original span IDs and no parent span. Their openai provider label is the documented 0.1.10 behavior. The streamed model remains in request parameters rather than a dedicated model attribute. This live Baseten default stream included usage, and the instrumentor exported the final totals. The Cloud payload scans found no populated credential values or absolute local filesystem paths.

Remaining gate — keep held

The real invalid-model response produced a local ERROR span and exception event, but the pinned fi-instrumentation-otel==1.1.0 exception stacktrace includes two absolute local dependency filenames. That span was withheld from Cloud rather than exporting those filenames. The path-safe Cloud ERROR-readback gate therefore remains open. The six-request budget is exhausted; no additional inference was sent. Resolve path-safe exception export before completing that gate.

Exact-head CI

All three runs succeeded at 05c51dbfee45c70ab6c1718d5ffdc4d5bb20f936 (31 successful checks, four skipped):

🤖 Generated with Claude Code

nik13 added 3 commits October 5, 2026 18:05
Add python/examples/baseten: trace Baseten Model APIs Chat Completions made
with the official openai SDK (base_url https://inference.baseten.co/v1) by
using the existing traceai-openai instrumentor. No Baseten package.

- src/app.py: register() + OpenAIInstrumentor before the client; the
  Baseten key goes only to the OpenAI client, Future AGI keys only to the
  tracer. check_base_url() refuses the Anthropic-beta root (no /v1) and
  dedicated-deployment hosts (model-*.api.baseten.co); it never rewrites.
- tests: loopback contract on the shared harness Receiver: request URL and
  bearer key at the documented host (MockTransport), one LLM span, model,
  usage (omitted when absent), provider label openai (D2), project resource
  and FI headers, no vendor key in any exported field, streaming, 401 error
  span, FI_HIDE_INPUTS/OUTPUTS with control, using_session, app subprocess
  under a socket guard, refused URLs exit 2 before tracing.
- Pins current traceai-openai behaviour: streamed and failed calls have no
  model attribute; the default stream has no usage attributes.
- README: install, configure, run, code, what you see, Baseten specifics,
  privacy, limits, tests and tested versions.

approved: Nikhil 2026-10-03 blanket
Refs: TH-8310
…EADME)

From pr-reviewer t_6453b6cd (APPROVED, P3 follow-ups):
- R1: check_base_url compares the host without an absolute-FQDN trailing
  dot, so https://inference.baseten.co. and model-*.api.baseten.co. are
  refused too (the URL is still returned/refused unchanged, never
  rewritten). Tests pin trailing-dot, upper-case, userinfo and port
  variants; the two trailing-dot cases failed before this change.
- R2: test_hide_outputs gains a non-hidden control.
- R3: README names gen_ai.request.model and notes the requested model stays
  in gen_ai.request.parameters on streamed and failed spans; tests pin it.
- R4: requirements.txt comment no longer mentions a worktree.
- R6: test_main_rejects_url_before_tracing replaces a vacuous fake-server
  check with a client-constructor spy.

approved: Nikhil 2026-10-03 blanket
Refs: TH-8310
The chat fixture echoed the request model, so a green span could not show
that traceai-openai stores completion.model. One installed-wheel case now
returns a synthetic response id, asserts the OpenAI chat.completion shape,
and checks the OTLP span stores that id while the request body keeps the
request id. No live vendor call. Host rules are unchanged.

RED: response.model equalled the request id.
GREEN: the new case passes on the installed traceAI-openai 0.1.10 wheel.

approved: Nikhil 2026-10-03 blanket
Refs: TH-8310
@nik13
nik13 requested a review from NVJKKartik October 10, 2026 08:42
@nik13

nik13 commented Oct 10, 2026

Copy link
Copy Markdown
Contributor Author

Test pin only, not a merge. Head 0cd35c7 adds one installed-wheel case where the chat fixture returns a synthetic response model different from the request slug and the OTLP span stores that id. App behavior is unchanged. The existing video is at 34ed9de and is not exact-head for this commit. No live Baseten call. Review requested from NVJKKartik; this comment is not an approval.

nik13 and others added 7 commits October 11, 2026 21:32
…ovider/model fix

The recipe pinned traceai-openai 0.1.10 behaviour: gen_ai.provider.name is
always "openai" and streamed/failed spans carry no gen_ai.request.model.
#249 labels known vendor hosts and records the requested
model, so these pins would break whichever PR merged second. Accept both
behaviours (and the related Responses/embedding output changes where this
recipe checks them), and say in the README which version does what.

Passes against published traceAI-openai 0.1.10 and against #249.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
setup_tracing() passed project_name or "baseten-openai" to register(),
so the default was always truthy and FI_PROJECT_NAME was never read:
`python src/app.py` always exported to baseten-openai. The Anannas
recipe reads FI_PROJECT_NAME with its name as the default; do the same
here. An explicit setup_tracing() argument still wins, and an unset or
empty FI_PROJECT_NAME keeps baseten-openai (never register()'s generic
default).

- README: FI_PROJECT_NAME in the Configure table and the Run section;
  the Code snippet reads it the same way as src/app.py.
- Tests: setup_tracing() default / empty / env / explicit cases; the
  guarded app.py subprocess runs with FI_PROJECT_NAME unset and set; the
  README contract checks the new row and snippet.

Passes against repository source, published traceAI-openai 0.1.10 and
#249.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Reject malformed and non-base Baseten URLs before tracing, reject empty configuration, close streams, document token pricing and System One scope, and add Baseten to the three-cell recipe CI matrix.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Preserve Baseten, Perplexity, Qwen, and all existing recipe CI cells.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
@nik13
nik13 merged commit 62a56e0 into dev Oct 28, 2026
44 checks passed
@nik13
nik13 deleted the feat/th-8310-baseten branch October 28, 2026 11:04
jabrailkhalil pushed a commit to jabrailkhalil/traceAI that referenced this pull request Nov 1, 2026
Adds Baseten Model APIs (python/examples/baseten, merged in future-agi#234) to the
recipe tables in README.md and python/README.md, next to the Perplexity and
Qwen rows added in future-agi#291.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nik13 added a commit to jjcav84/traceAI that referenced this pull request Nov 2, 2026
…ses and embedding spans (TH-8402)

Live E2E of the OpenAI-compatible provider recipes (future-agi#234-future-agi#243) showed:
- streamed and failed Chat Completions spans had no gen_ai.request.model;
- gen_ai.provider.name was always "openai" (the request-level label even
  overwrote the existing Azure/Google host detection);
- non-streamed Responses output.value was Python str(dict), a string input
  was JSON-quoted, and the Observe preview showed `{}` as the answer because
  assistant text was only emitted as message.contents.N.message_content.text;
- embedding spans put the full vector into output.value as str(dict), had no
  span model, ignored FI_HIDE_EMBEDDING_VECTORS for the raw
  embedding.embeddings JSON, and skipped items without `index` (Gemini).

Fixes, test-first (python/frameworks/openai/tests, real openai SDK through
httpx.MockTransport and a real OTel SDK exporter; 41 tests, openai 1.69.0 and
3.24.0):
- record the requested model at span start; served model still wins on a
  successful non-streamed response;
- map the client's base_url host to a provider via a small explicit table
  (_providers.py); unknown hosts, proxies and loopback stay "openai";
- Responses: JSON output.value + application/json mime, led by output_text;
  plain-string input recorded as text; text-only assistant messages emitted
  as a flat message.content;
- embeddings: gen_ai.request.model = embedding model, compact JSON summary
  in output.value, hide-vectors strips the raw JSON too, index falls back to
  position;
- mime types follow the recorded value; async request path also reads
  Responses `input`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant