Repository navigation
feat(examples): Perplexity Agent API recipe on traceai-openai via OpenAI SDK Responses (TH-8319) - #239
Merged
Conversation
… SDK Responses) Adds python/examples/perplexity/: the official openai SDK's Responses interface pointed at https://api.perplexity.ai/v1 (Perplexity's Agent API; /v1/responses is the documented alias of /v1/agent), traced by the existing traceai-openai Responses wrapper. No provider package. Perplexity ended Sonar Chat Completions support on 27 September 2026, so the recipe targets the Agent API rather than chat.completions. - check_base_url refuses the old Sonar root (https://api.perplexity.ai), the endpoint paths /v1/sonar and /v1/agent, and the private-preview Router API (/router, /router/v1), including upper-case and trailing-dot host spellings; it never rewrites a URL. main() exits 2 before tracing on a refused URL or a missing model or key. - Perplexity key only on the OpenAI client; Future AGI keys only on the tracer. - Fixtures use the Agent API shape with the non-OpenAI search_results output item. Pinned: the SDK parses it and output_text works (with Pydantic serializer warnings); one Response span per call with the requested model on non-streamed, streamed and 401 spans; provider label openai; usage from the response / response.completed; search-result URLs and snippets reach output.value on non-streamed calls only, and FI_HIDE_OUTPUTS removes them (with controls). No citation spans; no usage.cost mapping. - Loopback only: MockTransport, a local fake, the shared harness receiver and a socket guard with a negative control. approved: Nikhil 2026-10-03 blanket Refs: TH-8319
… privacy
pr-reviewer t_8edece5c raised CHANGES_REQUESTED on four P2 findings:
- R1: on a successful Responses call, traceai-openai records the model
the response returns (_attributes/_responses_api.py:132), not the
request model; failed calls keep the request model. Fixtures now return
a different synthetic dated model. Tests pin the returned model on the
non-streamed and streamed spans, the requested model on the 401 span,
and the requested model in gen_ai.request.parameters on all three.
The README states this as current traceai-openai behaviour.
- R2: Perplexity documents presets through the OpenAI SDK
(extra_body={"preset": ...}). The README is corrected and lists preset
requests as untested. One MockTransport preset fixture pins the
observed result: one span with the response model, and parameters {}
(preset absent).
- R3: the fixtures follow the vendor's documented Agent API response
shape: nested usage.cost, input/output/tool-call token details,
search_results.queries and last_updated, item ids and instructions.
Tests pin the cache-read and reasoning usage attributes and the raw
nested cost round-trip, and keep a usage-without-details case.
- R4: on non-streamed calls, output.value carries the prompt-derived
search queries and the response's instructions. FI_HIDE_INPUTS leaves
them; FI_HIDE_OUTPUTS removes them. Both are pinned with controls and
documented in Privacy. The streamed spans carry neither in the fixture.
P3: refuse /v1/responses as a base URL, the old chat-completions paths,
http on the Perplexity host, a query or fragment there, and credentials
in any URL. Each refusal is checked through main() (exit 2 before
tracing). Also: the Sonar wording matches the source, the Pydantic
UserWarning is pinned, a first-screen sentence on citations, and
verbose=False in the snippet.
RED (new tests and fixtures against the old app and README): 124 failed.
GREEN: 153 passed (twice).
approved: Nikhil 2026-10-03 blanket
Refs: TH-8319
The exact-head matrix at 09eada6 failed one test on the openai 1.69.0 floor cell (cell 5): test_preset_fixture. That SDK requires `model` on responses.create, so a preset-only call raises TypeError in the SDK. The test now pins both observed behaviours: - newer SDKs: one request and one span with the response model, and parameters {}; - older SDKs: TypeError naming `model`, with no request and no span. The README states the older-SDK behaviour, pinned by the README test. 153 passed on openai 3.24.0; the preset and README tests pass on 1.69.0. approved: Nikhil 2026-10-03 blanket Refs: TH-8319
main() printed str(KeyError), so running without PERPLEXITY_MODEL or PERPLEXITY_API_KEY showed only "'PERPLEXITY_MODEL'". Print "Set <VAR> before running this example." like the sibling recipes, and tighten the configuration test to require the instruction, not just the name. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… provider/model fix The recipe pinned traceai-openai 0.1.10 behaviour: gen_ai.provider.name is always "openai" and streamed/failed spans carry no gen_ai.request.model. #249 labels known vendor hosts and records the requested model, so these pins would break whichever PR merged second. Accept both behaviours (and the related Responses/embedding output changes where this recipe checks them), and say in the README which version does what. Passes against published traceAI-openai 0.1.10 and against #249. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Honor FI_PROJECT_NAME, reject malformed vendor base URLs, clarify stream privacy and usage, and pin the reviewed regression cases. Co-Authored-By: Claude Code <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the Perplexity Agent API recipe at
python/examples/perplexity/for TH-8319 / TH-8103, using the existingtraceAI-openaiinstrumentor and the official OpenAI SDK's Responses interface. No provider-specific package is introduced.https://api.perplexity.ai/v1.POST https://api.perplexity.ai/v1/responses, the documented alias of/v1/agent.--prompt, and env/CLI model selection.FI_PROJECT_NAME, defaulting toperplexity-agent-api, in both the app and README snippet./v1or/v1/. Endpoint paths, repeated slashes, encoded separators, queries/fragments, malformed hosts/ports, Unicode host spellings, whitespace/control characters and embedded credentials are refused. Valid proxy/loopback URLs retain their supplied spelling.FI_HIDE_OUTPUTSis needed for search-derived output privacy, including streams without answer text.Merged current
devnormally into this branch; no rebase or force-push.Vendor references
Checked 2026-10-28:
The README's runnable model is
openai/gpt-6-luna, listed at $0.10/M input and $0.50/M output for small contexts. The recipe selects a model directly rather than a preset.Verification at exact head
54d6120c5d417e7c3f1b2240d3d3ecf06d9c6431Local isolated contracts
Python 3.11.12; each cell: 187 passed, exit 0.
traceAI-openai==0.1.10+fi-instrumentation-otel==1.1.0Tests cover guarded loopback and MockTransport requests, key separation, returned model IDs, usage present/absent/detail variants, input/output masking, authentication errors, preset behavior at the pinned SDK floor, malformed base URLs,
FI_PROJECT_NAME, and output privacy for streams with no answer text.Real Perplexity → Future AGI Cloud
Installed the README's published pins into a fresh virtual environment. Separately built the two packages as wheels from the exact head and installed them into another fresh environment. Ran the recipe's
main()flow withopenai==3.24.0in both environments, with tiny prompts requesting onlyOK:FI_HIDE_INPUTS=true, with a unique input marker.FI_HIDE_OUTPUTS=true, with the visible response as control.BadRequestErrorand an ERROR span.Both cells used
openai/gpt-6-lunaat the endpoint above. All 10 directly exported traces were read back through the authenticated Future AGI Cloud API and checked against their locally captured IDs: one rootResponseLLM span per call, matching span IDs, no child/parent mismatch, correct model/provider, exact input/output/total token counts on successful calls, ERROR status and exception event for invalid-model calls, and expected content redaction. Cloud trace payloads had zero credential-value or local-filesystem-path hits.Provider labels differ as documented: published 0.1.10 records
openai; exact-head source recordsperplexity. Failed calls retain the invalid requested model and omit usage attributes; Cloud summary counters display zero tokens for those failures.1a0fced6-e103-4868-a3ee-2a89468851809999a2b9-d1f5-4801-bef7-0970393f08ca6cbc3bb3-5955-4b03-b2cd-63468ad2194f47d0df22-a5fb-477f-b19f-229cd169213ca98ded15-e255-453e-870e-1a866861abab053184df-64df-486c-be32-22c3ec6df6fb201938d3-f1f1-4d78-ac33-9598d238352c10e73023-5667-4173-b6bd-6c7b6a1d18fa53c2e6f3-d285-4a61-ad4a-824147dad5f7202445fd-52b0-4eeb-9a63-0a0a919cd5d8Error diagnostics caveat: default OpenTelemetry exception events contain local stack paths before collection. The Cloud readback API returned event names/timestamps without those event attributes, so the zero-path statement above is limited to the returned Cloud payloads. Current source core supports
FI_HIDE_ERROR_DETAILS=truefor suppressing error messages/stacks; published core 1.1.0 does not. Input/output masking alone is not a promise to sanitize every possible provider error or output echo.Exact-head CI
All workflows completed successfully at
54d6120c5d417e7c3f1b2240d3d3ecf06d9c6431:Scope
This recipe covers model-based Agent API Responses calls. Native Perplexity SDKs, preset selection in the CLI, Search/Router/Embeddings APIs, async/background runs, citation/retriever spans and dedicated
usage.costmapping remain outside this recipe. Search-result shapes, empty-text stream fallback and nested cost data are covered by synthetic fixtures; the tiny real calls did not invoke a search tool.🤖 Generated with Claude Code