Skip to content

feat(examples): Perplexity Agent API recipe on traceai-openai via OpenAI SDK Responses (TH-8319) - #239

Merged
nik13 merged 8 commits into
devfrom
feat/th-8319-perplexity
Oct 28, 2026
Merged

nik13 merged 8 commits into
devfrom
feat/th-8319-perplexity

Conversation

@nik13

@nik13 nik13 commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds the Perplexity Agent API recipe at python/examples/perplexity/ for TH-8319 / TH-8103, using the existing traceAI-openai instrumentor and the official OpenAI SDK's Responses interface. No provider-specific package is introduced.

  • SDK base URL: https://api.perplexity.ai/v1.
  • Request endpoint: POST https://api.perplexity.ai/v1/responses, the documented alias of /v1/agent.
  • CLI supports non-streamed and streamed responses, --prompt, and env/CLI model selection.
  • Honors FI_PROJECT_NAME, defaulting to perplexity-agent-api, in both the app and README snippet.
  • Validates configuration before tracing: Perplexity requires HTTPS and exactly /v1 or /v1/. Endpoint paths, repeated slashes, encoded separators, queries/fragments, malformed hosts/ports, Unicode host spellings, whitespace/control characters and embedded credentials are refused. Valid proxy/loopback URLs retain their supplied spelling.
  • README explains returned-vs-requested model behavior, provider labels, usage details, raw search output, and the empty-text stream fallback. FI_HIDE_OUTPUTS is needed for search-derived output privacy, including streams without answer text.
  • Adds Perplexity to the shared recipe CI matrix alongside the five sibling recipes.

Merged current dev normally into this branch; no rebase or force-push.

Vendor references

Checked 2026-10-28:

The README's runnable model is openai/gpt-6-luna, listed at $0.10/M input and $0.50/M output for small contexts. The recipe selects a model directly rather than a preset.

Verification at exact head 54d6120c5d417e7c3f1b2240d3d3ecf06d9c6431

Local isolated contracts

Python 3.11.12; each cell: 187 passed, exit 0.

OpenAI SDK traceAI packages
3.24.0 repository source, matching CI's source import paths
1.69.0 repository source, SDK floor
3.24.0 published traceAI-openai==0.1.10 + fi-instrumentation-otel==1.1.0

Tests cover guarded loopback and MockTransport requests, key separation, returned model IDs, usage present/absent/detail variants, input/output masking, authentication errors, preset behavior at the pinned SDK floor, malformed base URLs, FI_PROJECT_NAME, and output privacy for streams with no answer text.

Real Perplexity → Future AGI Cloud

Installed the README's published pins into a fresh virtual environment. Separately built the two packages as wheels from the exact head and installed them into another fresh environment. Ran the recipe's main() flow with openai==3.24.0 in both environments, with tiny prompts requesting only OK:

  1. Non-streamed response.
  2. Streamed response consumed through completion.
  3. FI_HIDE_INPUTS=true, with a unique input marker.
  4. FI_HIDE_OUTPUTS=true, with the visible response as control.
  5. Invalid model: HTTP 400 / BadRequestError and an ERROR span.

Both cells used openai/gpt-6-luna at the endpoint above. All 10 directly exported traces were read back through the authenticated Future AGI Cloud API and checked against their locally captured IDs: one root Response LLM span per call, matching span IDs, no child/parent mismatch, correct model/provider, exact input/output/total token counts on successful calls, ERROR status and exception event for invalid-model calls, and expected content redaction. Cloud trace payloads had zero credential-value or local-filesystem-path hits.

Provider labels differ as documented: published 0.1.10 records openai; exact-head source records perplexity. Failed calls retain the invalid requested model and omit usage attributes; Cloud summary counters display zero tokens for those failures.

Case Published packages trace ID Exact-head wheels trace ID
Non-streamed 1a0fced6-e103-4868-a3ee-2a8946885180 9999a2b9-d1f5-4801-bef7-0970393f08ca
Streamed 6cbc3bb3-5955-4b03-b2cd-63468ad2194f 47d0df22-a5fb-477f-b19f-229cd169213c
Hide inputs a98ded15-e255-453e-870e-1a866861abab 053184df-64df-486c-be32-22c3ec6df6fb
Hide outputs 201938d3-f1f1-4d78-ac33-9598d238352c 10e73023-5667-4173-b6bd-6c7b6a1d18fa
Invalid model 53c2e6f3-d285-4a61-ad4a-824147dad5f7 202445fd-52b0-4eeb-9a63-0a0a919cd5d8

Error diagnostics caveat: default OpenTelemetry exception events contain local stack paths before collection. The Cloud readback API returned event names/timestamps without those event attributes, so the zero-path statement above is limited to the returned Cloud payloads. Current source core supports FI_HIDE_ERROR_DETAILS=true for suppressing error messages/stacks; published core 1.1.0 does not. Input/output masking alone is not a promise to sanitize every possible provider error or output echo.

Exact-head CI

All workflows completed successfully at 54d6120c5d417e7c3f1b2240d3d3ecf06d9c6431:

Scope

This recipe covers model-based Agent API Responses calls. Native Perplexity SDKs, preset selection in the CLI, Search/Router/Embeddings APIs, async/background runs, citation/retriever spans and dedicated usage.cost mapping remain outside this recipe. Search-result shapes, empty-text stream fallback and nested cost data are covered by synthetic fixtures; the tiny real calls did not invoke a search tool.

🤖 Generated with Claude Code

nik13 added 3 commits October 5, 2026 20:17
… SDK Responses)

Adds python/examples/perplexity/: the official openai SDK's Responses
interface pointed at https://api.perplexity.ai/v1 (Perplexity's Agent
API; /v1/responses is the documented alias of /v1/agent), traced by the
existing traceai-openai Responses wrapper. No provider package.

Perplexity ended Sonar Chat Completions support on 27 September 2026, so
the recipe targets the Agent API rather than chat.completions.

- check_base_url refuses the old Sonar root (https://api.perplexity.ai),
  the endpoint paths /v1/sonar and /v1/agent, and the private-preview
  Router API (/router, /router/v1), including upper-case and trailing-dot
  host spellings; it never rewrites a URL. main() exits 2 before tracing
  on a refused URL or a missing model or key.
- Perplexity key only on the OpenAI client; Future AGI keys only on the
  tracer.
- Fixtures use the Agent API shape with the non-OpenAI search_results
  output item. Pinned: the SDK parses it and output_text works (with
  Pydantic serializer warnings); one Response span per call with the
  requested model on non-streamed, streamed and 401 spans; provider label
  openai; usage from the response / response.completed; search-result
  URLs and snippets reach output.value on non-streamed calls only, and
  FI_HIDE_OUTPUTS removes them (with controls). No citation spans; no
  usage.cost mapping.
- Loopback only: MockTransport, a local fake, the shared harness receiver
  and a socket guard with a negative control.

approved: Nikhil 2026-10-03 blanket
Refs: TH-8319
… privacy

pr-reviewer t_8edece5c raised CHANGES_REQUESTED on four P2 findings:
- R1: on a successful Responses call, traceai-openai records the model
  the response returns (_attributes/_responses_api.py:132), not the
  request model; failed calls keep the request model. Fixtures now return
  a different synthetic dated model. Tests pin the returned model on the
  non-streamed and streamed spans, the requested model on the 401 span,
  and the requested model in gen_ai.request.parameters on all three.
  The README states this as current traceai-openai behaviour.
- R2: Perplexity documents presets through the OpenAI SDK
  (extra_body={"preset": ...}). The README is corrected and lists preset
  requests as untested. One MockTransport preset fixture pins the
  observed result: one span with the response model, and parameters {}
  (preset absent).
- R3: the fixtures follow the vendor's documented Agent API response
  shape: nested usage.cost, input/output/tool-call token details,
  search_results.queries and last_updated, item ids and instructions.
  Tests pin the cache-read and reasoning usage attributes and the raw
  nested cost round-trip, and keep a usage-without-details case.
- R4: on non-streamed calls, output.value carries the prompt-derived
  search queries and the response's instructions. FI_HIDE_INPUTS leaves
  them; FI_HIDE_OUTPUTS removes them. Both are pinned with controls and
  documented in Privacy. The streamed spans carry neither in the fixture.
P3: refuse /v1/responses as a base URL, the old chat-completions paths,
http on the Perplexity host, a query or fragment there, and credentials
in any URL. Each refusal is checked through main() (exit 2 before
tracing). Also: the Sonar wording matches the source, the Pydantic
UserWarning is pinned, a first-screen sentence on citations, and
verbose=False in the snippet.
RED (new tests and fixtures against the old app and README): 124 failed.
GREEN: 153 passed (twice).

approved: Nikhil 2026-10-03 blanket
Refs: TH-8319
The exact-head matrix at 09eada6 failed one test on the openai 1.69.0
floor cell (cell 5): test_preset_fixture. That SDK requires `model` on
responses.create, so a preset-only call raises TypeError in the SDK. The
test now pins both observed behaviours:
- newer SDKs: one request and one span with the response model, and
  parameters {};
- older SDKs: TypeError naming `model`, with no request and no span.
The README states the older-SDK behaviour, pinned by the README test.
153 passed on openai 3.24.0; the preset and README tests pass on 1.69.0.

approved: Nikhil 2026-10-03 blanket
Refs: TH-8319
@nik13
nik13 requested a review from NVJKKartik October 10, 2026 08:42
main() printed str(KeyError), so running without PERPLEXITY_MODEL or
PERPLEXITY_API_KEY showed only "'PERPLEXITY_MODEL'". Print
"Set <VAR> before running this example." like the sibling recipes, and
tighten the configuration test to require the instruction, not just the name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nik13 and others added 4 commits October 11, 2026 21:32
… provider/model fix

The recipe pinned traceai-openai 0.1.10 behaviour: gen_ai.provider.name is
always "openai" and streamed/failed spans carry no gen_ai.request.model.
#249 labels known vendor hosts and records the requested
model, so these pins would break whichever PR merged second. Accept both
behaviours (and the related Responses/embedding output changes where this
recipe checks them), and say in the README which version does what.

Passes against published traceAI-openai 0.1.10 and against #249.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Honor FI_PROJECT_NAME, reject malformed vendor base URLs, clarify stream privacy and usage, and pin the reviewed regression cases.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@nik13
nik13 merged commit 9fa4895 into dev Oct 28, 2026
38 checks passed
@nik13
nik13 deleted the feat/th-8319-perplexity branch October 28, 2026 09:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant