Skip to content

fix(orcarouter): use released provider-aware tracing - #436

Open
nik13 wants to merge 1 commit into
devfrom
fix/TH-8307-orcarouter-released-recipe
Open

nik13 wants to merge 1 commit into
devfrom
fix/TH-8307-orcarouter-released-recipe

Conversation

@nik13

@nik13 nik13 commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fixes TH-8307's OrcaRouter recipe to install the already-published traceAI-openai==0.1.11 instead of 0.1.10. The old pin labels hosted OrcaRouter spans as openai and drops model names on streamed/failed calls. No new package, runtime patch, version bump, or package release is needed.

  • Keep the shared OpenAI-compatible instrumentor and the existing application entry points.
  • Make published-package CI install each recipe's actual requirements instead of hard-coding the obsolete instrumentor. Every other recipe's three dependency pins are unchanged.
  • Require orcarouter attribution at the documented host and requested models on stream/error spans, including with invocation parameters hidden. Loopback custom endpoints still assert openai.
  • Replace obsolete caveats and test-diary prose with a runnable customer setup, privacy settings, and streamed-usage guidance. Execute the README Python snippet in an offline regression test.
  • Select the MockTransport implementation used by the installed OpenAI SDK so tests work across the 1.x floor and current 3.x SDK.

Verification

All runs use synthetic responses and a local OTLP/HTTP receiver, with outbound network denied except localhost. No vendor/model API or hosted Future AGI call was made.

OpenAI SDK Released traceAI-openai 0.1.11 + FI core 1.1.0 Repository source
1.69.0 (floor) 34 passed 34 passed
3.24.0 (recipe pin) 34 passed 34 passed
3.42.0 (current) 34 passed 34 passed

The seven targeted provider/model regression cases fail with the old 0.1.10 instrumentor at each SDK version above (21 expected failures across floor, recipe pin, and current). Independently reproduced: sync/stream/error spans all say openai, and stream/error model attributes are missing when request parameters are hidden. The corrected published wheel passes the full contract suite, including OTLP wire checks, usage omission/final-chunk usage, content masking, credential separation, and CLI subprocesses.

Re-run the published recipe contract from the repository root after installing the recipe requirements plus pytest jsonschema 'wrapt<2', inside a network-denied sandbox that allows localhost:

PYTHONPATH="python/examples/orcarouter/src:python/tests" \
  python -m pytest python/examples/orcarouter/tests \
  --noconftest -p no:cacheprovider -o addopts='' -q

For repository source, add python:python/frameworks/openai to PYTHONPATH. Replace the SDK pin for floor/current comparisons.

Final-head CI (4d175dec5c69ce79c69087391dd657a424756362) completed with 32 successful checks, 4 intentionally skipped matrix checks, and no failures or pending checks. All 24 OpenAI-compatible recipe jobs passed, including all three OrcaRouter jobs. Recipe workflow: https://github.com/future-agi/traceAI/actions/runs/41223724169

git diff --check passes. No applicable Ruff configuration, migrations, frontend contract changes, or product E2E harness. Self-review checked the final diff against internal minimal-code, error-handling, PR, naming, and scenario-test standards. Human review is still required.

E2E coverage

not applicable (no e2e/ harness) · flows hit: n/a · marker: n/a · verdict: not applicable

References and rollout

Merge the recipe fix normally; customers can already install the corrected package. No docs-site dependency, deployment ordering, migration, merge, tag, or registry publication is part of this change. Hosted routing, billing, and a product screenshot were not verified here.

🤖 Generated with Claude Code

Install traceAI-openai 0.1.11 in the recipe and use recipe requirements in published CI. Require correct provider/model attributes on the OTLP wire and replace obsolete setup caveats.

Co-Authored-By: Claude Code <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant