Skip to content

fix(openai): attribute Novita chat requests to their provider - #397

Merged
nik13 merged 3 commits into
devfrom
fix/openai-novita-provider
Nov 9, 2026
Merged

nik13 merged 3 commits into
devfrom
fix/openai-novita-provider

Conversation

@nik13

@nik13 nik13 commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fix Novita attribution in the existing Python traceAI-openai package by adding the exact documented host api.novita.ai -> novita to the shared provider map. Novita's Quickstart uses https://api.novita.ai/openai/v1/chat/completions.

  • Keep OpenAIInstrumentor, the shared OpenAI-compatible route, and existing privacy defaults. No new wrapper, package, dependency, public API, or release/version change.
  • Keep unknown hosts, customer proxies, subdomains and suffix look-alikes on openai. Host normalization remains case-insensitive and accepts trailing dots.
  • Preserve existing host-only semantics: HTTP and non-default ports on the genuine host still identify Novita; they do not become unknown providers. Full URL/authority strings passed directly to the hostname-only resolver fall back instead.
  • Add 239 regression cases: 15 direct hostname boundaries and 224 real-SDK/localhost OTLP-protobuf cases (14 URLs x sync/async x success/stream/stream-with-usage/error x default/hidden content).
  • Assert gen_ai.provider.name, absence of legacy gen_ai.system, requested/served model behavior, usage, error status, exactly one span, hidden prompt/output behavior, and absence of the synthetic vendor key from exported payloads.

Current base and head

  • Base merged without conflicts: 525919ead43c29f7184263eec0ba6c51a3b87dcf (dev).
  • Qualified/pushed head: 454fc2da277ecd675e35136118cd208247d3a1ee.
  • Final diff against that base: one runtime map entry and the Novita regression file. No changes to other provider entries; the Novita entry sits immediately after Mistral without reordering unrelated entries. CometAPI fix(openai): attribute CometAPI requests to their provider #348 is independent and touches the same map.

Independent qualification — 2026-11-09

All installs and tests for this qualification ran in E2B, not on the local machine. Python 3.12.16, candidate source core 1.3.0, and the exact pushed source were used. OpenAI has a declared minimum of 1.69.0 and no upper cap; 3.48.0 was the latest PyPI release checked for this run.

OpenAI SDK Complete OpenAI package suite Novita cases within that suite Current-base-map control
1.69.0 2,209 passed, 28 skipped, 0 failed 239 passed 82 failed, 157 passed
3.48.0 2,233 passed, 4 skipped, 0 failed 239 passed 82 failed, 157 passed

The four skips common to both versions are unsupported legacy-instrumentor Responses cases. The floor's additional 24 skips are unavailable callable credentials (8) and AsyncStream.aclose (16). None of the new Novita cases were skipped.

For the base control, only _providers.py was replaced by its exact version from the base SHA above, leaving the candidate tests and other source unchanged. Every one of the 82 failures at each bound was exactly openai instead of novita (80 wire cases and two direct-host cases); fallback cases stayed green. This demonstrates the regression without replaying unrelated historical changes.

Vendor HTTP uses httpx.MockTransport and synthetic credentials; telemetry crosses an actual localhost OTLP/HTTP protobuf receiver. Pytest ran with socket restrictions permitting only loopback and Unix sockets. No real Novita request, provider credential, live Future AGI ingestion, or live cost-parity claim is involved.

Reproduce in E2B or CI

From the repository root at the qualified head, use a separate environment for each SDK:

python3 -m venv "/tmp/openai-$SDK"
"/tmp/openai-$SDK/bin/python" -m pip install \
  -e python -e python/frameworks/openai -e python/frameworks/vllm \
  -e python/frameworks/deepseek -e python/frameworks/fireworks \
  "openai==$SDK" 'pytest==8.4.2' 'pytest-asyncio==0.24.0' \
  'pytest-mock==3.14.0' 'pytest-socket==0.7.0' httpx numpy
PYTHONDONTWRITEBYTECODE=1 FI_BASE_URL=http://127.0.0.1:9 \
FI_GRPC_URL=http://127.0.0.1:9 NO_PROXY='*' \
"/tmp/openai-$SDK/bin/python" -m pytest python/frameworks/openai/tests \
  -q -p no:cacheprovider --disable-socket \
  --allow-hosts=127.0.0.1,::1,localhost --allow-unix-socket

Run with SDK=1.69.0 and SDK=3.48.0. To reproduce the negative control in a disposable checkout, replace python/frameworks/openai/traceai_openai/_providers.py using git show 525919ead43c29f7184263eec0ba6c51a3b87dcf:python/frameworks/openai/traceai_openai/_providers.py, then run the same pytest command targeting only python/frameworks/openai/tests/test_novita_provider.py; restore the head file afterwards.

E2B receipt label: qw-traceai-397-bounds-454fc2da; exit 0; sandbox termination verified. The receipt includes exact revisions, per-bound dependency freezes, full logs and JUnit XML. git diff --check is clean.

GitHub CI and review

On 454fc2da277ecd675e35136118cd208247d3a1ee, 55 checks succeeded, three unrelated language-matrix checks were skipped, and none remained pending or failed.

Reviewed the actual final diff against the internal minimal-code, contract, failure-mode, testing, security and PR standards. No introduced blocking code findings. The applicable mechanical gate was the new test suite; no Ruff configuration applies to these package files, and there are no migration or frontend-contract changes. The existing PR has no linked ticket; no ticket or closing reference has been invented.

The PR is non-draft/ready for review. The normal independent-human-review requirement remains in place; this qualification is not an approval or a merge override.

Release and scope

Release the shared traceAI-openai package through the normal release-please process after merge. No publication was performed here, and no separate Novita package is introduced. Customer documentation should name the released wheel only after that wheel has been qualified. Reverting the single map entry restores the previous attribution without changing request routing or privacy behavior.

E2E coverage

not applicable (no e2e/ harness) · flows hit: n/a · marker: n/a · verdict: not applicable

🤖 Generated with Claude Code

nik13 and others added 3 commits October 31, 2026 08:05
Keep the shared OpenAI instrumentor and exact-host fallback behavior. Verify sync/async success, streams, usage and errors over local OTLP HTTP.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Keep Novita adjacent to Mistral in the explicit host map and cover
port/scheme normalization, look-alike fallbacks, and hidden content.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@nik13
nik13 merged commit 3738b98 into dev Nov 9, 2026
68 checks passed
@nik13
nik13 deleted the fix/openai-novita-provider branch November 9, 2026 23:32
nik13 added a commit that referenced this pull request Nov 10, 2026
… traceAI-langchain 0.2.2, Java 1.0.3 (#636)

* chore(release): bump core, OpenAI, LangChain and Java patch versions

Synchronize runtime versions, reactor parent references and release changelogs for #155, #348, #397, #393 and #425. Dependency floors are unchanged.

Co-Authored-By: Claude Code <noreply@anthropic.com>

* chore(release): list every change shipping in core 1.3.1, OpenAI 0.1.14 and Java 1.0.3

Core 1.3.1 also ships the PII-redaction scoping fixes (#195/#244) that sat under
Unreleased, and the 1.3.0 heading gets its release date. OpenAI 0.1.14 also
ships the Azure metadata (#154), Baseten (#541), Azure Government (#502),
OpenRouter host (#547), full multi-message input (#245) and exhausted-stream
changes. Java 1.0.3 also ships #543 and #605. Notes the remaining blank
OTEL-env Authorization limitation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Code <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant