Contract drift audit 2026-09-21 - #35
Merged
Merged
Conversation
Per-provider summary:
- openai-compat: added stream_options.include_obfuscation and prompt_cache_key
request fields, usage.{prompt,completion}_tokens_details response fields, and
the 429 slow_down / 503 server_is_overloaded error-code split (all
NEW-CAPABILITY, none currently read by the adapter). Bot-walled
platform.openai.com pages re-verified via the developers.openai.com mirrors.
- openai: resolved the standing "VERIFY on next drift run" prompt-caching item
— cached tokens confirmed still counted in usage.input_tokens, threshold is
1,024 tokens for GPT-5.6+. Added cache_write_tokens, expanded server-side
tool list, async tool calling/mid-turn steering, /models shutdown_date, and
the same error-code split as openai-compat (all NEW-CAPABILITY). Corrected
the changelog source URL, which now 301s to developers.openai.com.
- openrouter: fixed a dead source URL (list-available-models -> the
list-all-models-and-their-properties path) — DRIFT, link only, response
shape unchanged. Added the new /models filter params, X-OpenRouter-Title/
X-OpenRouter-Categories headers, and OpenRouter's new Responses-shaped
endpoint as NEW-CAPABILITY.
- azure-foundry: the /reference upstream source has narrowed to image/audio
only; added the actual current chat-completions reference URL and
reconfirmed max_completion_tokens and reasoning_effort (now includes
xhigh) directly against it. Confirmed the /openai/v1 surface is GA (not
preview) and that both *.openai.azure.com and *.services.ai.azure.com
base_url forms are valid. Noted Microsoft's ai-foundry -> foundry doc path
rebrand (old links still resolve, not urgent) and the same
prompt_cache_key/usage-details gaps as openai-compat.
All four snapshots were never_live_verified before this run; findings above
are the expected first-pass real drift, not noise. No adapter code changed —
every field addition here is a proposed follow-up work item.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…sing The audit filed `usage.prompt_tokens_details.cached_tokens` and `usage.completion_tokens_details.reasoning_tokens` as unread on both the openai-compat and azure-foundry surfaces, and openai-compat.md went on to say cache-aware pricing has no read path at all. Both are already wired: `OpenAICompatAdapter._parse_usage` puts them in `Usage.cache_read_tokens` and `Usage.reasoning_tokens`, `capabilities/pricing.py` reprices the cached share against `cache_read_per_1m`, and `AzureFoundryAdapter` inherits that parse rather than overriding it. A snapshot is the record a later reader trusts about what the adapter does today, so a gap recorded here that does not exist invites the work to be done twice. The genuinely unread fields — the audio and prediction breakdowns, and Azure's `annotations[].url_citation` — stay listed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Audited the 4 snapshots selected by the deterministic stage
(
drift-report/contract-drift-report.json, budget 4, allnever_live_verified: true):openai-compat,openai,openrouter,azure-foundry. Per contracts/DRIFT-CHECK.md.Summary table
Findings (most severe first)
openai:
platform.openai.com/docs/changelogmoved [DRIFT]https://platform.openai.com/docs/changelogas a source.https://developers.openai.com/api/docs/changelog(fetched 2026-09-21, content unchanged in substance).Upstream sourcesto record the redirect and the new URL. Applied.azure-foundry:
/referenceupstream source no longer documents chat/embeddings [DRIFT]learn.microsoft.com/en-us/azure/ai-foundry/openai/referenceas the source for the chat-completions wire contract.https://learn.microsoft.com/en-us/rest/api/microsoft-foundry/azureopenai/chatfor chat completions, embeddings, and "all other operations."max_completion_tokens,reasoning_effort, etc.) were not actually checkable against the cited page anymore — they still happen to be correct, reconfirmed against the real page.openrouter:
list-available-modelssource URL is dead [DRIFT]https://openrouter.ai/docs/api-reference/list-available-models.source_checksand a live fetch 2026-09-21). Corrected URL:https://openrouter.ai/docs/api/api-reference/models/list-all-models-and-their-properties— the response shape (context_length,pricing.{prompt,completion},supported_parameters) is unchanged at the new location.openai: prompt-caching "VERIFY on next drift run" item — resolved
prompt_tokens/input_tokensand the caching threshold were both explicitly unverified.developers.openai.com/api/docs/guides/prompt-caching, fetched 2026-09-21): cached tokens are confirmed still included inusage.input_tokens, broken out atusage.input_tokens_details.cached_tokens. Threshold is 1,024 visible input tokens for GPT-5.6+; earlier models vary by request shape (documented as such, not a gap in our check).cache_write_tokensfield (Prompt Cache Diagnostics GA'd 2026-09-08) that isn't read by the adapter.cache_write_tokens/ diagnostics-endpoint support is a proposed adapter work item, not applied.NEW-CAPABILITY findings (not urgent; feed the capability catalog / roadmap)
stream_options.include_obfuscation,prompt_cache_key/prompt_cache_retention, and responseusage.{prompt,completion}_tokens_details.*breakdowns exist on the wire and aren't read. Azure additionally exposesparallel_tool_calls,safety_identifier,store,modalities,prediction, and message-levelannotations[].url_citation.codevalues (slow_downfor 429,server_is_overloadedfor 503) — both already fall inside the existing retryable-status set, so no break, just finer-grained typing available if wanted.reasoning.effortchanges, and expanded server-side tools (file_search,computer_use,image_generation, custom tools) beyond theweb_search/code_interpreterpair already documented.GET /v1/modelsresponses now include ashutdown_datefield per model./modelsgained filter/sort query params (min/max_intelligence_index,agentic_index,coding_index,tool_success_rate,output_price,age_days); attribution headers gainedX-OpenRouter-Categories(andX-OpenRouter-Titleis now the canonical name, withX-Titleas an accepted alias); OpenRouter also GA'd a Responses-API-shaped endpoint (out of scope — this adapter only implements Chat Completions)./openai/v1surface is now GA (resolves a standing Watchlist item), that both*.openai.azure.comand*.services.ai.azure.combase URL forms are valid, and thatreasoning_effortnow acceptsxhigh.Unverifiable
response.output_text.delta/response.completedstill exist but didn't enumerate the rest. Needs a more targeted fetch next run.platform.openai.com/docs/api-reference/{responses,models,chat}remain bot-walled (403) — verified instead via thedevelopers.openai.commirrors, the same substitution the embeddings section already used.: OPENROUTER PROCESSINGSSE keep-alive comment framing and the 402 insufficient-credits error shape weren't re-fetched this run; not contradicted, just not re-confirmed.Left unapplied (adapter/code follow-up, out of scope for this workflow)
usage.{prompt,completion}_tokens_details.*/cache_write_tokensfor cache-aware pricing on the openai-compat, openai, and azure-foundry adapters.prompt_cache_keyfor cache-affinity hints.stream_options.include_obfuscationinignored_parametersinstead of silently dropping it.file_search,computer_use,image_generation, custom tools).GET /v1/modelsshutdown_datesurfaced through model discovery./modelsfilter params and theX-OpenRouter-Categoriesheader.Proposed contract edits
All applied directly in this PR, each with an inline citation and date. See the diff.
🤖 Generated with Claude Code