Repository navigation
docs(gateways): Azure API Management and AI Gateway tracing (TH-8298) - #966
Merged
Merged
Conversation
Variant A: APIM as an OTLP egress (policy, stand-in, real traceAI client run). Variant B: AI Gateway tier traces through an OpenTelemetry Collector that names the Observe project (synthetic trace; not run against Azure). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…Consumption (2026-10-19) Variant A policy deployed unchanged on a real APIM Consumption instance into the hosted Future AGI cloud; variant B remains synthetic (AI Gateway tier monitoring is portal-only). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…le (TH-8298) The example committed seven .pyc files (CPython 3.13 / pytest 9.1.1 caches), which embed a local home path. Remove them and add a .gitignore for __pycache__, *.pyc, .venv and .pytest_cache, as the other gateway examples do. pytest (21 tests) still passes on Python 3.13. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… variant A run (TH-8298) The README still said nothing had touched Azure, the policy was never deployed and listed the policy deploy and hosted backend as open checks. The page's Live vendor verification section records them as confirmed on a Consumption instance. Keep the remaining honest limits (APIM header/payload logging, request-size limits, other tiers, hosted stored-column scan) and variant B as unverified. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nik13
marked this pull request as ready for review
October 21, 2026 22:59
Co-Authored-By: Claude Code <noreply@anthropic.com>
…gateway Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The hosted collector and released receivers (v1.56.0+) accept gzip-encoded OTLP/HTTP. Remove gzip-rejection from the contract table, troubleshooting and the live-check summary, drop the e2e negative step that expected a gzip 400, and mark the collector's compression: none optional. The qualified APIM path stays uncompressed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Status 2026-10-21: ready for review. The STAND-IN/BLOCKED-ON sections below predate the live run. Variant A ran live on a real APIM Consumption instance into hosted Future AGI on 2026-10-19 (see "Live vendor verification" below). Variant B (AI Gateway tier) stays synthetic and is marked unverified on the page, because its monitoring destination is portal-only. Tracker result: DONE (core). 2026-10-21 commits: removed seven committed
__pycache__/*.pycfiles (they embedded a local home path) and added a.gitignore(9669b21e); updated the example README to the live variant A state (686a00a2). No navigation edit: #989 adds the nav entry and merges last.Summary
One docs page, two Azure variants, plus runnable assets under
examples/gateways/azure-apim/.set-headerpolicy addsX-Api-Key/X-Secret-Keyand forwards to Future AGI. The caller holds no Future AGI keys.azure.ai_gateway.identity.claims.<claim>attributes. The Learn page (govern-secure-assets) still carries the old "token metric only / traces coming soon" sentence; the page follows the newer release notes and flags the conflict. Recipe: a small OpenTelemetry Collector (contrib 0.162.0) that setsproject_name, accepts gzip, filters content, and forwards uncompressed with the key headers. PRD stop rule tripped and followed (AC-01/AC-04 of r2 would be stale).navigation.tschange.STAND-IN flags (BLOCKED-ON)
No Azure resource was created or called. APIM is a local stand-in (
apim_standin.py, models the policy). The AI Gateway tier is a synthetic trace (gateway_trace.py), shaped from Microsoft's release notes; every attribute name other thangen_ai.*andazure.ai_gateway.identity.claims.*is invented. Draft until one live run on each.BLOCKED-ON: Nikhil approving a non-production Azure subscription and spend for (a) an APIM instance (Developer tier) and (b) an AI Gateway tier instance (preview, East US 2 / Sweden Central, pricing unannounced). The README lists exactly what each live run must confirm.
Versions tested
traceAI-openai0.1.10,fi-instrumentation-otel1.1.0,openai3.31.0 (latest stable, 2026-10-16),opentelemetry-sdk1.45.1, Python 3.13.15,otel/opentelemetry-collector-contrib0.162.0 (latest tag available locally; a newer release was not independently ruled out). Future AGI: self-hosted local stack.How to run
E2E evidence (run
r3, all checks passed)Variant A (real client, real Future AGI collector, project
th8103-rest-azure-apim-r3-a), trace16926ed7-2f44-45c5-9942-f8daa9e82cee:Variant B (synthetic trace via real collector,
th8103-rest-azure-apim-r3-b), traces8b095a8e29caa1577e8d9b5c669ad8c2(ok) andab58d058536e5a779520ce8fa8cba3e3(blocked, ERROR):fi_verify PASS, 7 spans: gateway requestchain> 2 policychainspans + model callllm(model, tokens 118/42); blocked trace has ERROR spans. Opt-in projectr3-b-optin: traces14bc0592a54dcc48f3e7ab9b363add1d,5d5a745a0dbd8fbd76b84f4e3f1a1b90, fi_verify PASS. UI screenshots checked.Negatives: missing/wrong APIM subscription key 401 (stand-in); unknown path 404; wrong Future AGI keys 401 (collector, passed through); no
project_name400; gzip 400; rejected batches created no project; collector wrong/no bearer token 401; logs path 404.Privacy proof
FI_CAPTURE_CONTENT=true= marker in 4 span rows, object id in 2. Caveat on the page: proves the filter for the listed attribute names; what the real gateway trace carries is unverified.FI_HIDE_INPUTS/OUTPUTS=true= 0 occurrences of marker, home path, username, hostname in all stored rows; default run = marker in 26 span rows. Page warns about APIM's own header/payload logging.Limitations
Real APIM and AI Gateway tier behaviour unverified (policy deploy, header/body/
Content-Encodinghandling, request limits, destination form, header support, real trace shape and attribute names, content presence, compression, trace-context propagation). Hostedapi.futureagi.comURL not exercised.AzureOpenAIclient not tested. Cost metrics, logs and token metrics are not ingested (Future AGI takes traces only). The SDK ignores any path inFI_BASE_URL, so the APIM API suffix must betracer(found infi-instrumentation-otel1.1.0). The ClickHouse in the shared stack hit MEMORY_LIMIT_EXCEEDED during two earlier runs;run_e2e.shretries verify/scan.Nav suggestion
Group "Gateways", title "Azure API Management and AI Gateway". Nav is not edited here (single nav PR at the end).
Checks
docs_build.sh: build exit 0, audit-links 0 broken nav/content links.Live vendor verification (2026-10-19)
Variant A ran live on a real API Management Consumption instance (East US) into hosted api.futureagi.com with the page's policy unchanged: trace
ab4de4f8-c745-4da3-9c9f-f47738057169(agent > llm/tool + one ERROR llm) read back viaGET /tracer/trace/<id>/; policy responses 401/404/400/400/200 as documented; withFI_HIDE_*the marker is absent (6e7e995c-fb6e-47b9-8368-6cafeada90bf). Variant B: anaigatewaysresource was created, but models/keys/OTLP destination are portal-only (Entra interactive sign-in), and Learn still says traces are "coming soon"; variant B stays synthetic/unverified. Local-stack replay and DB-column scan were not repeated for variant A. Docs build and audit-links pass. Azure resources deleted afterwards.Update 2026-10-29 (gzip): the
gzip 400negative above was recorded before Future AGI v1.56.0. Released receivers from v1.56.0 and the hosted endpoint accept gzip (hosted check today returned 200), so the page no longer lists gzip as rejected, the e2e gzip negative step is removed, and the collector'scompression: noneis marked optional. The qualified APIM path stays uncompressed.🤖 Generated with Claude Code