From 167987c4e10ab33e8e7708c0705fc48451f273ec Mon Sep 17 00:00:00 2001 From: Nikhil Pareek Date: Fri, 16 Oct 2026 17:28:39 -0700 Subject: [PATCH 1/3] docs(integrations): Microsoft Agent Framework tracing (TH-8225) Co-Authored-By: Claude Sonnet 5.5 --- .../traceai/microsoft-agent-framework.mdx | 129 ++++++++++++++++++ 1 file changed, 129 insertions(+) create mode 100644 src/pages/docs/integrations/traceai/microsoft-agent-framework.mdx diff --git a/src/pages/docs/integrations/traceai/microsoft-agent-framework.mdx b/src/pages/docs/integrations/traceai/microsoft-agent-framework.mdx new file mode 100644 index 000000000..7f3c8f87d --- /dev/null +++ b/src/pages/docs/integrations/traceai/microsoft-agent-framework.mdx @@ -0,0 +1,129 @@ +--- +title: "Microsoft Agent Framework Integration with Future AGI for Agent Tracing" +description: "Trace Microsoft Agent Framework (Python) agents, tools and workflows in Future AGI with traceAI-agent-framework. Not AutoGen, not Semantic Kernel." +--- + +## About + +This page covers the Python package of Microsoft Agent Framework (PyPI `agent-framework`). It is not Microsoft AutoGen ([AutoGen](/docs/integrations/traceai/autogen)), not Semantic Kernel, and not AG2. The .NET SDK is not covered. + +Agent Framework already emits OpenTelemetry spans (GenAI semantic conventions) on the `agent_framework` tracer. `traceAI-agent-framework` does not wrap the framework. It adds one span processor that gives those native spans the Future AGI span kind, input/output, session id and token total, so they render as agent, LLM and tool spans. + +## 1. Installation + +```bash +pip install traceAI-agent-framework agent-framework +``` + +## 2. Set environment variables + +```python +import os + +os.environ["FI_API_KEY"] = "your-futureagi-api-key" +os.environ["FI_SECRET_KEY"] = "your-futureagi-secret-key" +os.environ["OPENAI_API_KEY"] = "your-openai-api-key" +``` + +## 3. Register the tracer provider + +Agent Framework emits on the global tracer provider only, so register with `set_global_tracer_provider=True`. + +```python +from fi_instrumentation import register +from fi_instrumentation.fi_types import ProjectType + +trace_provider = register( + project_type=ProjectType.OBSERVE, + project_name="agent_framework_app", + set_global_tracer_provider=True, +) +``` + +## 4. Turn on the mapping + +```python +from agent_framework.observability import enable_instrumentation +from traceai_agent_framework import enable_fi_attribute_mapping + +enable_instrumentation(enable_sensitive_data=True) +enable_fi_attribute_mapping() +``` + +`enable_sensitive_data=True` is the framework's own switch for putting prompts, completions and tool arguments on its spans. Without it the framework does not emit them and Future AGI has nothing to show. `enable_fi_attribute_mapping()` is idempotent. + +## 5. Run an agent + +```python +import asyncio +from agent_framework import Agent +from agent_framework.openai import OpenAIChatCompletionClient + +def get_weather(city: str) -> str: + """Get the current weather for a city.""" + return f"Sunny, 21C in {city}" + +agent = Agent( + OpenAIChatCompletionClient(model="gpt-4o-mini"), + name="weather_agent", + instructions="You are a concise weather assistant. Always call get_weather.", + tools=[get_weather], +) + +async def main(): + print(await agent.run("What's the weather in Paris?")) + +asyncio.run(main()) +trace_provider.force_flush() +``` + +Short scripts must flush the provider before exit. + +## What is captured + +| Span | Future AGI type | Notes | +|---|---|---| +| `invoke_agent ` | agent | Whole run, summed token usage, agent name | +| `chat ` | llm | Model, provider, input/output tokens, request parameters, messages | +| `execute_tool ` | tool | Tool name, arguments, result | +| workflow / executor / edge-group spans | chain | Input/output bubbled up from child spans | + +- Every span of one run carries the same `session.id`. It comes from `using_session(...)` when you set it, otherwise from the run's `gen_ai.conversation.id`. +- A failing tool or model call is recorded with ERROR status and the exception class name. +- Cost is not emitted by the framework. Retrieval spans are not emitted. + +## Privacy + +- **Content is captured by default** (like every traceAI integration) once the framework's `enable_sensitive_data` is on. Hide it with the standard traceAI switches: `FI_HIDE_INPUTS=true`, `FI_HIDE_OUTPUTS=true`, `FI_HIDE_INPUT_MESSAGES`, `FI_HIDE_OUTPUT_MESSAGES`, `FI_HIDE_INPUT_TEXT`, `FI_HIDE_OUTPUT_TEXT`, or pass a `TraceConfig` to `enable_fi_attribute_mapping(config=...)`. +- With `FI_HIDE_INPUTS` and `FI_HIDE_OUTPUTS` set, prompts, responses, system instructions, tool arguments and tool results are removed or redacted. Model, token counts, timings, span structure and tool names stay. +- To keep content out entirely, leave `enable_sensitive_data` off (the framework then never emits it). +- Raw exception text and stack traces are never exported. A failed span keeps ERROR status with only the exception class name. + +## Tested versions + +| Component | Version | +|---|---| +| agent-framework | 1.19.0 (latest stable on 2026-10-16), 1.0.0 (package floor) | +| fi-instrumentation-otel | 1.1.0 | +| Python | 3.12 | + +Verified end to end against a local Future AGI: agent, two LLM calls and one tool call (hierarchy, model, tokens, session), a failing tool, a failing model call, and a hide-flags run in which a unique marker was absent from every stored column. + +## Troubleshooting + +- **No spans.** `register(..., set_global_tracer_provider=True)` is required. `enable_fi_attribute_mapping()` logs a warning if the provider is not the global one. +- **No prompts or answers in the trace.** Call `enable_instrumentation(enable_sensitive_data=True)` and check that no `FI_HIDE_*` variable is set. +- **One session per turn on the Responses API.** Wrap the turns in `using_session("my-session")`. +- **Duplicate LLM spans.** Do not also enable `traceAI-openai` under Agent Framework; the framework already emits the model span. +- **Spans missing in a short script.** Call `trace_provider.force_flush()` before exit. + +## Limitations + +- Python only. No .NET. +- Do not call `configure_otel_providers()`; it can install a second exporter. +- Traces only (no metrics or logs). + +## Difference from related integrations + +- [AutoGen](/docs/integrations/traceai/autogen) is a different framework with its own package. +- Provider instrumentors such as `traceAI-openai` patch the provider SDK. Agent Framework uses its own native spans, so only this package is needed. From 148cd5b6a31a6b70ad93a75b8d7959076cb39c39 Mon Sep 17 00:00:00 2001 From: Nikhil Pareek Date: Wed, 21 Oct 2026 15:12:21 -0700 Subject: [PATCH 2/3] docs(integrations): Agent Framework tested versions add 1.22.0 and the Python range (TH-8225) traceAI-agent-framework now requires Python 3.10 to 3.13 (fi-instrumentation-otel 1.1.0 does not install on 3.14); its unit tests pass on agent-framework 1.22.0. Co-Authored-By: Claude Opus 5.5 --- .../docs/integrations/traceai/microsoft-agent-framework.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/pages/docs/integrations/traceai/microsoft-agent-framework.mdx b/src/pages/docs/integrations/traceai/microsoft-agent-framework.mdx index 7f3c8f87d..a72bd28f2 100644 --- a/src/pages/docs/integrations/traceai/microsoft-agent-framework.mdx +++ b/src/pages/docs/integrations/traceai/microsoft-agent-framework.mdx @@ -103,9 +103,9 @@ Short scripts must flush the provider before exit. | Component | Version | |---|---| -| agent-framework | 1.19.0 (latest stable on 2026-10-16), 1.0.0 (package floor) | +| agent-framework | 1.19.0 (latest stable on 2026-10-16, end to end), 1.22.0 (unit tests, 2026-10-21), 1.0.0 (package floor) | | fi-instrumentation-otel | 1.1.0 | -| Python | 3.12 | +| Python | 3.12 (end to end), 3.11 (unit tests). The package installs on Python 3.10 to 3.13. | Verified end to end against a local Future AGI: agent, two LLM calls and one tool call (hierarchy, model, tokens, session), a failing tool, a failing model call, and a hide-flags run in which a unique marker was absent from every stored column. From 2973cd8b1d5969680bbd6ec2cf6ac745cbdf1fb9 Mon Sep 17 00:00:00 2001 From: Nikhil Pareek Date: Thu, 5 Nov 2026 20:50:56 -0800 Subject: [PATCH 3/3] docs(integrations): publish the Agent Framework guide in the catalog The guide carried an internal test-version table and a live-verification narrative that customers cannot act on. Replace them with the requirement the released package enforces (Python 3.10 to 3.13, Agent Framework 1.0.0 or later), pin the served traceAI-agent-framework 0.1.0 release, and add a short "view the trace" step. Correct the agent-span token note: the released processor moves the framework's run totals to agent_framework.usage.* so tokens are counted once on the chat spans. Add the OTEL_SEMCONV_STABILITY_OPT_IN case, which stops current Agent Framework releases from putting messages on spans. Register the page in src/lib/integrations.mjs so it is discoverable from the directory and sidebar, and add a page-contract test for the setup and the absence of test-diary wording. Co-Authored-By: Claude Code --- scripts/integrations.test.mjs | 11 ++++ src/lib/integrations.mjs | 1 + .../traceai/microsoft-agent-framework.mdx | 51 +++++++++---------- 3 files changed, 36 insertions(+), 27 deletions(-) diff --git a/scripts/integrations.test.mjs b/scripts/integrations.test.mjs index 30ff909ed..facd38dd7 100644 --- a/scripts/integrations.test.mjs +++ b/scripts/integrations.test.mjs @@ -487,3 +487,14 @@ test('Temporal Go uses the official extension and authenticated traces-only OTLP assert.doesNotMatch(page, /AddOtlpMetricExporter|AddOtlpLogExporter|traceai-temporal-go|Live vendor verification/); assert.match(read('src/pages/docs/integrations/frameworks/temporal.mdx'), /frameworks\/temporal-go/); }); + +test('Microsoft Agent Framework guide uses the released package and global provider without a test diary', () => { + const page = read('src/pages/docs/integrations/traceai/microsoft-agent-framework.mdx'); + const catalog = integrations.find(i => i.id === 'microsoft-agent-framework'); + assert.equal(catalog?.href, '/docs/integrations/traceai/microsoft-agent-framework'); + assert.equal(catalog?.category, 'frameworks'); + assert.match(page, /pip install "traceAI-agent-framework==0\.1\.0" agent-framework/); + assert.match(page, /Python 3\.10 to 3\.13 and Agent Framework 1\.0\.0 or later/); + for (const term of ['set_global_tracer_provider=True', 'enable_instrumentation(enable_sensitive_data=True)', 'enable_fi_attribute_mapping()', 'trace_provider.force_flush()', 'agent_framework.usage.*', 'configure_otel_providers()']) assert.ok(page.includes(term), term); + assert.doesNotMatch(page, /Tested versions|end to end|unit tests|unique marker|Verified|\b20\d\d-\d\d-\d\d\b/); +}); diff --git a/src/lib/integrations.mjs b/src/lib/integrations.mjs index df0f390a2..99b98fe7d 100644 --- a/src/lib/integrations.mjs +++ b/src/lib/integrations.mjs @@ -81,6 +81,7 @@ export const integrations = [ { id: 'google-adk', name: 'Google ADK', href: '/docs/integrations/traceai/google_adk', category: 'frameworks', summary: 'Instrument agents built with Google ADK for Python.', method: 'traceAI SDK', logo: 'google', aliases: ['Agent Development Kit'] }, { id: 'crewai', name: 'CrewAI', href: '/docs/integrations/traceai/crewai', category: 'frameworks', summary: 'Trace crews, standalone agents, Flows and model calls.', method: 'Native OTLP', logo: 'crewai' }, { id: 'autogen', name: 'AutoGen', href: '/docs/integrations/traceai/autogen', category: 'frameworks', summary: 'Trace conversations between AutoGen agents.', method: 'traceAI SDK', logo: 'autogen' }, + { id: 'microsoft-agent-framework', name: 'Microsoft Agent Framework', href: '/docs/integrations/traceai/microsoft-agent-framework', category: 'frameworks', summary: 'Map native Python agent, model, tool and workflow spans.', method: 'OTel + traceAI', logo: 'generic-framework', aliases: ['Agent Framework', 'MAF', 'agent-framework'] }, { id: 'haystack', name: 'Haystack', href: '/docs/integrations/traceai/haystack', category: 'frameworks', summary: 'Trace component execution in Haystack pipelines.', method: 'traceAI SDK', logo: 'haystack' }, { id: 'dspy', name: 'DSPy', href: '/docs/integrations/traceai/dspy', category: 'frameworks', summary: 'Capture DSPy module and language model calls.', method: 'traceAI SDK', logo: 'dspy' }, { id: 'smol-agents', name: 'Hugging Face smolagents', href: '/docs/integrations/traceai/smol_agents', category: 'frameworks', summary: 'Trace agent execution and tool calls in smolagents.', method: 'traceAI SDK', logo: 'huggingface' }, diff --git a/src/pages/docs/integrations/traceai/microsoft-agent-framework.mdx b/src/pages/docs/integrations/traceai/microsoft-agent-framework.mdx index a72bd28f2..91ad43112 100644 --- a/src/pages/docs/integrations/traceai/microsoft-agent-framework.mdx +++ b/src/pages/docs/integrations/traceai/microsoft-agent-framework.mdx @@ -1,33 +1,35 @@ --- title: "Microsoft Agent Framework Integration with Future AGI for Agent Tracing" -description: "Trace Microsoft Agent Framework (Python) agents, tools and workflows in Future AGI with traceAI-agent-framework. Not AutoGen, not Semantic Kernel." +description: "Trace Microsoft Agent Framework (Python) agents, model calls, tools and workflows in Future AGI with traceAI-agent-framework." --- ## About -This page covers the Python package of Microsoft Agent Framework (PyPI `agent-framework`). It is not Microsoft AutoGen ([AutoGen](/docs/integrations/traceai/autogen)), not Semantic Kernel, and not AG2. The .NET SDK is not covered. +Trace Microsoft Agent Framework agent runs, model calls, tool calls and workflows in Future AGI. This page covers the Python package (PyPI `agent-framework`), not the .NET SDK. It is a different framework from [AutoGen](/docs/integrations/traceai/autogen), Semantic Kernel and AG2. -Agent Framework already emits OpenTelemetry spans (GenAI semantic conventions) on the `agent_framework` tracer. `traceAI-agent-framework` does not wrap the framework. It adds one span processor that gives those native spans the Future AGI span kind, input/output, session id and token total, so they render as agent, LLM and tool spans. +Agent Framework already emits OpenTelemetry spans that follow the GenAI semantic conventions. `traceAI-agent-framework` does not wrap the framework. It adds one span processor that gives those native spans the Future AGI span kind, input and output, session ID and token totals, so they appear as agent, LLM, tool and chain spans. ## 1. Installation +Use Python 3.10 to 3.13 and Agent Framework 1.0.0 or later. + ```bash -pip install traceAI-agent-framework agent-framework +pip install "traceAI-agent-framework==0.1.0" agent-framework ``` ## 2. Set environment variables -```python -import os +Set credentials in your shell, not in source code: -os.environ["FI_API_KEY"] = "your-futureagi-api-key" -os.environ["FI_SECRET_KEY"] = "your-futureagi-secret-key" -os.environ["OPENAI_API_KEY"] = "your-openai-api-key" +```bash +export FI_API_KEY="your-futureagi-api-key" +export FI_SECRET_KEY="your-futureagi-secret-key" +export OPENAI_API_KEY="your-openai-api-key" ``` ## 3. Register the tracer provider -Agent Framework emits on the global tracer provider only, so register with `set_global_tracer_provider=True`. +Agent Framework emits spans on the global tracer provider only, so register with `set_global_tracer_provider=True`. ```python from fi_instrumentation import register @@ -50,7 +52,7 @@ enable_instrumentation(enable_sensitive_data=True) enable_fi_attribute_mapping() ``` -`enable_sensitive_data=True` is the framework's own switch for putting prompts, completions and tool arguments on its spans. Without it the framework does not emit them and Future AGI has nothing to show. `enable_fi_attribute_mapping()` is idempotent. +`enable_sensitive_data=True` is the framework's own switch for recording prompts, completions and tool arguments on its spans. Without it, the framework does not record that content, so traces show no prompts or answers. `enable_fi_attribute_mapping()` is idempotent. ## 5. Run an agent @@ -77,43 +79,38 @@ asyncio.run(main()) trace_provider.force_flush() ``` -Short scripts must flush the provider before exit. +`force_flush()` sends the last spans before a short script exits. + +## 6. View the trace + +Open **Observe**, select `agent_framework_app`, and open the latest trace. The `invoke_agent weather_agent` span contains the `chat` model spans and the `execute_tool get_weather` tool span. ## What is captured | Span | Future AGI type | Notes | |---|---|---| -| `invoke_agent ` | agent | Whole run, summed token usage, agent name | +| `invoke_agent ` | agent | Whole run, agent name, run input and output | | `chat ` | llm | Model, provider, input/output tokens, request parameters, messages | | `execute_tool ` | tool | Tool name, arguments, result | | workflow / executor / edge-group spans | chain | Input/output bubbled up from child spans | +- Token usage is counted on each `chat` span. The agent span keeps the framework's run totals as `agent_framework.usage.*`, so the same tokens are not counted twice. - Every span of one run carries the same `session.id`. It comes from `using_session(...)` when you set it, otherwise from the run's `gen_ai.conversation.id`. - A failing tool or model call is recorded with ERROR status and the exception class name. -- Cost is not emitted by the framework. Retrieval spans are not emitted. +- The framework does not emit retrieval spans. ## Privacy - **Content is captured by default** (like every traceAI integration) once the framework's `enable_sensitive_data` is on. Hide it with the standard traceAI switches: `FI_HIDE_INPUTS=true`, `FI_HIDE_OUTPUTS=true`, `FI_HIDE_INPUT_MESSAGES`, `FI_HIDE_OUTPUT_MESSAGES`, `FI_HIDE_INPUT_TEXT`, `FI_HIDE_OUTPUT_TEXT`, or pass a `TraceConfig` to `enable_fi_attribute_mapping(config=...)`. - With `FI_HIDE_INPUTS` and `FI_HIDE_OUTPUTS` set, prompts, responses, system instructions, tool arguments and tool results are removed or redacted. Model, token counts, timings, span structure and tool names stay. -- To keep content out entirely, leave `enable_sensitive_data` off (the framework then never emits it). +- To keep content out entirely, leave `enable_sensitive_data` off (the framework then never records it). - Raw exception text and stack traces are never exported. A failed span keeps ERROR status with only the exception class name. -## Tested versions - -| Component | Version | -|---|---| -| agent-framework | 1.19.0 (latest stable on 2026-10-16, end to end), 1.22.0 (unit tests, 2026-10-21), 1.0.0 (package floor) | -| fi-instrumentation-otel | 1.1.0 | -| Python | 3.12 (end to end), 3.11 (unit tests). The package installs on Python 3.10 to 3.13. | - -Verified end to end against a local Future AGI: agent, two LLM calls and one tool call (hierarchy, model, tokens, session), a failing tool, a failing model call, and a hide-flags run in which a unique marker was absent from every stored column. - ## Troubleshooting - **No spans.** `register(..., set_global_tracer_provider=True)` is required. `enable_fi_attribute_mapping()` logs a warning if the provider is not the global one. -- **No prompts or answers in the trace.** Call `enable_instrumentation(enable_sensitive_data=True)` and check that no `FI_HIDE_*` variable is set. -- **One session per turn on the Responses API.** Wrap the turns in `using_session("my-session")`. +- **No prompts or answers in the trace.** Call `enable_instrumentation(enable_sensitive_data=True)` and check that no `FI_HIDE_*` variable is set. If your environment sets `OTEL_SEMCONV_STABILITY_OPT_IN`, include `gen_ai_latest_experimental` in it; otherwise current Agent Framework releases do not put messages and tool arguments on spans. +- **One session per turn on the Responses API.** Wrap the turns in `with using_session("my-session"):` (`from fi_instrumentation import using_session`). - **Duplicate LLM spans.** Do not also enable `traceAI-openai` under Agent Framework; the framework already emits the model span. - **Spans missing in a short script.** Call `trace_provider.force_flush()` before exit.