Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
5936e09
test(harness): add shared OTLP harness for TH-8103 contract tests
nik13 Oct 4, 2026
32f5c9d
test(harness): accept the collector /tracer/v1/traces path
nik13 Oct 4, 2026
3eaadc8
fix(harness): decode chunked OTLP bodies and record request metadata
nik13 Oct 4, 2026
31f910c
feat(semantic-kernel): add traceAI-semantic-kernel on native SK diagn…
nik13 Oct 4, 2026
05c26ce
fix(semantic-kernel): kind id-less auto-invoked tools as TOOL
nik13 Oct 5, 2026
5d08e9b
fix(semantic-kernel): warn and skip instead of raising from instrument()
nik13 Oct 5, 2026
6a852c2
fix(semantic-kernel): own only the processor instrument() installs
nik13 Oct 5, 2026
8f01d67
feat(semantic-kernel): honour TraceConfig hide_inputs / hide_outputs
nik13 Oct 5, 2026
3f67128
docs(semantic-kernel): document processor ordering and client double-…
nik13 Oct 5, 2026
dfc510c
test(semantic-kernel): run the matrix the classifiers and floor declare
nik13 Oct 5, 2026
9fa1149
feat(semantic-kernel): put model-call prompts and completions on chat…
nik13 Oct 12, 2026
9651b34
fix(semantic-kernel): show a single completion as the assistant reply
nik13 Oct 12, 2026
c0eb2dd
fix(semantic-kernel): require fi-instrumentation-otel>=1.1.0
nik13 Oct 12, 2026
525069c
fix(semantic-kernel): never export stack traces; drop exception text …
nik13 Oct 17, 2026
8078d15
Merge dev into feat/th-8241-semantic-kernel-python
nik13 Oct 21, 2026
2ca5f64
docs(semantic-kernel): fold the unreleased notes into 0.1.0 (TH-8241)
nik13 Oct 21, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions python/frameworks/semantic-kernel/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Changelog

## 0.1.0

- Initial release. `SemanticKernelInstrumentor().instrument()` enables Semantic Kernel's native OpenTelemetry GenAI diagnostics in process and installs a mapping-only span processor (span kind, provider alias, total tokens, session, content default). No `wrapt`, no wrapping of `Kernel.invoke`.
- Supports `semantic-kernel` 1.38.0 to 1.x; tested on 1.38.0 and 1.44.1 with Python 3.10 to 3.13, on 1.45.0 with Python 3.12, and with `opentelemetry-sdk` 1.29.0 (floor) and 1.45.0.
- An `execute_tool` span is `TOOL` when it carries a tool call id or runs directly under `AutoFunctionInvocationLoop`, so connectors that send no tool call id (Ollama) still show tools.
- `instrument()` logs a WARNING and skips a Semantic Kernel module it cannot import, and logs a WARNING and changes nothing for a non-SDK tracer provider, instead of raising.
- `instrument()` installs and owns its own processor; a `SemanticKernelSpanProcessor` you installed is left in place by both `instrument()` and `uninstrument()`.
- `instrument(config=TraceConfig(...))`, `FI_HIDE_INPUTS` and `FI_HIDE_OUTPUTS` drop the input / output content keys that `sensitive=True` would keep.
- With `sensitive=True`, model-call (`chat`) spans carry the prompt and completion. Semantic Kernel only logs those; a proxy in front of its model-diagnostics logger also records them on the span and passes every call on to the real logger. Not installed with `sensitive=False`; `uninstrument()` removes it. Tested on semantic-kernel 1.38.0 and 1.45.0.
- Privacy: the processor always removes `exception.stacktrace` from span events (it carried local file paths and the user name). With `sensitive=False` it also removes `exception.message` and cuts the ERROR status description to the exception type, so a tool or model error that echoes its input is not exported.
148 changes: 148 additions & 0 deletions python/frameworks/semantic-kernel/README.md

Large diffs are not rendered by default.

121 changes: 121 additions & 0 deletions python/frameworks/semantic-kernel/examples/basic_agent.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
"""Trace a Semantic Kernel ChatCompletionAgent with Future AGI. No model key needed.

The model is mocked: Semantic Kernel's real OpenAI connector talks to an
in-process ``httpx.MockTransport``, so no request leaves the machine for the
model. Spans are exported with ``fi_instrumentation.register()``; set
``FI_API_KEY`` and ``FI_SECRET_KEY`` (and optionally ``FI_PROJECT_NAME``,
``FI_BASE_URL``) to see the trace in Future AGI. Without keys the export fails
and is logged; the agent still answers.

pip install traceAI-semantic-kernel
python examples/basic_agent.py

You do not export any SEMANTICKERNEL_* variable: ``instrument()`` turns Semantic
Kernel's diagnostics on in process. Message bodies stay off (``sensitive=False``).
"""

from __future__ import annotations

import asyncio
import json
import os
from typing import Annotated

import httpx
from openai import AsyncOpenAI

from fi_instrumentation import register, using_session
from fi_instrumentation.fi_types import ProjectType
from traceai_semantic_kernel import SemanticKernelInstrumentor

trace_provider = register(
project_type=ProjectType.OBSERVE,
project_name=os.getenv("FI_PROJECT_NAME", "semantic-kernel-example"),
verbose=False,
)
# Enables Semantic Kernel's native diagnostics. Does not wrap Kernel.invoke.
# sensitive=False is the default; True stores prompts and tool results.
SemanticKernelInstrumentor().instrument(tracer_provider=trace_provider, sensitive=False)

from semantic_kernel import Kernel # noqa: E402
from semantic_kernel.agents import ChatCompletionAgent # noqa: E402
from semantic_kernel.connectors.ai.function_choice_behavior import FunctionChoiceBehavior # noqa: E402
from semantic_kernel.connectors.ai.open_ai import ( # noqa: E402
OpenAIChatCompletion,
OpenAIChatPromptExecutionSettings,
)
from semantic_kernel.functions import KernelArguments, kernel_function # noqa: E402


class WeatherPlugin:
@kernel_function(name="get_weather", description="Current weather for a city")
def get_weather(self, city: Annotated[str, "City name"]) -> str:
return "sunny, 21 C in " + city


def mock_chat_completions(request: httpx.Request) -> httpx.Response:
"""A stand-in for the OpenAI chat completions endpoint: one tool call, then an answer."""
body = json.loads(request.content or b"{}")
messages = body.get("messages") or []
if body.get("tools") and messages and messages[-1].get("role") != "tool":
tool = body["tools"][0]["function"]["name"]
message = {
"role": "assistant",
"content": None,
"tool_calls": [
{
"id": "call_mock_1",
"type": "function",
"function": {"name": tool, "arguments": json.dumps({"city": "Paris"})},
}
],
}
finish = "tool_calls"
else:
tool_result = messages[-1].get("content") if messages else ""
message = {"role": "assistant", "content": "It is {0}.".format(tool_result)}
finish = "stop"
return httpx.Response(
200,
json={
"id": "chatcmpl-mock",
"object": "chat.completion",
"created": 1700000000,
"model": body.get("model", "gpt-4o-mini"),
"choices": [{"index": 0, "message": message, "finish_reason": finish}],
"usage": {"prompt_tokens": 20, "completion_tokens": 5, "total_tokens": 25},
},
)


async def main() -> None:
client = AsyncOpenAI(
api_key="mock-key-not-used",
base_url="http://mock-openai.invalid/v1",
http_client=httpx.AsyncClient(transport=httpx.MockTransport(mock_chat_completions)),
)
kernel = Kernel()
kernel.add_service(OpenAIChatCompletion(ai_model_id="gpt-4o-mini", async_client=client))
kernel.add_plugin(WeatherPlugin(), "Weather")

agent = ChatCompletionAgent(
kernel=kernel,
name="WeatherAgent",
instructions="Answer weather questions with the Weather plugin.",
arguments=KernelArguments(
settings=OpenAIChatPromptExecutionSettings(function_choice_behavior=FunctionChoiceBehavior.Auto())
),
)
# Semantic Kernel does not put a thread id on its spans; set the session yourself.
with using_session("example-session-1"):
response = await agent.get_response("What is the weather in Paris?")
print(response.message.content)


if __name__ == "__main__":
try:
asyncio.run(main())
finally:
# Short scripts must flush; the batch exporter sends in the background.
trace_provider.force_flush()
trace_provider.shutdown()
50 changes: 50 additions & 0 deletions python/frameworks/semantic-kernel/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
[tool.poetry]
name = "traceAI-semantic-kernel"
version = "0.1.0"
description = "Future AGI tracing for Semantic Kernel (Python) through its native OpenTelemetry diagnostics"
authors = ["Future AGI <no-reply@futureagi.com>"]
license = "Apache-2.0"
readme = "README.md"
homepage = "https://github.com/future-agi/traceAI"
repository = "https://github.com/future-agi/traceAI"
keywords = ["opentelemetry", "semantic-kernel", "tracing", "observability", "agents", "llm"]
classifiers = [
"Development Status :: 3 - Alpha",
"Intended Audience :: Developers",
"License :: OSI Approved :: Apache Software License",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Topic :: Software Development :: Libraries :: Python Modules",
"Topic :: System :: Monitoring",
]
packages = [
{ include = "traceai_semantic_kernel" },
]

[tool.poetry.dependencies]
python = ">=3.10,<3.14"
# Semantic Kernel (MIT) is a dependency floor only; never vendored or bundled.
# 1.38.0 is the oldest release whose diagnostics surface (settings instances,
# operation strings, agent attribute keys, module-level tracers) is identical
# to 1.44.1. Tested on 1.38.0 and 1.44.1.
semantic-kernel = ">=1.38.0,<2"
fi-instrumentation-otel = ">=1.1.0"
# Floor tested: the full suite passes with opentelemetry-api/-sdk/-exporter-otlp-proto-http
# 1.29.0 as well as 1.45.0. Python 3.10-3.13 tested on semantic-kernel 1.38.0 and 1.44.1.
opentelemetry-api = ">=1.29.0"
opentelemetry-sdk = ">=1.29.0"

[tool.poetry.group.dev.dependencies]
pytest = ">=8.0.0"

[tool.pytest.ini_options]
testpaths = ["tests"]
python_files = ["test_*.py"]
python_functions = ["test_*"]

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
154 changes: 154 additions & 0 deletions python/frameworks/semantic-kernel/tests/_contract_app.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
"""Contract app: real Semantic Kernel + real OTel SDK + real Future AGI OTLP/HTTP exporter.

Run as a subprocess by ``test_contract_harness.py`` so Semantic Kernel's
process-wide diagnostics switches start from their defaults every time. The
parent points ``FI_BASE_URL`` at a loopback harness ``Receiver``;
``register()`` then exports OTLP/HTTP protobuf to
``{FI_BASE_URL}/tracer/v1/traces``. The model is a loopback fake
OpenAI-compatible server; placeholder keys only, no vendor call.

Usage: ``python _contract_app.py off|on|deadcollector``.
"""

from __future__ import annotations

import asyncio
import json
import os
import sys

HERE = os.path.dirname(os.path.abspath(__file__))
if HERE not in sys.path:
sys.path.insert(0, HERE)


def _snapshot_processor():
"""Test-only: record each span's attributes before the mapping processor runs (AC-02)."""
from opentelemetry.sdk.trace import SpanProcessor

class SnapshotProcessor(SpanProcessor):
def __init__(self) -> None:
self.raw = {}

def on_end(self, span):
self.raw[format(span.context.span_id, "016x")] = dict(span.attributes or {})

def force_flush(self, timeout_millis: int = 30000) -> bool:
# opentelemetry-sdk 1.29's base SpanProcessor.force_flush returns None,
# which makes the provider's force_flush report False.
return True

return SnapshotProcessor()


async def journeys(fake, out):
from openai import AsyncOpenAI
from semantic_kernel import Kernel
from semantic_kernel.agents import ChatCompletionAgent
from semantic_kernel.connectors.ai.function_choice_behavior import FunctionChoiceBehavior
from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion, OpenAIChatPromptExecutionSettings
from semantic_kernel.contents import ChatHistory
from semantic_kernel.functions import KernelArguments

from fi_instrumentation import using_session
from _scenarios import SECRET_PROMPT, SESSION, broken_plugin, weather_plugin

client = AsyncOpenAI(api_key="placeholder-openai-key", base_url=fake.base_url, max_retries=0)
chat = OpenAIChatCompletion(ai_model_id="gpt-4o-mini", async_client=client)
auto = OpenAIChatPromptExecutionSettings(function_choice_behavior=FunctionChoiceBehavior.Auto())

# 1. Plain chat completion through the connector.
history = ChatHistory()
history.add_user_message(SECRET_PROMPT)
reply = await chat.get_chat_message_contents(history, OpenAIChatPromptExecutionSettings())
out["chat_reply"] = str(reply[0].content)

# 2. Kernel function that makes the model call one tool (auto function invocation).
kernel = Kernel()
kernel.add_service(chat)
kernel.add_plugin(weather_plugin(), "Weather")
result = await kernel.invoke_prompt(
SECRET_PROMPT, function_name="ask_weather", plugin_name="Prompts", arguments=KernelArguments(settings=auto)
)
out["prompt_reply"] = str(result)

# 3. ChatCompletionAgent invocation, inside an app-set session.
agent = ChatCompletionAgent(
kernel=kernel, name="Assistant", instructions="Answer briefly.", arguments=KernelArguments(settings=auto)
)
with using_session(SESSION):
response = await agent.get_response(SECRET_PROMPT)
out["agent_reply"] = str(response.message.content)
out["agent_thread_id"] = str(response.thread.id)

# 4. Streaming chat completion.
stream_chat = OpenAIChatCompletion(ai_model_id="gpt-4o-mini-stream", async_client=client)
chunks = []
async for chunk in stream_chat.get_streaming_chat_message_contents(history, OpenAIChatPromptExecutionSettings()):
chunks.extend(str(c.content or "") for c in chunk)
out["stream_reply"] = "".join(chunks)

# 5. Errors: model rejects (non-streaming and streaming); a tool raises.
broken = OpenAIChatCompletion(ai_model_id="broken-model", async_client=client)
errors = []
try:
await broken.get_chat_message_contents(history, OpenAIChatPromptExecutionSettings())
except Exception as error: # noqa: BLE001 - the error is the point
errors.append(type(error).__name__)
try:
async for _ in broken.get_streaming_chat_message_contents(history, OpenAIChatPromptExecutionSettings()):
pass
except Exception as error: # noqa: BLE001
errors.append(type(error).__name__)
out["model_errors"] = errors

broken_kernel = Kernel()
broken_kernel.add_service(chat)
broken_kernel.add_plugin(broken_plugin(), "Broken")
result = await broken_kernel.invoke_prompt(
SECRET_PROMPT, function_name="ask_broken", plugin_name="Prompts", arguments=KernelArguments(settings=auto)
)
out["broken_tool_reply"] = str(result)


def main(argv):
mode = argv[1] if len(argv) > 1 else "off"

from fi_instrumentation import register
from fi_instrumentation.fi_types import ProjectType
from opentelemetry import trace as trace_api

from _fake_openai import FakeOpenAI
from _scenarios import PROJECT_NAME, TOOL_SECRET_CITY
from traceai_semantic_kernel import SemanticKernelInstrumentor

assert not any(key.startswith("SEMANTICKERNEL_") for key in os.environ), "env flags must not be needed"

out = {"mode": mode}
out["global_provider_before"] = type(trace_api.get_tracer_provider()).__name__
provider = register(project_type=ProjectType.OBSERVE, project_name=PROJECT_NAME, verbose=False)

sensitive = mode == "on"
SemanticKernelInstrumentor().instrument(tracer_provider=provider, sensitive=sensitive)
SemanticKernelInstrumentor().instrument(tracer_provider=provider, sensitive=sensitive) # AC-01: idempotent
out["processors"] = [type(p).__name__ for p in provider._active_span_processor._span_processors]

snapshot = _snapshot_processor()
active = provider._active_span_processor
with active._lock:
active._span_processors = (snapshot,) + tuple(active._span_processors)

with FakeOpenAI(tool_arguments={"city": TOOL_SECRET_CITY}) as fake:
asyncio.run(journeys(fake, out))
out["llm_requests"] = len(fake.requests)

out["flushed"] = bool(provider.force_flush(timeout_millis=10000))
out["global_provider_after"] = type(trace_api.get_tracer_provider()).__name__
out["raw"] = snapshot.raw
provider.shutdown()
print("CONTRACT_RESULT " + json.dumps(out, default=str))
return 0


if __name__ == "__main__":
sys.exit(main(sys.argv))
Loading
Loading