Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .github/python-tests-known-broken.txt
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@ guardrails | its core test file does not load (SpanAttributes.FI_SPAN_KIND no lo
haystack | 5 tests expect pre-GenAI attribute names (llm.token_count.*, fi.span.kind) and a list of dependencies
huggingface | 3 tests expect llm.system; 1 expects generated text that is no longer recorded
instructor | 6 tests: instructor.core no longer exists in current instructor (5), and a dependency string check (1)
langchain | 2 tests: dependency count and a set_attributes call no longer made
litellm | 3 tests use SpanAttributes.FI_SPAN_KIND, which no longer exists
llama_index | 14 tests: llama_index.core.base.agent no longer exists; 1 expects SpanAttributes.LLM_MODEL_NAME; 1 a dependency count
mcp | 1 test expects two instrumentation dependencies, the package declares one
Expand Down
45 changes: 45 additions & 0 deletions .github/workflows/deepagents-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
name: Deep Agents tests

on:
pull_request:
paths:
- 'python/frameworks/langchain/**'
- 'python/fi_instrumentation/**'
- 'python/tests/harness/**'
- 'python/tests/test_framework_langchain.py'
- '.github/workflows/deepagents-tests.yml'
push:
branches: [dev]
paths:
- 'python/frameworks/langchain/**'
- 'python/fi_instrumentation/**'
- 'python/tests/harness/**'
- 'python/tests/test_framework_langchain.py'
- '.github/workflows/deepagents-tests.yml'

permissions:
contents: read

jobs:
deepagents:
name: Deep Agents and core privacy
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
persist-credentials: false
- uses: actions/setup-python@v5
with:
python-version: '3.12'
- name: Install integration, current core and real framework
run: python -m pip install -e python -e python/frameworks/langchain 'deepagents==0.7.21' langgraph 'pytest-asyncio==0.24.0' 'pytest==8.4.2' pytest-mock
- name: Test Deep Agents, sessions and error hiding
run: |
python -c 'from fi_instrumentation import TraceConfig; assert hasattr(TraceConfig(), "hide_error_details")'
PYTHONPATH=python/tests python -m pytest python/frameworks/langchain/tests python/tests/test_framework_langchain.py -q -p no:cacheprovider -o addopts='' -o asyncio_mode=auto
- name: Test compatibility with published core
run: |
python -m pip install --force-reinstall 'fi-instrumentation-otel==1.1.0'
PYTHONPATH=python/tests python -P -m pytest python/frameworks/langchain/tests python/tests/test_framework_langchain.py -q -p no:cacheprovider -o addopts='' -o asyncio_mode=auto
26 changes: 9 additions & 17 deletions python/frameworks/langchain/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,15 @@
## [0.2.1] - 2026-10-18
## [0.2.1] - 2026-10-26
### Fixed
- The `ERROR` status description of a failed LangChain run is now the exception message
(`str(error)` of the error passed to `on_*_error`). It used to be LangChain's
`run.error`: `repr(error)` plus the full Python traceback, including local file paths.
The stacktrace is still recorded, only in the exception event's `exception.stacktrace`.
- Non-`Exception` errors (e.g. `KeyboardInterrupt`) record the error's own traceback and
its fully qualified type, instead of whatever `traceback.format_exc()` returned in the
callback.
- LangGraph's `configurable.thread_id` is copied to `session.id` on every span, including Deep Agents subagents. Metadata `session_id` and `using_session(...)` retain precedence; blank thread ids set nothing.
- Chat model token counts prefer the message's `usage_metadata`, so cached prompt tokens are included; provider `llm_output` is a fallback.
- Failed-run status descriptions use the original exception message, never LangChain's traceback-bearing `run.error`. Stacktraces remain only in exception events when error details are visible. Non-`Exception` errors retain their own traceback and fully qualified type.

### Changed
- With `fi-instrumentation-otel>=1.2.0`, the content hide flags (`hide_inputs`,
`hide_outputs`, ...) also hide error details on LangChain spans: no status
description, no `exception.message` / `exception.stacktrace`. This is a behaviour
change for users of those flags; see the core 1.2.0 changelog. The dependency pin
stays `>=1.0.0` until core 1.2.0 is on PyPI, so upgrade both packages together.
- With error details hidden, a failed run records no output attributes (`output.value`,
output messages, function calls). LangChain fills a failed chat-model run's outputs
with the provider's error response: the error body (which can quote the prompt),
the response headers and request ids, and AI-gateway metadata.
- Core `fi-instrumentation-otel>=1.2.0` owns error hiding, including derived content-hide flags and an explicit `hide_error_details=False` override. Hidden failures retain their type but no status description, exception message or stacktrace. On older supported core versions, `hide_inputs` or `hide_outputs` applies the same hiding as a compatibility fallback.
- Failed-run output attributes (including raw output, messages and function calls) are omitted when error details are hidden: provider error bodies and response headers can echo private content. Session mapping and token usage are retained.

### Added
- Deep Agents cookbook (`examples/deep_agents.py`) and compatibility tests against the published `deepagents==0.7.21` wheel. No new package or runtime dependency.

## [0.2.0] - 2026-07-30
### Fixed
Expand Down
101 changes: 99 additions & 2 deletions python/frameworks/langchain/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ LangGraph-specific setup. You get:
- **Per-node spans** named after the graph node (`agent`, `tools`, `ask_human`, …)
- **Tool and LLM spans** nested under their node
- **Graph-node enrichment** — the node's own span carries `gen_ai.agent.graph.node_name` and `gen_ai.agent.graph.node_id`
- **Session grouping** — bind `session.id = thread_id` (see the quickstart)
- **Session grouping** — LangGraph's `configurable.thread_id` becomes `session.id` on every span (an explicit `using_session(...)` or `metadata={"session_id": ...}` wins). New in 0.2.1; with 0.2.0 or earlier, wrap the call in `using_session(thread_id)`.
- **HITL interrupts** traced correctly — an interrupted node/tool span stays `OK` (not `ERROR`) and is marked with `langgraph.interrupt`

> **`LangGraphInstrumentor` is a deprecated no-op** kept for backwards compatibility.
Expand Down Expand Up @@ -137,11 +137,105 @@ Node, tool and LLM runs are standard LangChain spans (`gen_ai.span.kind` =

- `gen_ai.agent.graph.node_name`, `gen_ai.agent.graph.node_id` — the graph node (on the node's own span)
- `langgraph_node`, `langgraph_step`, `langgraph_triggers`, `langgraph_path`, `langgraph_checkpoint_ns` — LangGraph's raw callback metadata
- `session.id` — when set via `using_session(...)` or `config={"metadata": {"session_id": ...}}`
- `session.id` — from `config={"metadata": {"session_id": ...}}`, else `using_session(...)`, else LangGraph's `configurable.thread_id` (0.2.1 and later; a blank `thread_id` sets nothing). The choice is made per run: set an explicit session around the whole `invoke`/`stream`, because a session set for only part of a run (inside a node, or in an inner `with_config`) covers only the runs inside it, and the outer spans fall back to `thread_id`.
- `langgraph.interrupt` (attribute + event) on a HITL pause; a `langgraph.resume` event on resume

---

## Deep Agents

[Deep Agents](https://github.com/langchain-ai/deepagents) builds a LangGraph graph
with `create_deep_agent`. There is no Deep Agents instrumentor and no separate
package: `LangChainInstrumentor` traces the graph, its model calls, its built-in
tools and its `task` subagents. The cookbook is
[`examples/deep_agents.py`](examples/deep_agents.py); the compatibility test
(`tests/test_deepagents_compat.py`) runs that same script offline against a
scripted fake model and a loopback collector.

Tested versions, as printed by the compatibility test:

| Package | Version |
|---|---|
| `deepagents` | 0.7.21 (PyPI wheel); 0.7.23 and 0.7.28 (latest on 2026-10-16) also pass |
| `langchain` | 1.4.3; 1.4.4 with deepagents 0.7.23 |
| `langchain-core` | 1.6.6 (the floor `deepagents` 0.7.21 requires); 1.6.9 and 1.6.12 (latest) also pass |
| `langgraph` | 1.2.12; 1.2.14 and 1.2.16 (latest) also pass |
| `langgraph-prebuilt` | as resolved with `langgraph` 1.2.12 (printed by the test), and 1.1.0 with `langgraph` 1.2.14; its `ToolNode` re-raises tool errors |
| `traceAI-langchain` | 0.2.1 (tested from this repository before release): the `thread_id` → `session.id` change is **not** in 0.2.0 |
| Python | 3.11.12 and 3.13.7; 3.13.15 with deepagents 0.7.28; 3.12 with deepagents 0.7.23 |

`traceAI-langchain`'s own lower bound (`langchain-core>=0.2.43`) is unchanged.
Deep Agents itself needs `langchain-core>=1.6.6` and `langgraph`.

```bash
pip install "traceAI-langchain>=0.2.1" "deepagents==0.7.21" langgraph
```

With `traceAI-langchain` 0.2.0 or earlier, `thread_id` is not copied to `session.id`: upgrade, or wrap the call in `using_session(thread_id)`.

```python
from deepagents import create_deep_agent
from deepagents.backends import StateBackend
from fi_instrumentation import TraceConfig, register
from fi_instrumentation.fi_types import ProjectType
from traceai_langchain import LangChainInstrumentor

# 1. register(), 2. instrument(), 3. build the agent.
trace_provider = register(project_type=ProjectType.OBSERVE, project_name="deep-agents-app")
LangChainInstrumentor().instrument(
tracer_provider=trace_provider,
# Filesystem tool arguments and results can contain file contents.
config=TraceConfig(hide_inputs=True, hide_outputs=True),
)

# Pass a model explicitly: model=None selects the deprecated claude-sonnet-4-6 default.
agent = create_deep_agent(model=model, tools=[my_tool], backend=StateBackend())
result = agent.invoke(
{"messages": [{"role": "user", "content": "..."}]},
config={"configurable": {"thread_id": "t-1"}}, # becomes session.id (0.2.1+; with 0.2.0 use using_session)
)
trace_provider.force_flush()
```

What you get (all from `traceAI-langchain`; Deep Agents adds no keys):

- **LLM** spans with `gen_ai.request.model` and `gen_ai.usage.*` from the model's
`usage_metadata`. Token counts are only on LLM spans, so a trace's token total is
the sum of its model calls.
- **TOOL** spans named after the tool that ran. With the default `StateBackend`,
0.7.21 binds `ls`, `read_file`, `write_file`, `edit_file`, `delete`, `glob`,
`grep` and `task`, plus your tools. `execute` is bound only for a sandbox backend.
`write_todos` is not a default tool at this version; harness profiles can add tools
(the OpenAI Codex profile adds `write_todos` for `openai:gpt-5.x-codex` models).
- **Subagents**: the `task` TOOL span is the parent of the subagent's span (named
after the subagent, e.g. `general-purpose`, kind `CHAIN`) and of its model and
tool spans, all in one trace. Subagent spans also carry `lc_agent_name`.
- **Graph nodes** (`model`, `tools`, middleware nodes) are `CHAIN` spans; a node whose
name contains "agent" (e.g. `PatchToolCallsMiddleware.before_agent`) is reported as
`AGENT` by the existing name heuristic.
- **Session**: `configurable.thread_id` → `session.id` on every span, subagents included
(0.2.1 and later; see the session note above).
- **Errors**: a tool that raises ends its TOOL span `ERROR` with an `exception`
event (LangGraph's default tool error handler re-raises, so `invoke` raises too).
The status description is the exception message, without a traceback. Stacktraces
remain in exception events only when error details are visible. With core 1.2.0+,
`TraceConfig.hide_error_details` controls this: content-hide flags derive it by
default, and an explicit `False` keeps error details. With older core versions,
`hide_inputs=True` or `hide_outputs=True` also hides error details. Hidden failures
keep their exception type, but no status description, message, stacktrace or
provider-error output (body, response headers or request ids).
Cancelling an `astream` early closes every span; the root span ends `ERROR`
with a `GeneratorExit` exception event (asserted by the compatibility test).
- **CI**: the Deep Agents job installs the real `deepagents==0.7.21` wheel and runs
the compatibility, session and error-policy tests with the current core. It also
checks input/output privacy and sessions against published core 1.1.0.

Not emitted: cost, user id (set `using_attributes(user_id=...)` yourself),
retrieval (Deep Agents has no retriever tool), and a `gen_ai.provider.name`
(LangChain's raw `ls_provider` metadata is passed through as-is).

---

## Examples

### LangChain Examples
Expand All @@ -155,6 +249,9 @@ Node, tool and LLM runs are standard LangChain spans (`gen_ai.span.kind` =
- `examples/langgraph_agent_supervisor.py` - Multi-agent supervisor pattern
- `examples/langgraph_human_in_the_loop.py` - Human-in-the-loop interrupt workflow

### Deep Agents Example
- `examples/deep_agents.py` - `create_deep_agent` with a custom tool, built-in file tools and a `task` subagent (runs offline)

---

## API Reference
Expand Down
Loading
Loading