Add CrewAI agent and workflow instrumentation by venkata-pavani · Pull Request #384 · open-telemetry/opentelemetry-python-genai

venkata-pavani · 2026-08-14T01:54:57Z

Description

I implemented and tested invoke_agent() and invoke_workflow() mechanism. I was able to reproduce and test this via Conformance test , VCR tests and was able confirm this by installing local Grafana tempo test

Features

invoke_agent () and invoke_workflow()

Grafana Logs

Also added the feature for enabling or disabling native telemetry as mentioned in the the comments below (in a less invasive way) using instrument() and uninstrument()

#308 (review)

#363 (comment)

Why I did above?

I followed the same pattern that we have for OpenAI-agents framework

https://github.com/open-telemetry/opentelemetry-python-genai/blob/main/instrumentation/opentelemetry-instrumentation-genai-openai-agents/src/opentelemetry/instrumentation/genai/openai_agents/__init__.py

Please let me know if this works

Future PR

Planning to implement other features like plan() and execute() for this instrumentation

Assisted-by: ChatGPT 5.6

opentelemetry-pr-dashboard · 2026-08-20T15:38:31Z

Pull request dashboard status

Waiting on the author · refreshed 2026-09-01 13:19 UTC

Two things need attention:

Required checks are failing — investigate the failures.
11 review items — respond to each (e.g. link a commit, explain why not, ask a follow-up):
- Inline threads: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11

Status above doesn't look right?

Just replied or pushed? Anything around or after the refresh time above may not be picked up yet — give it a few minutes.
Should this be with reviewers? Comment /dashboard route:reviewers to route it to them.
Anything wrong — including the routing? Report it with what you expected; it helps us improve the dashboard.

# Conflicts: # tox.ini

Assisted-by: ChatGPT 5.6

Resolve the tox matrix conflict while retaining CrewAI conformance and upstream DSPy environments. Assisted-by: ChatGPT 5.6

Assisted-by: ChatGPT 5.6

…i-orchestration

Copilot

Pull request overview

Adds CrewAI orchestration-level instrumentation to the opentelemetry-instrumentation-genai-crewai package, translating CrewAI event-bus signals into GenAI semantic-convention spans/metrics via opentelemetry-util-genai, and wires in tests (unit, VCR, and conformance).

Changes:

Introduces a CrewAIEventListener that turns CrewAI kickoff/agent execution events into invoke_workflow and invoke_agent telemetry.
Adds CrewAI test coverage: unit tests for event translation/instrumentor behavior, VCR-backed integration tests, and a Weaver conformance scenario.
Updates CI/test wiring and docs: tox conformance env + ignore rules, README behavior notes, dependency floor bump, and CrewAI OTel override pins.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
tox.ini	Adds CrewAI conformance env and splits unit vs conformance test execution.
instrumentation/opentelemetry-instrumentation-genai-crewai/src/opentelemetry/instrumentation/genai/crewai/event_listener.py	New listener translating CrewAI events into util-genai invocations.
instrumentation/opentelemetry-instrumentation-genai-crewai/src/opentelemetry/instrumentation/genai/crewai/init.py	Instantiates/shuts down the CrewAI event listener during (un)instrumentation.
instrumentation/opentelemetry-instrumentation-genai-crewai/tests/conftest.py	Enables shared VCR/fixture plugins and adds cassette scrubbing config.
instrumentation/opentelemetry-instrumentation-genai-crewai/tests/test_operations.py	New VCR + unit tests asserting workflow/agent spans and error handling.
instrumentation/opentelemetry-instrumentation-genai-crewai/tests/test_instrumentor.py	Adds tests ensuring env vars and existing event handlers are preserved.
instrumentation/opentelemetry-instrumentation-genai-crewai/tests/test_conformance.py	New conformance runner for the CrewAI scenario.
instrumentation/opentelemetry-instrumentation-genai-crewai/tests/conformance/orchestration.py	New conformance scenario emitting Crew kickoff + agent execution events.
instrumentation/opentelemetry-instrumentation-genai-crewai/tests/conformance/init.py	Package marker for conformance scenarios.
instrumentation/opentelemetry-instrumentation-genai-crewai/tests/overrides.txt	Extends CrewAI OpenTelemetry override set for coherent workspace OTel versions.
instrumentation/opentelemetry-instrumentation-genai-crewai/tests/cassettes/successful_kickoff.yaml	New VCR cassette for “successful kickoff” path.
instrumentation/opentelemetry-instrumentation-genai-crewai/tests/cassettes/failed_kickoff.yaml	New VCR cassette for failure path (marked AI-generated).
instrumentation/opentelemetry-instrumentation-genai-crewai/README.rst	Documents emitted operations and interaction with CrewAI native telemetry.
instrumentation/opentelemetry-instrumentation-genai-crewai/pyproject.toml	Bumps required `opentelemetry-util-genai` lower bound.
instrumentation/opentelemetry-instrumentation-genai-crewai/.changelog/384.added	Towncrier fragment describing added orchestration instrumentation.

Suppressed comments (1)

instrumentation/opentelemetry-instrumentation-genai-crewai/tests/cassettes/successful_kickoff.yaml:291

This second interaction also records a request "cookie" header with Cloudflare cookie tokens. Replace with a placeholder or remove the header to avoid committing sensitive values.

      cookie:
      - __cf_bm=cJo9TcrHf.ofxIJyDuI6NxLwNP9gg7Pcz8RhrFPt7qE-1787265811.7414775-1.0.1.1-eWjq4MqWRZLWLbmxId_inKMEy6rqVd7zOsHEUxd83zhK9H_SyTHB_dFe9Wmk8qn07Ygt9q7mWtYx_hw9Wi0GU.Rf3veGcyWpcTDfzouXL_CvMYkfuS1gcY5MXZ3OLEt5;
        _cfuvid=am9oQZ2E39h_JipRfaOgJGpOupaxcDFPq2EhN5ty32U-1787265811.7414775-1.0.1.1-yOAI9bg2q69lpQNX5ZG6e2b9osZqDcZViWZHOWn7sIU

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

    ; instrumentation-genai-crewai
-    py3{10,14}-test-instrumentation-genai-crewai-latest
+    py3{10,13}-test-instrumentation-genai-crewai-latest
    py310-test-instrumentation-genai-crewai-oldest
+    py313-test-instrumentation-genai-crewai-conformance


+      cookie:
+      - __cf_bm=cJo9TcrHf.ofxIJyDuI6NxLwNP9gg7Pcz8RhrFPt7qE-1787265811.7414775-1.0.1.1-eWjq4MqWRZLWLbmxId_inKMEy6rqVd7zOsHEUxd83zhK9H_SyTHB_dFe9Wmk8qn07Ygt9q7mWtYx_hw9Wi0GU.Rf3veGcyWpcTDfzouXL_CvMYkfuS1gcY5MXZ3OLEt5;
+        _cfuvid=am9oQZ2E39h_JipRfaOgJGpOupaxcDFPq2EhN5ty32U-1787265811.7414775-1.0.1.1-yOAI9bg2q69lpQNX5ZG6e2b9osZqDcZViWZHOWn7sIU


xrmx · 2026-08-24T14:46:17Z

@@ -0,0 +1,38 @@
+# TODO: this is generated by AI, re-record


lmolkova · 2026-08-25T04:46:30Z

+            invocation, AgentInvocation
+        ):
+            invocation.fail(
+                _error(event.error, "CrewAI agent execution failed")


"CrewAI agent execution failed"

why? we don't need to invent anything that lib does not report

hi @lmolkova ,

Instead of using manually written error messages, I now use the error information reported by CrewAI

lmolkova · 2026-08-25T04:47:45Z

+    return (
+        value
+        if isinstance(value, BaseException)
+        else RuntimeError(str(value or fallback))


no need to invent exception type, you can use Error object from utils that has error type and error message - the type of the value is always available and no need for fallback

lmolkova · 2026-08-25T04:48:05Z

+    )
+
+
+def _input_message(value: object) -> InputMessage | None:


is there a more precise type than object?

lmolkova · 2026-08-25T04:48:47Z

+        content = value
+    else:
+        try:
+            content = json.dumps(value, default=str)


this does not look right. we need to understand the structure of this value to properly convert it into input message part

lmolkova · 2026-08-25T04:49:18Z

+def _output_message(value: object) -> OutputMessage | None:
+    if value is None:
+        return None
+    content = value if isinstance(value, str) else str(value)


same here, we should support all modalities - please take a look at how openai anthropic or google-genai do it.

lmolkova · 2026-08-25T04:49:40Z

+        self._handlers.append((event_type, registered))
+
+    def _remember(self, event: BaseEvent, invocation: GenAIInvocation) -> None:
+        with self._lock:


do we have to have a lock here?

lmolkova · 2026-08-25T04:50:23Z

+
+    def _remember(self, event: BaseEvent, invocation: GenAIInvocation) -> None:
+        with self._lock:
+            previous = self._invocations.setdefault(event.event_id, invocation)


do we have to accumulate state? can we instrument without it?

lmolkova · 2026-08-25T04:51:15Z

+        if not is_instrumentation_enabled():
+            return
+        invocation = self._telemetry_handler.workflow(event.crew_name)
+        message = _input_message(event.inputs)


why just one? there is clearly more than one input

lmolkova · 2026-08-25T04:51:59Z

+        invocation = self._telemetry_handler.invoke_local_agent(
+            agent_name=agent.role
+        )
+        invocation.agent_id = str(agent.id)


agent id should not be used for local agents, it's only for remote ones

lmolkova · 2026-08-25T04:53:18Z

+            crew = CrewKickoffStartedEvent(
+                crew_name="research crew",
+                crew=None,
+                inputs={"topic": "OpenTelemetry"},
+            )
+            _emit(crew)
+
+            agent = SimpleNamespace(
+                id="agent-1", role="Researcher", goal="Research"
+            )
+            agent_started = AgentExecutionStartedEvent.model_construct(
+                event_id="agent-start",
+                started_event_id=None,
+                agent=agent,
+                task=object(),
+                tools=None,
+                task_prompt="Research OpenTelemetry",
+            )
+            _emit(agent_started)
+
+            _emit(
+                AgentExecutionCompletedEvent.model_construct(
+                    event_id="agent-complete",
+                    started_event_id=agent_started.event_id,
+                    agent=agent,
+                    task=object(),
+                    output="report",
+                )
+            )
+            _emit(
+                CrewKickoffCompletedEvent(
+                    crew_name="research crew",
+                    crew=None,
+                    output="report",
+                    started_event_id=crew.event_id,
+                )
+            )


this is supposed to be top level api like crew kickoff, not hand-made combination of events

xrmx · 2026-08-24T14:51:07Z

+handlers registered by the application or other integrations.
+
+To emit only OpenTelemetry GenAI telemetry, users can disable CrewAI's native
+telemetry before importing CrewAI while leaving this instrumentation enabled::


Suggested change

telemetry before importing CrewAI while leaving this instrumentation enabled::

telemetry before importing CrewAI while leaving this instrumentation enabled:

::

opentelemetry-pr-dashboard · 2026-08-31T13:43:40Z

Hi @venkata-pavani — just a friendly reminder that this pull request is waiting on you. The dashboard status comment has the open items and is kept current.

Replying is enough to hand it off — answer, explain why no change is needed, or ask a follow-up. The dashboard routes it onward once nothing on the list is waiting on you.
To hand it back for any other reason, including the dashboard getting this wrong, comment /dashboard route:reviewers.

Assisted-by: ChatGPT 5.6

Add CrewAI agent and workflow instrumentation

59546de

Assisted-by: ChatGPT 5.6

opentelemetry-pr-dashboard Bot mentioned this pull request Aug 14, 2026

Pull Request Dashboard #171

Open

lmolkova mentioned this pull request Aug 14, 2026

Tracking: migrate OpenInference Python instrumentations into this repo #141

Open

venkata-pavani added 3 commits August 20, 2026 17:42

Merge remote-tracking branch 'upstream/main' into crewai-orchestration

bf22d24

# Conflicts: # tox.ini

test(crewai): cover telemetry coexistence

87d9c32

Assisted-by: ChatGPT 5.6

Merge upstream main into crewai-orchestration

093419c

Resolve the tox matrix conflict while retaining CrewAI conformance and upstream DSPy environments. Assisted-by: ChatGPT 5.6

venkata-pavani mentioned this pull request Aug 24, 2026

adding crew ai instrumentation #363

Closed

2 tasks

venkata-pavani added 4 commits August 24, 2026 00:03

Merge branch 'main' into crewai-orchestration

be7dadd

Merge branch 'main' into crewai-orchestration

6a39d30

fix(crewai): stabilize orchestration CI

0a1d500

Assisted-by: ChatGPT 5.6

Merge remote-tracking branch 'origin/crewai-orchestration' into crewa…

df5e103

…i-orchestration

venkata-pavani marked this pull request as ready for review August 24, 2026 13:03

venkata-pavani requested a review from a team as a code owner August 24, 2026 13:03

Copilot AI lite review requested due to automatic review settings August 24, 2026 13:03

Merge branch 'main' into crewai-orchestration

e9368ea

Copilot started reviewing on behalf of venkata-pavani August 24, 2026 13:05 View session

Copilot AI reviewed Aug 24, 2026

View reviewed changes

github-project-automation Bot added this to GenAI Semantic Conventions and Instrumentation libraries Aug 24, 2026

xrmx reviewed Aug 24, 2026

View reviewed changes

lmolkova requested changes Aug 25, 2026

View reviewed changes

xrmx reviewed Aug 25, 2026

View reviewed changes

venkata-pavani added 2 commits September 1, 2026 08:10

Merge branch 'main' into crewai-orchestration

f9d344e

fix(crewai): preserve reported error details

2772f77

Assisted-by: ChatGPT 5.6

Uh oh!

Conversation

venkata-pavani commented Aug 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Features

Grafana Logs

Why I did above?

Future PR

Uh oh!

opentelemetry-pr-dashboard Bot commented Aug 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull request dashboard status

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

opentelemetry-pr-dashboard Bot commented Aug 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

venkata-pavani commented Aug 14, 2026 •

edited

Loading

opentelemetry-pr-dashboard Bot commented Aug 20, 2026 •

edited

Loading