From e4bc6a455a3e364c79c0d2ed05f529eb860bd4a4 Mon Sep 17 00:00:00 2001 From: Ling-Sen Peng Date: Sat, 18 Jul 2026 14:39:01 -0700 Subject: [PATCH 1/2] docs(examples): add example-catalog verification report; document agent server-version requirement MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Report-only PR — no code changes. Tracks the verified status of every SDK example against conductor-oss 3.32.0-rc.8 (agents suite: 96 PASS / 46 SKIP / 2 ERROR / 2 HUNG with the companion fixes PR applied; 3 PASS / 103 ERROR on the CLI's `latest` 3.30.x server), the bugs found, and the remaining known failures. Fixes are proposed separately as a draft PR. Docs: the agent examples require conductor-oss >= 3.32.0-rc.8 (agent runtime on by default); documented in examples/agents/README.md and examples/README.md with `conductor server start --version 3.32.0-rc.8`. Also fixed the provider/model table (OpenAI row showed the Anthropic default string). Co-Authored-By: Claude Fable 5 --- examples/README.md | 6 ++ examples/VERIFICATION_REPORT.md | 166 ++++++++++++++++++++++++++++++++ examples/agents/README.md | 36 ++++++- 3 files changed, 203 insertions(+), 5 deletions(-) create mode 100644 examples/VERIFICATION_REPORT.md diff --git a/examples/README.md b/examples/README.md index 5c5304d0d..53f6b5968 100644 --- a/examples/README.md +++ b/examples/README.md @@ -89,6 +89,12 @@ Durable agent authoring (`Agent`, `AgentRuntime`, tools, guardrails, handoffs, m strategies) — a separate, more extensive catalog of 270+ examples, requiring `pip install 'conductor-python[agents]'`. See [agents/README.md](agents/README.md). +> **Server version:** the agent examples require a Conductor server with the agent +> runtime — conductor-oss **`3.32.0-rc.8` or newer**. The stable `latest` installed by +> `conductor server start` (3.30.x) does not expose `/api/agent/*` and every agent +> example fails with `AgentNotFoundError: HTTP 404`. Use +> `conductor server start --version 3.32.0-rc.8`. + --- ### Monitoring diff --git a/examples/VERIFICATION_REPORT.md b/examples/VERIFICATION_REPORT.md new file mode 100644 index 000000000..ad134ca8d --- /dev/null +++ b/examples/VERIFICATION_REPORT.md @@ -0,0 +1,166 @@ +# Examples Verification Report + +**Date:** 2026-07-18 · **Base:** `7481aed2` + +Full verification of the SDK example catalog against a live Conductor server. This +report tracks the status of each suite so it can be re-checked as the SDK and server +evolve. **This PR is report + documentation only** — fixes for the bugs found are +proposed separately as a draft PR, pending a decision on whether to take them. + +## Environment + +| Component | Value | +|-----------|-------| +| Server | conductor-oss **3.32.0-rc.8** boot JAR (Maven Central), `java -jar --server.port=8080` | +| Python | 3.12.13 (uv venv), `pip install -e '.[agents]'` | +| Platform | macOS (darwin arm64) — worker processes use the **spawn** start method | +| LLM | `AGENTSPAN_LLM_MODEL=openai/gpt-4o-mini`, `OPENAI_API_KEY` + `ANTHROPIC_API_KEY` set | +| Harness | `examples/agents/run_all_examples.py --jobs 8 --timeout 360` + curated core-example runner | + +### ⚠️ Server version requirement (root cause of the most common failure) + +The agent examples need the **agent runtime**, on by default from conductor-oss +**3.32.0-rc.8** (the version pinned in `.github/workflows/agent-e2e.yml`). The `latest` +JAR installed by `conductor server start` is the 3.30.x stable line, which has **no +`/api/agent/*` endpoints** — every agent example fails with +`AgentNotFoundError: HTTP 404 ... api/agent/start`. On 3.30.2 the suite scored +3 PASS / 103 ERROR; on 3.32.0-rc.8 (same code) it scored 96 PASS / 2 ERROR. + +Workaround (works today, `conductor-server-3.32.0-rc.8.jar` is already in the CLI's S3 +bucket): `conductor server start --version 3.32.0-rc.8`. Decision: do **not** repoint +the CLI's floating `latest` at an rc — this resolves itself when 3.32.0 GA ships. + +## Results — agents suite (`examples/agents/`, 146 files) + +`run_all_examples.py` with the fixes from the companion fixes PR applied: + +| Status | Count | Notes | +|--------|------:|-------| +| PASS | 96 | end-to-end against the live server, real LLM calls | +| SKIP | 46 | interactive / daemon-by-design / external infra (MCP, Docker, Kafka, Slack, OCG, skills repo, messages API) | +| ERROR | 2 | see "Remaining failures" | +| HUNG | 2 | see "Remaining failures" | + +Before fixes (same server): 91 PASS / 12 ERROR / 3 HUNG / 40 SKIP. + +## Results — core suite (curated, 18 examples) + +| Example | Status | Notes | +|---------|--------|-------| +| workers_e2e.py | ✅ fixed | workers run clean; e2e workflow COMPLETED server-side; blocks on `join_processes()` by design | +| worker_example.py | ✅ fixed | runs clean as a worker daemon | +| task_listener_example.py | ✅ fixed | all workers start, listeners fire, no spawn errors | +| helloworld/helloworld.py | ✅ fixed | exits 0 | +| worker_configuration_example.py | ✅ | | +| task_workers.py | ✅ | | +| dynamic_workflow.py | ✅ | | +| workflow_ops.py | ✅ | | +| workflow_status_listner.py | ✅ | | +| kitchensink.py | ✅ | | +| task_configure.py | ✅ | | +| metadata_journey_oss.py | ✅ | | +| schedule_journey.py | ✅ | | +| event_listener_examples.py | ✅ | | +| lease_extension_example.py | ✅ | ~62s | +| agentic_workflows/llm_chat.py | ✅ fixed | previously a **false pass** — reported success while the workflow FAILED server-side | +| task_context_example.py | ➖ daemon | runs until Ctrl+C by design; workers run clean | +| agentic_workflows/function_calling_example.py | ➖ interactive | reads stdin; not runnable headless | + +Static check: every `.py` under `examples/` compiles (`compileall`). Unit tests: +`tests/unit/ai` — 1744 passed. + +## Bugs found (fixes proposed in the companion draft PR) + +Statuses marked "fixed" below and in the tables above reflect runs **with the +companion PR's fixes applied**; without them, each item reproduces as described. + +1. **`examples/helloworld/greetings_worker.py` — duplicate `def greet`.** The module + defined `greet` twice (tasks `greet` and `greet_sync`); the second shadowed the + first, so spawn-pickling the first failed with + `PicklingError: not the same object as helloworld.greetings_worker.greet`. Broke + `workers_e2e.py`, `helloworld.py`, and `task_listener_example.py` on macOS/Windows. + Renamed the second function `greet_sync`. + +2. **`src/conductor/ai/agents/tool.py` — SDK bug: worker-task tool lookup broken after + re-registration.** `ToolRegistry.register_tool_workers` overwrites a + `@worker_task` tool's `_decorated_functions` entry with a spawn-safe + `ToolWorkerEntry` wrapper; the identity check in `_try_worker_task` then failed for + the same tool later in the same run + (`TypeError: Expected a @tool-decorated function ...`). Broke + `14_existing_workers.py`. The lookup now walks `__wrapped__` chains and the + wrapper's `fn_direct`/`fn_ref` carriers. Two regression tests added. + +3. **`examples/worker_example.py` — hardcoded `/Users/viren/` metrics path.** + `PermissionError` on any other machine. Now uses `tempfile.gettempdir()`. + +4. **`examples/user_example/user_workers.py` — inconsistent package identity.** + Imported `examples.user_example.models` while every consumer loads the module as + `user_example.user_workers`; spawn children (which only inherit `PYTHONPATH`, not + runtime `sys.path` edits) crashed. Import is now package-relative-consistent. + +5. **`examples/agents/kitchen_sink.py` — `@agent` classifiers declared a required + `prompt` arg.** `@agent`-decorated functions are invoked with zero args at compile + time for dynamic instructions → `TypeError`. Signatures fixed. + +6. **`examples/agents/16i/16j` — tools defined inside factory functions.** + `` callables can't be re-imported by spawn workers → + `SpawnSafetyError`. Tools hoisted to module level; both now pass. + +7. **`examples/agents/94_openai_runner_tools.py` — `@function_tool` rebinding.** The + decorator rebinds the module global to a `FunctionTool`, so the extracted original + couldn't be pickled by reference. The example now keeps the plain function at module + level and applies `function_tool()` at `Agent(...)` construction; passes end-to-end. + +8. **`examples/agentic_workflows/llm_chat.py` — two bugs.** (a) Two LLM tasks sent + **system-only** message lists; the server requires a non-empty user message and + failed the workflow before the first task ran. (b) The example treated any terminal + state as success (`is_completed()`), printing "Conversation complete." over a FAILED + workflow and exiting 0. Both fixed; a full 3-turn conversation now completes. + +9. **`examples/agents/run_all_examples.py` — skip-list updates.** Six examples that + need infra this environment (and CI's pinned server) don't provide are now + classified SKIP with reasons: `30`/`32` (need the `dg` skill cloned into + `~/.claude/skills/dg`), `75`/`82`/`83`/`84` (need the workflow *messages* API, + `GET /api/workflow/{id}/messages`, which is not in conductor-oss 3.32.0-rc.8). + +10. **`examples/agents/README.md`** — added the server-version requirement + start + commands; fixed the provider/model table (OpenAI row showed the Anthropic default + string); renumbered setup steps. **`examples/README.md`** — added the same server + note to the AI Agents section. + +## Remaining failures (known, not fixed here) + +| Example | Status | Diagnosis | +|---------|--------|-----------| +| `agents/kitchen_sink.py` | ERROR | Fixed classifier TypeError, now fails server-side: `HTTP 500 — The Task translation_swarm defined as a sub-workflow has no workflow definition available`. Sub-workflow (SWARM strategy) definitions aren't registered before `/agent/start` on conductor-oss rc.8. Needs SDK/server investigation. | +| `agents/74_cli_error_output.py` | FLAKY | LLM-behavior assertion: the agent paraphrases stderr instead of quoting it verbatim; passes/fails depending on model output. Consider loosening the assertion or pinning a stricter prompt. | +| `agents/86_coding_agent.py` | HUNG | Does not finish within 420s even running solo. Needs investigation (or reclassification if it is expected to be long-running). | +| `agents/68_context_condensation.py` | SLOW | Passes solo in ~346s; exceeds the suite timeout under 8-way contention. Consider `--timeout 480` for suite runs or trimming the example. | + +## SDK observations (follow-ups, out of scope for this PR) + +- **`AgentNotFoundError` should name the server-version requirement.** When + `/api/agent/start` 404s ("No static resource"), the error in + `orkes_agent_client.py` should suggest conductor-oss ≥ 3.32.0-rc.8 and the + `conductor server start --version` command. This would have made the most common + failure self-explanatory. +- **`run_all_examples.py` preflight**: check `GET /api/version` + probe an agent + endpoint before launching 100+ subprocesses against a server that can't run them. +- **openai-agents `@function_tool` at module level is spawn-unsafe by construction** + (the global is rebound to a `FunctionTool`; the original function is only reachable + through a closure, which `FunctionRef` can't express). Teaching the spawn-safety + layer a closure hop — or documenting the `function_tool(fn)`-at-construction pattern + — would let users keep the upstream OpenAI sample shape. + +## How to reproduce + +```bash +# server +curl -fL -o conductor-server.jar "https://repo1.maven.org/maven2/org/conductoross/conductor-server/3.32.0-rc.8/conductor-server-3.32.0-rc.8-boot.jar" +OPENAI_API_KEY=... ANTHROPIC_API_KEY=... java -jar conductor-server.jar --server.port=8080 + +# SDK + suite +uv venv --python 3.12 .venv && VIRTUAL_ENV=$PWD/.venv uv pip install -e '.[agents]' +export CONDUCTOR_SERVER_URL=http://localhost:8080/api AGENTSPAN_LLM_MODEL=openai/gpt-4o-mini +python examples/agents/run_all_examples.py --jobs 8 --timeout 360 +``` diff --git a/examples/agents/README.md b/examples/agents/README.md index 5a68f0bc1..29805435f 100644 --- a/examples/agents/README.md +++ b/examples/agents/README.md @@ -120,7 +120,33 @@ To install all framework dependencies at once: uv pip install langchain langchain-core langchain-openai langgraph openai-agents google-adk ``` -### 2. Configure your environment +### 2. Start a Conductor server + +The agent examples need a Conductor server with the **agent runtime**, which is on by +default from **conductor-oss `3.32.0-rc.8`** onward (the same version pinned by this +repo's agent-e2e CI). Older servers — including the `latest` stable line installed by +`conductor server start` (3.30.x at the time of writing) — do not expose the +`/api/agent/*` endpoints, and every example fails with +`AgentNotFoundError: HTTP 404 ... api/agent/start`. + +Start a known-good version with the [Conductor CLI](https://github.com/conductor-oss/conductor-cli): + +```bash +conductor server start --version 3.32.0-rc.8 +``` + +Or run the boot JAR from Maven Central directly: + +```bash +curl -fL -o conductor-server.jar \ + "https://repo1.maven.org/maven2/org/conductoross/conductor-server/3.32.0-rc.8/conductor-server-3.32.0-rc.8-boot.jar" +java -jar conductor-server.jar --server.port=8080 +``` + +Export your LLM provider API keys (e.g. `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`) in the +shell that starts the server — the server auto-enables the matching providers. + +### 3. Configure your environment Export environment variables: @@ -131,14 +157,14 @@ export AGENTSPAN_SERVER_URL=http://localhost:8080/api # export AGENTSPAN_AUTH_SECRET= ``` -#### 2.1. Choose a model +#### 3.1. Choose a model The `AGENTSPAN_LLM_MODEL` variable uses the `provider/model-name` format. Examples: | Provider | Model string | API key env var | |----------|-------------|-----------------| -| OpenAI | `anthropic/claude-sonnet-4-6` (default) | `OPENAI_API_KEY` | -| Anthropic | `anthropic/claude-sonnet-4-20250514` | `ANTHROPIC_API_KEY` | +| OpenAI | `openai/gpt-4o-mini` | `OPENAI_API_KEY` | +| Anthropic | `anthropic/claude-sonnet-4-6` (default) | `ANTHROPIC_API_KEY` | | Google Gemini | `google_gemini/gemini-2.0-flash` | `GOOGLE_GEMINI_API_KEY` | | AWS Bedrock | `aws_bedrock/...` | AWS credentials | | Azure OpenAI | `azure_openai/...` | Azure credentials | @@ -147,7 +173,7 @@ All supported providers: `openai`, `anthropic`, `google_gemini`, `google_vertex_ `azure_openai`, `aws_bedrock`, `cohere`, `mistral`, `groq`, `perplexity`, `hugging_face`, `deepseek`. -### 3. Run an example +### 4. Run an example ```bash # Core SDK examples From 8a7deb51da214d8d75a8e4465401607dc7688adf Mon Sep 17 00:00:00 2001 From: Ling-Sen Peng Date: Sat, 18 Jul 2026 14:40:48 -0700 Subject: [PATCH 2/2] docs(examples): cross-link verification report to fixes PR #439 Co-Authored-By: Claude Fable 5 --- examples/VERIFICATION_REPORT.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/examples/VERIFICATION_REPORT.md b/examples/VERIFICATION_REPORT.md index ad134ca8d..872d18b96 100644 --- a/examples/VERIFICATION_REPORT.md +++ b/examples/VERIFICATION_REPORT.md @@ -5,7 +5,9 @@ Full verification of the SDK example catalog against a live Conductor server. This report tracks the status of each suite so it can be re-checked as the SDK and server evolve. **This PR is report + documentation only** — fixes for the bugs found are -proposed separately as a draft PR, pending a decision on whether to take them. +proposed separately in draft PR +[#439](https://github.com/conductor-oss/python-sdk/pull/439), pending a decision on +whether to take them. ## Environment @@ -32,7 +34,7 @@ the CLI's floating `latest` at an rc — this resolves itself when 3.32.0 GA shi ## Results — agents suite (`examples/agents/`, 146 files) -`run_all_examples.py` with the fixes from the companion fixes PR applied: +`run_all_examples.py` with the fixes from the companion fixes PR (#439) applied: | Status | Count | Notes | |--------|------:|-------| @@ -69,7 +71,7 @@ Before fixes (same server): 91 PASS / 12 ERROR / 3 HUNG / 40 SKIP. Static check: every `.py` under `examples/` compiles (`compileall`). Unit tests: `tests/unit/ai` — 1744 passed. -## Bugs found (fixes proposed in the companion draft PR) +## Bugs found (fixes proposed in the companion draft PR, #439) Statuses marked "fixed" below and in the tables above reflect runs **with the companion PR's fixes applied**; without them, each item reproduces as described.