Skip to content

[AAASM-6445] 🐛 (python-sdk): Govern LangChain tool calls from init_assembly() - #389

Merged
Chisanan232 merged 16 commits into
mainfrom
v0.0.1/AAASM-6445/fix/langchain_govern_on_init
Nov 10, 2026
Merged

Chisanan232 merged 16 commits into
mainfrom
v0.0.1/AAASM-6445/fix/langchain_govern_on_init

Conversation

@Chisanan232

Copy link
Copy Markdown
Contributor

Behaviour change

init_assembly() alone now governs LangChain tool calls, as the README Quick Start has always claimed. Before, the SDK only stored an AssemblyCallbackHandler singleton and never gave it to LangChain, so a denied tool still executed (QA J08: TOOL_EXECUTED True). Now an allowed tool runs, a denied tool never runs, and a runtime or handler failure under enforce blocks the tool. No application change and no global monkey-patching: the adapter registers one process-wide proxy handler with langchain_core's public register_configure_hook.

Owner decision 2026-11-06 applied: original J08 outcome kept, no docs-only reduction. ADR 0033 E2 ("checkpoint reachable only if the agent opts in") is satisfied by calling init_assembly(); no ADR conflict, no init-contract change.

Evidence (real stack, exact README quick start)

Real aa-gateway + aa-runtime (F3 binaries, isolated HOME, harness LocalBackend), policy durable via AA_POLICY (not aasm policy apply), README FakeListLLM code verbatim (denied variant only renames the tool to delete_file and adds a sentinel file write). Wheel = candidate rc.8 wheel with these two Python files overlaid (see "Not yet verified").

  • allowed whoami: output alice is in engineering, sentinel written; audit ToolCallIntercepted action=whoami verdict=allow.
  • denied delete_file: ToolExecutionBlockedError: tool denied by policy, sentinel absent; audit PolicyViolation action=delete_file verdict=deny policy_rule=tool:delete_file.
  • control (unmodified candidate wheel): denied tool executed, sentinel present.
  • aa-runtime killed: tool blocked with runtime unreachable; failing closed under enforce, sentinel absent.

Design

ContextVar(default=ProcessGovernanceHandler()) registered once with register_configure_hook(var, True, AssemblyCallbackHandler). The default (not a set()) is deliberate: a plain threading.Thread has an empty context and would skip the gate. handle_class makes LangChain skip the proxy if the app passes a handler itself (no double check). The proxy forwards tool start/end to the active handler and does nothing after shutdown (the hook cannot be unregistered). raise_error=True is inherited so LangChain never swallows a failure.
Rejected: explicit callbacks=[handler] in the README (rewrites the promise, an API change); global monkey-patching of BaseTool.

Tests

test/unit/test_quickstart_langchain_governs_on_init.py (needs langchain-test group; skipped in the default suite): allow, deny, plain thread, async, no double check, runtime query failure, handler exception, observe, shutdown. Full suite 1454 passed; ruff clean; mypy clean on the changed files (5 pre-existing agent_assembly._core import errors in a worktree without the native build).

Not yet verified

The rebuilt maturin wheel: the E2E used the candidate wheel plus a Python overlay. Rebuild and re-pin to confirm on the final artifact. LLM-scan callbacks (on_llm_start) are intentionally not enabled globally.

Other PRs

Supersedes #387 (docs-only; its README text is now false, close it). Independent of #388 (snippet resync); README Quick Start code is unchanged here, so either order merges.

Jira: https://lightning-dust-mite.atlassian.net/browse/AAASM-6445

🤖 Generated with Claude Code

https://claude.ai/code/session_01TLpw696KjUDUg4QjH83En4

Chisanan232 and others added 3 commits November 6, 2026 09:15
init_assembly() stored an AssemblyCallbackHandler but never gave it to
LangChain, so a denied tool still ran. Register a process-wide proxy handler
through langchain_core's public register_configure_hook (default-valued
ContextVar so plain threads are covered; handle_class dedupes an explicitly
passed handler). raise_error stays True so any handler failure blocks the tool.

Refs AAASM-6445

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TLpw696KjUDUg4QjH83En4
Refs AAASM-6445

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TLpw696KjUDUg4QjH83En4
Refs AAASM-6445

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TLpw696KjUDUg4QjH83En4
@codecov

codecov Bot commented Nov 6, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.58824% with 3 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...gent_assembly/adapters/langchain/configure_hook.py 93.18% 3 Missing ⚠️

📢 Thoughts on this report? Let us know!

Chisanan232 and others added 7 commits November 6, 2026 09:31
…lready allowed

handle_class made LangChain drop the process-wide proxy whenever any
AssemblyCallbackHandler was in callbacks, so a handler built around a weaker
interceptor silently suppressed the active governance. Track allowed runs by
interceptor instead.

Refs AAASM-6445

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TLpw696KjUDUg4QjH83En4
Also narrow the runtime-failure assertion to ToolExecutionBlockedError.

Refs AAASM-6445

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TLpw696KjUDUg4QjH83En4
Refs AAASM-6445

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TLpw696KjUDUg4QjH83En4
Without it an explicit handler on the active interceptor ended each run twice
(duplicate audit/budget side effects). Mark skipped runs, clear on end/error,
and use a unique miss sentinel so a registry miss never equals an interceptor.

Refs AAASM-6445

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TLpw696KjUDUg4QjH83En4
Refs AAASM-6445

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TLpw696KjUDUg4QjH83En4
Refs AAASM-6445

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TLpw696KjUDUg4QjH83En4
…ure test

The plain-thread test only needs the typed governance error the tool raises.
Catching BaseException also swallowed KeyboardInterrupt and SystemExit in the
worker thread; a non-Exception raise now ends the thread and the assertion on
the captured errors still fails.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TLpw696KjUDUg4QjH83En4
@Chisanan232

Copy link
Copy Markdown
Contributor Author

OWNER-MERGE-REVIEW head=af584d510bb1d95f85947392d2c43c2241128eaa verdict=PASS blocking=0 reviewer=claude-independent-agent

Brings in the FUNDING.yml-only change (HORO-2531) so the branch is current.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TLpw696KjUDUg4QjH83En4
@Chisanan232

Copy link
Copy Markdown
Contributor Author

OWNER-MERGE-REVIEW head=eda34d45251a5e853137679e9e141e5a76e73944 verdict=PASS blocking=0 reviewer=claude-independent-agent

Chisanan232 and others added 4 commits November 10, 2026 21:59
…ngchain behavior

An unreachable runtime blocks the tool with ToolExecutionBlockedError, registering
the hook twice still checks each call once, and a missing langchain_core leaves the
hook unregistered. Addresses independent review findings on #389.

Refs: AAASM-6445

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TLpw696KjUDUg4QjH83En4
… calls

The earlier idempotence test passed even without the early-return guard; it now
spies on register_configure_hook and checks the registered var is unchanged.

Refs: AAASM-6445

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TLpw696KjUDUg4QjH83En4
@Chisanan232

Copy link
Copy Markdown
Contributor Author

OWNER-MERGE-REVIEW head=3dbcaa592309537995d46bbdd0c5229c225e7804 verdict=PASS blocking=0 reviewer=claude-independent-agent

@sonarqubecloud

Copy link
Copy Markdown

@Chisanan232

Copy link
Copy Markdown
Contributor Author

OWNER-MERGE-REVIEW head=bd80be588ab8e845cd794a62a78b06db6bce3c27 verdict=PASS blocking=0 reviewer=claude-independent-agent

@Chisanan232

Copy link
Copy Markdown
Contributor Author

Merging under the owner-only same-identity admin-merge exception (AAASM-5858 / engineering-workflow exception 2). Verified identity: Chisanan232, org admin of ai-agent-assembly, also the PR author. Sole blocker: GitHub REVIEW_REQUIRED (author and only available approver are the same identity). Conditions at head bd80be5: independent review PASS on the LangChain change (allow/deny/runtime-failure/idempotence/no-langchain tests added from review findings; later heads only add tests and merge main), all 30 checks green, both review threads resolved, mergeable, branch current with main. Not yet done: J08 on a wheel rebuilt from this head (queued q000347); the earlier J08 6/6 was on an older head. Merge commit only, pinned to this head. Ticket: AAASM-6445.

@Chisanan232
Chisanan232 merged commit 54a9619 into main Nov 10, 2026
30 checks passed
@Chisanan232
Chisanan232 deleted the v0.0.1/AAASM-6445/fix/langchain_govern_on_init branch November 10, 2026 14:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant