From 865fe0d90589043d6110e4454b6a9a39e6dc6499 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 22 Sep 2026 03:52:55 +0000 Subject: [PATCH] feat(ruby_llm): compose nested with_agent scopes and add correlation_tracer A with_agent scope opened inside another one replaced it outright, and a turn captures whichever scope is active on its first round. So when an evaluation replay wrapped the code under test in an outer scope carrying `eval.*` attributes, an `on_trace` callback and `synchronous: true`, and that code opened its own scope around the actual `chat.ask`, the trace lost the evaluation attributes, the callback never received the trace id and delivery went back to the background thread, with no spec noticing. Nested scopes now compose: attributes merge (enclosing first, nested keys win), `on_trace` and `synchronous` are inherited unless the nested call passes its own, and `synchronous:` defaults to nil ("inherit, else false") so an explicit false still overrides an inherited true. Without `pin` a nested scope still replaces name and action. `pin: true` makes a scope own the turn's identity: nested scopes may only add attributes while its name, action, on_trace and synchronous stay in force. The previous scope is restored exactly on exit, including when the block raises. `correlation_tracer(synchronous: true, pin: true)` returns the `tracer:` lambda `ActiveAgent::Evals::Correlation` (activeagent >= 1.6.3) expects, `->(name, action:, attributes:, on_trace:, &block)`, opening a pinned, synchronous scope. activeagent is not a dependency; the contract is described in the YARD comment and README. Core and adapter suites pass (72 tests, 223 assertions) and every Ruby file passes the Omakase lint configuration. Both gems are bumped to 0.3.2 (the core gem is republished unchanged so the two stay on one version). After merging, tag `v0.3.2` to publish both through the release workflow. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_011wAK6yEsed7nUqfvnDRfr5 --- CHANGELOG.md | 36 ++++ adapters/ruby_llm/README.md | 64 +++++- .../lib/activeagents/telemetry/ruby_llm.rb | 85 +++++++- .../telemetry/ruby_llm/version.rb | 2 +- adapters/ruby_llm/test/test_nested_scopes.rb | 186 ++++++++++++++++++ docs/branches/nested-agent-scopes.md | 6 + docs/issues/nested-agent-scopes.md | 18 ++ docs/milestones/nested-agent-scopes.md | 8 + docs/pull-requests/nested-agent-scopes.md | 15 ++ lib/activeagents/telemetry/version.rb | 2 +- 10 files changed, 411 insertions(+), 11 deletions(-) create mode 100644 adapters/ruby_llm/test/test_nested_scopes.rb create mode 100644 docs/branches/nested-agent-scopes.md create mode 100644 docs/issues/nested-agent-scopes.md create mode 100644 docs/milestones/nested-agent-scopes.md create mode 100644 docs/pull-requests/nested-agent-scopes.md diff --git a/CHANGELOG.md b/CHANGELOG.md index d133cdc..3d648b1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,41 @@ # Changelog +## [0.3.2] - 2026-09-22 + +### Added + +- `RubyLLM.with_agent` scopes compose. Inside an enclosing scope, a nested + call merges its `attributes` onto the enclosing ones (a nested key wins on + collision) and inherits `on_trace` and `synchronous` unless it passes its + own. `synchronous:` now defaults to `nil`, meaning "inherit, else false", + so an explicit `synchronous: false` still overrides an inherited `true`. + Without `pin`, a nested scope replaces `name` and `action` as before. +- `RubyLLM.with_agent` takes `pin: true`. A pinned scope owns the turn's + identity: while it is active a nested `with_agent` may only add + attributes, and the pinned scope's `name`, `action`, `on_trace` and + `synchronous` stay in force whatever the nested call passes. The previous + scope is still restored exactly on exit, including when the block raises. +- `RubyLLM.correlation_tracer(synchronous: true, pin: true)` returns the + `tracer:` lambda `ActiveAgent::Evals::Correlation` (activeagent >= 1.6.3) + expects, `->(name, action:, attributes:, on_trace:, &block)`, opening a + pinned, synchronous `with_agent` scope with those arguments. activeagent + is not a dependency; only the lambda's signature is shared. + +### Fixed + +- A `with_agent` scope opened inside another one replaced it outright, so + when an evaluation replay wrapped the code under test in a scope carrying + `eval.*` attributes, an `on_trace` callback and `synchronous: true`, and + that code opened its own scope around the actual `chat.ask`, the trace + lost the evaluation attributes, the callback never received the trace id + and delivery went back to the background thread. Nested scopes now + compose as described above, and an evaluation pins its scope so the inner + one cannot detach the trace from it. + +The core gem is republished at 0.3.2 with no changes, so the two gems stay on +one version. + + ## [0.3.1] - 2026-09-20 ### Fixed diff --git a/adapters/ruby_llm/README.md b/adapters/ruby_llm/README.md index 000dcac..1aadebf 100644 --- a/adapters/ruby_llm/README.md +++ b/adapters/ruby_llm/README.md @@ -124,10 +124,66 @@ exceptions. A turn keeps the scope it started under, so a turn left open by a pending tool call and closed later by `flush!` still reports with this agent, attributes and callback, and a turn that started outside any scope never adopts one. Context -is restored after the block, including when it raises, and nested scopes use -their own attributes. A callback error is logged by exception class without -dropping the trace. Attribute redaction and content-capture settings continue to -apply. +is restored after the block, including when it raises. A callback error is +logged by exception class without dropping the trace. Attribute redaction and +content-capture settings continue to apply. + +### Nested scopes + +Scopes compose, so an evaluation can wrap code that names its own agent +without losing what the evaluation attached. When a `with_agent` block opens +inside another one: + +| Setting | Nested scope inside a plain scope | Nested scope inside a `pin: true` scope | +|---------|-----------------------------------|------------------------------------------| +| `attributes` | Merged: enclosing first, nested keys win on collision | Merged the same way | +| `on_trace` | Inherited unless the nested call passes its own | The pinned scope's callback stays in force | +| `synchronous` | Inherited unless the nested call passes its own (`nil`, the default, inherits; an explicit `false` overrides an inherited `true`) | The pinned scope's setting stays in force | +| `name` / `action` | Replaced by the nested scope | The pinned scope's identity stays in force | + +A pinned scope owns the turn's identity: while it is active, a nested +`with_agent` may only add attributes, and it stays pinned however deep the +nesting goes. The previous scope is restored exactly on exit, including when +the block raises. The `agent.*` identity keys are always written last, so an +attribute with one of those names never overrides the reported agent. + +```ruby +ActiveAgents::Telemetry::RubyLLM.with_agent( + "SupportAgent", action: "replay", + attributes: { "eval.run_id" => run_id, "eval.result_id" => result_id }, + on_trace: ->(trace) { result.trace_id = trace.trace_id }, + synchronous: true, pin: true +) do + # The code under test names its own scope around the actual call. The + # trace reports as SupportAgent.replay with eval.* and sparkle.role, the + # evaluation's on_trace receives it, and it is delivered before this + # block returns. + ActiveAgents::Telemetry::RubyLLM.with_agent("SupportAgent", action: "respond", + attributes: { "sparkle.role" => "assistant" }) do + chat.ask(prompt) + end +end +``` + +### Correlating with ActiveAgent evaluations + +`ActiveAgent::Evals::Correlation` (activeagent >= 1.6.3) takes a `tracer:` +that it calls as `tracer.call(name, action:, attributes:, on_trace:) { ... }` +around each replay, and expects `on_trace` to receive something answering +`#trace_id`. `correlation_tracer` returns exactly that lambda, opening a +pinned, synchronous `with_agent` scope with the arguments it is given and +returning the block's value: + +```ruby +ActiveAgent::Evals::Correlation.new( + agent_name: "SupportAgent", + tracer: ActiveAgents::Telemetry::RubyLLM.correlation_tracer +) +``` + +`correlation_tracer(synchronous: false)` or `correlation_tracer(pin: false)` +relaxes either default. This gem does not depend on activeagent; the only +contract shared with it is the lambda's signature. Report publication and trace ingestion are separate operations. Persist each run and result ID in the evaluation report and attach the same IDs to its response and diff --git a/adapters/ruby_llm/lib/activeagents/telemetry/ruby_llm.rb b/adapters/ruby_llm/lib/activeagents/telemetry/ruby_llm.rb index f3b16a6..578c8dd 100644 --- a/adapters/ruby_llm/lib/activeagents/telemetry/ruby_llm.rb +++ b/adapters/ruby_llm/lib/activeagents/telemetry/ruby_llm.rb @@ -105,17 +105,71 @@ def reporter # `on_trace` runs only for a trace the reporter accepted, which means # it passed the enabled, configured and sampling checks: a delivery # that then fails is logged by the reporter, not announced here. - def with_agent(name, action: "chat", attributes: {}, on_trace: nil, synchronous: false) + # + # Scopes compose. Inside an enclosing scope, a nested call merges its + # `attributes` onto the enclosing ones (a nested key wins on + # collision) and inherits `on_trace` and `synchronous` unless it + # passes its own; `synchronous: nil`, the default, means "inherit, + # else false", so an explicit `synchronous: false` still overrides an + # inherited `true`. Without `pin`, the nested scope replaces `name` + # and `action`, so a call site that names its own agent keeps doing + # so under an evaluation's scope. A scope opened with `pin: true` owns + # the turn's identity: while it is active a nested `with_agent` may + # only add attributes, and the pinned scope's `name`, `action`, + # `on_trace` and `synchronous` stay in force whatever the nested call + # passes. An evaluation replay pins its scope so the code under test, + # which opens its own scope around the actual `chat.ask`, cannot + # detach the trace from the evaluation's attributes, callback and + # synchronous delivery. The previous scope is restored on exit, + # including when the block raises. + # + # @param synchronous [Boolean, nil] nil inherits the enclosing scope's + # setting, or false at the top level. + # @param pin [Boolean] whether this scope keeps its identity, callback + # and delivery mode against nested scopes. + def with_agent(name, action: "chat", attributes: {}, on_trace: nil, synchronous: nil, pin: false) previous = Thread.current[AGENT_KEY] - Thread.current[AGENT_KEY] = { - name: name, action: action, attributes: attributes.to_h.transform_keys(&:to_s), - on_trace: on_trace, synchronous: synchronous - } + Thread.current[AGENT_KEY] = compose_scope( + previous, name: name, action: action, attributes: attributes, + on_trace: on_trace, synchronous: synchronous, pin: pin + ) yield ensure Thread.current[AGENT_KEY] = previous end + # A `tracer:` for `ActiveAgent::Evals::Correlation` (activeagent >= + # 1.6.3), which wraps each evaluation replay in + # `tracer.call(name, action:, attributes:, on_trace:) { ... }` and + # expects `on_trace` to receive something answering `#trace_id`. The + # returned lambda opens a pinned, synchronous `with_agent` scope with + # exactly those arguments, so the replayed code's own `with_agent` + # around `chat.ask` can add attributes but cannot replace the + # evaluation's identity, drop its `on_trace` or turn delivery + # asynchronous, and the trace id the callback receives is the one + # the reporter delivered before the replay returns. The block's value + # is returned. There is no dependency on activeagent: the contract is + # only the lambda's signature. + # + # ActiveAgent::Evals::Correlation.new( + # agent_name: "SupportAgent", + # tracer: ActiveAgents::Telemetry::RubyLLM.correlation_tracer + # ) + # + # @param synchronous [Boolean] whether the scope delivers in the + # calling thread; true keeps the trace id available when the + # replay returns and before a short-lived process exits. + # @param pin [Boolean] whether nested scopes may only add attributes. + # @return [Proc] `->(name, action:, attributes:, on_trace:, &block)` + def correlation_tracer(synchronous: true, pin: true) + lambda do |name, action: "chat", attributes: {}, on_trace: nil, &block| + with_agent( + name, action: action, attributes: attributes, on_trace: on_trace, + synchronous: synchronous, pin: pin, &block + ) + end + end + def state Thread.current[STATE_KEY] ||= State.new(0, nil, [], 0, Span::ZERO_TOKENS.dup, nil, nil) end @@ -182,6 +236,27 @@ def build_tool_span(payload, started_at, finished_at) private + # The scope a nested `with_agent` installs, given the enclosing one. + # Attributes always merge with the nested keys winning; the rest + # depends on whether the enclosing scope is pinned. + def compose_scope(enclosing, name:, action:, attributes:, on_trace:, synchronous:, pin:) + own = attributes.to_h.transform_keys(&:to_s) + if enclosing.nil? + return { name: name, action: action, attributes: own, on_trace: on_trace, + synchronous: synchronous ? true : false, pin: pin ? true : false } + end + + merged = (enclosing[:attributes] || {}).merge(own) + return enclosing.merge(attributes: merged) if enclosing[:pin] + + { + name: name, action: action, attributes: merged, + on_trace: on_trace.nil? ? enclosing[:on_trace] : on_trace, + synchronous: synchronous.nil? ? enclosing[:synchronous] : synchronous, + pin: pin ? true : false + } + end + def report_turn(payload, turn) agent = turn.agent || resolve_agent(payload) || DEFAULT_AGENT started_at = turn.started_at || Time.now diff --git a/adapters/ruby_llm/lib/activeagents/telemetry/ruby_llm/version.rb b/adapters/ruby_llm/lib/activeagents/telemetry/ruby_llm/version.rb index 73b65cf..7357c78 100644 --- a/adapters/ruby_llm/lib/activeagents/telemetry/ruby_llm/version.rb +++ b/adapters/ruby_llm/lib/activeagents/telemetry/ruby_llm/version.rb @@ -3,7 +3,7 @@ module ActiveAgents module Telemetry module RubyLLM - VERSION = "0.3.1" + VERSION = "0.3.2" end end end diff --git a/adapters/ruby_llm/test/test_nested_scopes.rb b/adapters/ruby_llm/test/test_nested_scopes.rb new file mode 100644 index 0000000..6fed1a1 --- /dev/null +++ b/adapters/ruby_llm/test/test_nested_scopes.rb @@ -0,0 +1,186 @@ +# frozen_string_literal: true + +require "test_helper" + +class TestNestedScopes < Minitest::Test + include RubyLLMTelemetryTestHelpers + + def test_nested_scope_merges_attributes_and_inherits_on_trace_and_synchronous + ids = [] + delivery_threads = deliver_recording_threads + Adapter.with_agent("EvalReplay", action: "replay", + attributes: { "eval.run_id" => "run-1", "eval.result_id" => "result-1" }, + on_trace: ->(trace) { ids << trace.trace_id }, synchronous: true) do + Adapter.with_agent("Support", action: "respond", attributes: { "sparkle.role" => "assistant" }) do + instrument("chat.ruby_llm", chat_payload) { nil } + end + end + + trace = traces.fetch(0) + root = spans_of(trace, "root").fetch(0) + assert_equal "run-1", root["attributes"]["eval.run_id"] + assert_equal "result-1", root["attributes"]["eval.result_id"] + assert_equal "assistant", root["attributes"]["sparkle.role"] + assert_equal [ trace["trace_id"] ], ids, "the enclosing on_trace is inherited" + assert_equal [ Thread.current ], delivery_threads, "synchronous delivery is inherited" + end + + def test_nested_scope_replaces_name_and_action_when_the_enclosing_scope_is_not_pinned + Adapter.with_agent("EvalReplay", action: "replay", attributes: { "eval.run_id" => "run-1", "shared" => "outer" }) do + Adapter.with_agent("Support", action: "respond", attributes: { "shared" => "inner" }) do + instrument("chat.ruby_llm", chat_payload) { nil } + end + end + + root = spans_of(traces.fetch(0), "root").fetch(0) + assert_equal "Support.respond", root["name"] + assert_equal "Support", root["attributes"]["agent.class"] + assert_equal "respond", root["attributes"]["agent.action"] + assert_equal "run-1", root["attributes"]["eval.run_id"] + assert_equal "inner", root["attributes"]["shared"], "a nested key wins on collision" + end + + def test_nested_scope_inside_a_pinned_scope_keeps_its_identity_callback_and_delivery + ids = [] + delivery_threads = deliver_recording_threads + nested_ids = [] + Adapter.with_agent("EvalReplay", action: "replay", attributes: { "eval.run_id" => "run-1" }, + on_trace: ->(trace) { ids << trace.trace_id }, synchronous: true, pin: true) do + Adapter.with_agent("Support", action: "respond", attributes: { "sparkle.role" => "assistant" }, + on_trace: ->(trace) { nested_ids << trace.trace_id }, synchronous: false) do + instrument("chat.ruby_llm", chat_payload) { nil } + end + end + + trace = traces.fetch(0) + root = spans_of(trace, "root").fetch(0) + assert_equal "EvalReplay.replay", root["name"] + assert_equal "EvalReplay", root["attributes"]["agent.class"] + assert_equal "replay", root["attributes"]["agent.action"] + assert_equal "run-1", root["attributes"]["eval.run_id"] + assert_equal "assistant", root["attributes"]["sparkle.role"], "the nested scope still adds attributes" + assert_equal [ trace["trace_id"] ], ids, "the pinned on_trace stays in force" + assert_empty nested_ids, "the nested on_trace does not replace the pinned one" + assert_equal [ Thread.current ], delivery_threads, "the pinned synchronous delivery stays in force" + end + + def test_pinned_scope_stays_pinned_two_levels_down + Adapter.with_agent("EvalReplay", action: "replay", pin: true) do + Adapter.with_agent("Support", action: "respond", attributes: { "depth" => "one" }) do + Adapter.with_agent("Tool", action: "run", attributes: { "depth" => "two" }) do + instrument("chat.ruby_llm", chat_payload) { nil } + end + end + end + + root = spans_of(traces.fetch(0), "root").fetch(0) + assert_equal "EvalReplay.replay", root["name"] + assert_equal "two", root["attributes"]["depth"] + end + + def test_explicit_synchronous_false_overrides_an_inherited_true + subscribe(async: true) + delivery_threads = Queue.new + Adapter.reporter.define_singleton_method(:deliver) { |_body| delivery_threads << Thread.current } + + Adapter.with_agent("EvalReplay", synchronous: true) do + Adapter.with_agent("Support", synchronous: false) do + instrument("chat.ruby_llm", chat_payload) { nil } + end + end + + delivery_thread = delivery_threads.pop(timeout: 2) + refute_nil delivery_thread, "the trace was still delivered" + refute_equal Thread.current, delivery_thread, "delivery went to the reporter's background thread" + end + + def test_scope_is_restored_after_the_block_and_when_it_raises + outer_ids = [] + Adapter.with_agent("EvalReplay", action: "replay", attributes: { "eval.run_id" => "run-1" }, + on_trace: ->(trace) { outer_ids << trace.trace_id }, synchronous: true, pin: true) do + outer_scope = Thread.current[Adapter::AGENT_KEY] + Adapter.with_agent("Support", action: "respond", attributes: { "sparkle.role" => "assistant" }) do + refute_equal outer_scope, Thread.current[Adapter::AGENT_KEY] + end + assert_same outer_scope, Thread.current[Adapter::AGENT_KEY] + + assert_raises(RuntimeError) do + Adapter.with_agent("Support", action: "respond", attributes: { "sparkle.role" => "assistant" }) do + raise "synthetic error" + end + end + assert_same outer_scope, Thread.current[Adapter::AGENT_KEY] + instrument("chat.ruby_llm", chat_payload) { nil } + end + assert_nil Thread.current[Adapter::AGENT_KEY] + instrument("chat.ruby_llm", chat_payload) { nil } + + roots = traces.map { |trace| spans_of(trace, "root").first } + assert_equal [ "EvalReplay.replay", "RubyLLM::Chat.chat" ], roots.map { |root| root["name"] } + assert_equal [ "run-1", nil ], roots.map { |root| root["attributes"]["eval.run_id"] } + assert_equal [ nil, nil ], roots.map { |root| root["attributes"]["sparkle.role"] }, + "the raised nested scope leaves nothing behind" + assert_equal [ traces.fetch(0)["trace_id"] ], outer_ids + end + + def test_top_level_scope_defaults_stay_unpinned_and_asynchronous + Adapter.with_agent("Support") do + scope = Thread.current[Adapter::AGENT_KEY] + assert_equal false, scope[:synchronous] + assert_equal false, scope[:pin] + end + end + + def test_correlation_tracer_opens_a_pinned_synchronous_scope_and_forwards_on_trace + ids = [] + delivery_threads = deliver_recording_threads + received = [] + tracer = Adapter.correlation_tracer + result = tracer.call("SupportAgent", action: "replay", attributes: { "eval.run_id" => "run-1" }, + on_trace: ->(trace) { received << trace; ids << trace.trace_id }) do + scope = Thread.current[Adapter::AGENT_KEY] + assert_equal true, scope[:pin] + assert_equal true, scope[:synchronous] + Adapter.with_agent("SupportAgent", action: "respond", attributes: { "sparkle.role" => "assistant" }) do + instrument("chat.ruby_llm", chat_payload) { :answer } + end + end + + assert_equal :answer, result, "the tracer returns the block's value" + trace = traces.fetch(0) + root = spans_of(trace, "root").fetch(0) + assert_equal "SupportAgent.replay", root["name"] + assert_equal "run-1", root["attributes"]["eval.run_id"] + assert_equal "assistant", root["attributes"]["sparkle.role"] + assert_equal [ trace["trace_id"] ], ids + assert_respond_to received.fetch(0), :trace_id + assert_equal [ Thread.current ], delivery_threads + end + + def test_correlation_tracer_forwards_its_own_synchronous_and_pin_settings + tracer = Adapter.correlation_tracer(synchronous: false, pin: false) + tracer.call("SupportAgent", action: "replay", attributes: {}, on_trace: nil) do + scope = Thread.current[Adapter::AGENT_KEY] + assert_equal false, scope[:synchronous] + assert_equal false, scope[:pin] + Adapter.with_agent("Inner", action: "respond") { instrument("chat.ruby_llm", chat_payload) { nil } } + end + + assert_equal "Inner.respond", spans_of(traces.fetch(0), "root").fetch(0)["name"] + end + + private + + # Subscribes with async delivery and records the thread each delivery ran + # on, so a spec can tell synchronous delivery from the background thread. + def deliver_recording_threads + subscribe(async: true) + delivery_threads = [] + captured = posted + Adapter.reporter.define_singleton_method(:deliver) do |body| + delivery_threads << Thread.current + captured << body + end + delivery_threads + end +end diff --git a/docs/branches/nested-agent-scopes.md b/docs/branches/nested-agent-scopes.md new file mode 100644 index 0000000..beece94 --- /dev/null +++ b/docs/branches/nested-agent-scopes.md @@ -0,0 +1,6 @@ +# Branch + +- Branch: `claude/nested-agent-scopes`. +- Base: `main` at `314a203b1bff2e1b44ae80d6eb1792202e10fc95`. +- Scope: nested `with_agent` scopes compose in the RubyLLM adapter, `pin:` and `correlation_tracer`; both gems bumped to 0.3.2. +- Release: tag `v0.3.2` after the merge; the release workflow publishes both gems. diff --git a/docs/issues/nested-agent-scopes.md b/docs/issues/nested-agent-scopes.md new file mode 100644 index 0000000..1a5127c --- /dev/null +++ b/docs/issues/nested-agent-scopes.md @@ -0,0 +1,18 @@ +# A nested with_agent scope replaces the evaluation's scope + +`RubyLLM.with_agent` set the thread-local scope to a fresh hash and restored +the previous one on exit, and a turn captures whichever scope is active on its +first round. When an evaluation replay wrapped the code under test in an outer +scope (the evaluation's name, `eval.*` attributes, an `on_trace` recording the +trace id, `synchronous: true`) and that code opened its own scope around the +actual `chat.ask` (its own name and action, `sparkle.role`-style attributes), +the inner scope silently replaced the outer one: the `eval.*` attributes were +gone from the trace, the callback never ran, delivery went back to the +background thread, and no spec noticed. + +Nested scopes now compose: attributes merge with the nested keys winning, +`on_trace` and `synchronous` are inherited unless passed, and a scope opened +with `pin: true` keeps its name, action, callback and delivery mode against +nested scopes, which may only add attributes. `correlation_tracer` returns the +`tracer:` lambda `ActiveAgent::Evals::Correlation` expects, opening a pinned, +synchronous scope. diff --git a/docs/milestones/nested-agent-scopes.md b/docs/milestones/nested-agent-scopes.md new file mode 100644 index 0000000..0ee8fed --- /dev/null +++ b/docs/milestones/nested-agent-scopes.md @@ -0,0 +1,8 @@ +# Milestone: nested agent scopes + +- [x] Merge attributes and inherit `on_trace` / `synchronous` across nested `with_agent` scopes; `synchronous:` defaults to `nil` so an explicit `false` still overrides. +- [x] Add `pin: true` so an evaluation's scope keeps its identity, callback and delivery mode against the code under test's own scope. +- [x] Add `correlation_tracer(synchronous: true, pin: true)` for `ActiveAgent::Evals::Correlation`'s `tracer:`, without depending on activeagent. +- [x] Test merging, inheritance, replacement, pinning at depth, the explicit `false` override, restoration after a raise, and the tracer. +- [x] Document the nesting rules and the tracer; bump both gems to 0.3.2 with a changelog entry, so the merge is releasable. +- [ ] Merge the PR, then tag `v0.3.2` to publish both gems through the release workflow. diff --git a/docs/pull-requests/nested-agent-scopes.md b/docs/pull-requests/nested-agent-scopes.md new file mode 100644 index 0000000..eea9a7a --- /dev/null +++ b/docs/pull-requests/nested-agent-scopes.md @@ -0,0 +1,15 @@ +# Draft PR: nested agent scopes + +Branch `claude/nested-agent-scopes` makes nested `RubyLLM.with_agent` scopes +compose instead of replacing each other: attributes merge (nested keys win), +`on_trace` and `synchronous` are inherited unless passed, and `pin: true` +keeps a scope's name, action, callback and delivery mode in force while nested +scopes only add attributes. `correlation_tracer` returns the `tracer:` lambda +`ActiveAgent::Evals::Correlation` (activeagent >= 1.6.3) expects, opening a +pinned, synchronous scope, with no dependency on activeagent. + +Core and adapter suites pass: 72 tests, 223 assertions, zero failures. Every +Ruby file passes the Omakase lint configuration. Versions and the changelog +are bumped to 0.3.2 on the branch (the core gem is republished unchanged so +the two stay on one version); tagging `v0.3.2` after the merge publishes both +gems. diff --git a/lib/activeagents/telemetry/version.rb b/lib/activeagents/telemetry/version.rb index d450cf6..13a9b27 100644 --- a/lib/activeagents/telemetry/version.rb +++ b/lib/activeagents/telemetry/version.rb @@ -2,6 +2,6 @@ module ActiveAgents module Telemetry - VERSION = "0.3.1" + VERSION = "0.3.2" end end