feat(ruby_llm): compose nested with_agent scopes and add correlation_tracer

CHANGELOG.md

@@ -1,5 +1,41 @@
 # Changelog
 
+## [0.3.2] - 2026-09-22
+
+### Added
+
+- `RubyLLM.with_agent` scopes compose. Inside an enclosing scope, a nested
+  call merges its `attributes` onto the enclosing ones (a nested key wins on
+  collision) and inherits `on_trace` and `synchronous` unless it passes its
+  own. `synchronous:` now defaults to `nil`, meaning "inherit, else false",
+  so an explicit `synchronous: false` still overrides an inherited `true`.
+  Without `pin`, a nested scope replaces `name` and `action` as before.
+- `RubyLLM.with_agent` takes `pin: true`. A pinned scope owns the turn's
+  identity: while it is active a nested `with_agent` may only add
+  attributes, and the pinned scope's `name`, `action`, `on_trace` and
+  `synchronous` stay in force whatever the nested call passes. The previous
+  scope is still restored exactly on exit, including when the block raises.
+- `RubyLLM.correlation_tracer(synchronous: true, pin: true)` returns the
+  `tracer:` lambda `ActiveAgent::Evals::Correlation` (activeagent >= 1.6.3)
+  expects, `->(name, action:, attributes:, on_trace:, &block)`, opening a
+  pinned, synchronous `with_agent` scope with those arguments. activeagent
+  is not a dependency; only the lambda's signature is shared.
+
+### Fixed
+
+- A `with_agent` scope opened inside another one replaced it outright, so
+  when an evaluation replay wrapped the code under test in a scope carrying
+  `eval.*` attributes, an `on_trace` callback and `synchronous: true`, and
+  that code opened its own scope around the actual `chat.ask`, the trace
+  lost the evaluation attributes, the callback never received the trace id
+  and delivery went back to the background thread. Nested scopes now
+  compose as described above, and an evaluation pins its scope so the inner
+  one cannot detach the trace from it.
+
+The core gem is republished at 0.3.2 with no changes, so the two gems stay on
+one version.
+
+
 ## [0.3.1] - 2026-09-20
 
 ### Fixed

adapters/ruby_llm/README.md

@@ -124,10 +124,66 @@ exceptions. A turn keeps the scope it started under, so a turn left open by a
 pending tool call and closed later by `flush!` still reports with this agent,
 attributes and callback, and a turn that started outside any scope never adopts
 one. Context
-is restored after the block, including when it raises, and nested scopes use
-their own attributes. A callback error is logged by exception class without
-dropping the trace. Attribute redaction and content-capture settings continue to
-apply.
+is restored after the block, including when it raises. A callback error is
+logged by exception class without dropping the trace. Attribute redaction and
+content-capture settings continue to apply.
+
+### Nested scopes
+
+Scopes compose, so an evaluation can wrap code that names its own agent
+without losing what the evaluation attached. When a `with_agent` block opens
+inside another one:
+
+| Setting | Nested scope inside a plain scope | Nested scope inside a `pin: true` scope |
+|---------|-----------------------------------|------------------------------------------|
+| `attributes` | Merged: enclosing first, nested keys win on collision | Merged the same way |
+| `on_trace` | Inherited unless the nested call passes its own | The pinned scope's callback stays in force |
+| `synchronous` | Inherited unless the nested call passes its own (`nil`, the default, inherits; an explicit `false` overrides an inherited `true`) | The pinned scope's setting stays in force |
+| `name` / `action` | Replaced by the nested scope | The pinned scope's identity stays in force |
+
+A pinned scope owns the turn's identity: while it is active, a nested
+`with_agent` may only add attributes, and it stays pinned however deep the
+nesting goes. The previous scope is restored exactly on exit, including when
+the block raises. The `agent.*` identity keys are always written last, so an
+attribute with one of those names never overrides the reported agent.
+
+```ruby
+ActiveAgents::Telemetry::RubyLLM.with_agent(
+  "SupportAgent", action: "replay",
+  attributes: { "eval.run_id" => run_id, "eval.result_id" => result_id },
+  on_trace: ->(trace) { result.trace_id = trace.trace_id },
+  synchronous: true, pin: true
+) do
+  # The code under test names its own scope around the actual call. The
+  # trace reports as SupportAgent.replay with eval.* and sparkle.role, the
+  # evaluation's on_trace receives it, and it is delivered before this
+  # block returns.
+  ActiveAgents::Telemetry::RubyLLM.with_agent("SupportAgent", action: "respond",
+    attributes: { "sparkle.role" => "assistant" }) do
+    chat.ask(prompt)
+  end
+end
+```
+
+### Correlating with ActiveAgent evaluations
+
+`ActiveAgent::Evals::Correlation` (activeagent >= 1.6.3) takes a `tracer:`
+that it calls as `tracer.call(name, action:, attributes:, on_trace:) { ... }`
+around each replay, and expects `on_trace` to receive something answering
+`#trace_id`. `correlation_tracer` returns exactly that lambda, opening a
+pinned, synchronous `with_agent` scope with the arguments it is given and
+returning the block's value:
+
+```ruby
+ActiveAgent::Evals::Correlation.new(
+  agent_name: "SupportAgent",
+  tracer: ActiveAgents::Telemetry::RubyLLM.correlation_tracer
+)
+```
+
+`correlation_tracer(synchronous: false)` or `correlation_tracer(pin: false)`
+relaxes either default. This gem does not depend on activeagent; the only
+contract shared with it is the lambda's signature.
 
 Report publication and trace ingestion are separate operations. Persist each run
 and result ID in the evaluation report and attach the same IDs to its response and

adapters/ruby_llm/lib/activeagents/telemetry/ruby_llm.rb

@@ -105,17 +105,71 @@ def reporter
         # `on_trace` runs only for a trace the reporter accepted, which means
         # it passed the enabled, configured and sampling checks: a delivery
         # that then fails is logged by the reporter, not announced here.
-        def with_agent(name, action: "chat", attributes: {}, on_trace: nil, synchronous: false)
+        #
+        # Scopes compose. Inside an enclosing scope, a nested call merges its
+        # `attributes` onto the enclosing ones (a nested key wins on
+        # collision) and inherits `on_trace` and `synchronous` unless it
+        # passes its own; `synchronous: nil`, the default, means "inherit,
+        # else false", so an explicit `synchronous: false` still overrides an
+        # inherited `true`. Without `pin`, the nested scope replaces `name`
+        # and `action`, so a call site that names its own agent keeps doing
+        # so under an evaluation's scope. A scope opened with `pin: true` owns
+        # the turn's identity: while it is active a nested `with_agent` may
+        # only add attributes, and the pinned scope's `name`, `action`,
+        # `on_trace` and `synchronous` stay in force whatever the nested call
+        # passes. An evaluation replay pins its scope so the code under test,
+        # which opens its own scope around the actual `chat.ask`, cannot
+        # detach the trace from the evaluation's attributes, callback and
+        # synchronous delivery. The previous scope is restored on exit,
+        # including when the block raises.
+        #
+        # @param synchronous [Boolean, nil] nil inherits the enclosing scope's
+        #   setting, or false at the top level.
+        # @param pin [Boolean] whether this scope keeps its identity, callback
+        #   and delivery mode against nested scopes.
+        def with_agent(name, action: "chat", attributes: {}, on_trace: nil, synchronous: nil, pin: false)
           previous = Thread.current[AGENT_KEY]
-          Thread.current[AGENT_KEY] = {
-            name: name, action: action, attributes: attributes.to_h.transform_keys(&:to_s),
-            on_trace: on_trace, synchronous: synchronous
-          }
+          Thread.current[AGENT_KEY] = compose_scope(
+            previous, name: name, action: action, attributes: attributes,
+            on_trace: on_trace, synchronous: synchronous, pin: pin
+          )
           yield
         ensure
           Thread.current[AGENT_KEY] = previous
         end
 
+        # A `tracer:` for `ActiveAgent::Evals::Correlation` (activeagent >=
+        # 1.6.3), which wraps each evaluation replay in
+        # `tracer.call(name, action:, attributes:, on_trace:) { ... }` and
+        # expects `on_trace` to receive something answering `#trace_id`. The
+        # returned lambda opens a pinned, synchronous `with_agent` scope with
+        # exactly those arguments, so the replayed code's own `with_agent`
+        # around `chat.ask` can add attributes but cannot replace the
+        # evaluation's identity, drop its `on_trace` or turn delivery
+        # asynchronous, and the trace id the callback receives is the one
+        # the reporter delivered before the replay returns. The block's value
+        # is returned. There is no dependency on activeagent: the contract is
+        # only the lambda's signature.
+        #
+        #   ActiveAgent::Evals::Correlation.new(
+        #     agent_name: "SupportAgent",
+        #     tracer: ActiveAgents::Telemetry::RubyLLM.correlation_tracer
+        #   )
+        #
+        # @param synchronous [Boolean] whether the scope delivers in the
+        #   calling thread; true keeps the trace id available when the
+        #   replay returns and before a short-lived process exits.
+        # @param pin [Boolean] whether nested scopes may only add attributes.
+        # @return [Proc] `->(name, action:, attributes:, on_trace:, &block)`
+        def correlation_tracer(synchronous: true, pin: true)
+          lambda do |name, action: "chat", attributes: {}, on_trace: nil, &block|
+            with_agent(
+              name, action: action, attributes: attributes, on_trace: on_trace,
+              synchronous: synchronous, pin: pin, &block
+            )
+          end
+        end
+
         def state
           Thread.current[STATE_KEY] ||= State.new(0, nil, [], 0, Span::ZERO_TOKENS.dup, nil, nil)
         end
@@ -182,6 +236,27 @@ def build_tool_span(payload, started_at, finished_at)
 
         private
 
+        # The scope a nested `with_agent` installs, given the enclosing one.
+        # Attributes always merge with the nested keys winning; the rest
+        # depends on whether the enclosing scope is pinned.
+        def compose_scope(enclosing, name:, action:, attributes:, on_trace:, synchronous:, pin:)
+          own = attributes.to_h.transform_keys(&:to_s)
+          if enclosing.nil?
+            return { name: name, action: action, attributes: own, on_trace: on_trace,
+                     synchronous: synchronous ? true : false, pin: pin ? true : false }
+          end
+
+          merged = (enclosing[:attributes] || {}).merge(own)
+          return enclosing.merge(attributes: merged) if enclosing[:pin]
+
+          {
+            name: name, action: action, attributes: merged,
+            on_trace: on_trace.nil? ? enclosing[:on_trace] : on_trace,
+            synchronous: synchronous.nil? ? enclosing[:synchronous] : synchronous,
+            pin: pin ? true : false
+          }
+        end
+
         def report_turn(payload, turn)
           agent = turn.agent || resolve_agent(payload) || DEFAULT_AGENT
           started_at = turn.started_at || Time.now

adapters/ruby_llm/lib/activeagents/telemetry/ruby_llm/version.rb

@@ -3,7 +3,7 @@
 module ActiveAgents
   module Telemetry
     module RubyLLM
-      VERSION = "0.3.1"
+      VERSION = "0.3.2"
     end
   end
 end