Skip to content

feat(replicate): traceAI-replicate native client tracing (TH-8320) - #219

Merged
nik13 merged 39 commits into
devfrom
feat/th-8320-replicate
Oct 26, 2026
Merged

nik13 merged 39 commits into
devfrom
feat/th-8320-replicate

Conversation

@nik13

@nik13 nik13 commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Stream spans carry no status, predict_time or token counts by default. With stream_reload_seconds=0 (the default), a replicate.stream span ends on the caller's thread at done (or the end of iteration) with the prediction id and the stream's output, and without replicate.prediction.status, replicate.metrics.predict_time or gen_ai.usage.*. No request is made after the stream and no package thread is ever started, so force_flush(), shutdown(), uninstrument(), interpreter exit and os.fork() wait for no read and no package thread (a fork ends no span). force_flush(), shutdown(), uninstrument() and exit still end, inside the call, the spans that are due (held create spans and the spans of streams dropped since the last traced call); with an inline exporter those are exported there, even by force_flush(timeout_millis=0).

A call still running at shutdown or exit is exported, as cancelled. A traced call whose vendor code is still running (run, async_run, a create, wait() or cancel()), or a stream or run iterator still open, when the provider shuts down (including the SIGTERM/SIGINT handler register() installs) or the interpreter exits, ends then as ERROR cancelled with replicate.cancelled=true and what it knows, so it reaches the processors before they stop.

The caller stopping a call is recorded the same way wherever it lands. Ctrl-C or SystemExit during run(), a create, wait() or cancel(), or inside a stream's next(), a cancelled asyncio task or trio cancel scope, throw(GeneratorExit), and closing or dropping a stream before done: ERROR cancelled, replicate.cancelled=true, no exception event. A stream or run iterator that stops early keeps the output it had received (subject to TraceConfig), so a text stream stays LLM.

Opt-in read. stream_reload_seconds above 0 (up to 10) reads GET /v1/predictions/{id} once (twice for a non-final answer) for status, predict_time and tokens. It is a real request to the client's base_url, so a stub server in tests (pytest-httpserver, WireMock, Prism) receives it; keep 0 in those tests. It runs only through the package's own pool for a plain httpx transport picked by httpx's own _transport_for_url, and is skipped with a recorded replicate.stream.skip_reason otherwise: custom_transport (custom/subclassed transports, instance changes including HTTPXClientInstrumentor.instrument_client(), a subclass's _transport_for_url, custom network backends, proxies), patched_http (respx, pytest-httpx, VCR.py, mock.patch, Sentry's HttpxIntegration), patched_socket (mocket, httpretty, gevent, eventlet, truststore.inject_into_ssl()), at_capacity (64 waiting spans or live reads), and the lifecycle reasons. By default a flush, shutdown or exit does not wait for a read (stream_flush_wait_seconds opts in for flush, stream_shutdown_wait_seconds for shutdown and also interpreter exit). uninstrument() never waits for a read; like flush, shutdown and exit, it waits up to 5 s for the one span the finisher is exporting. All four cancel reads in flight. The opt-in in its current form has not been run against the live API.

Round 8 (commits c5b9505, e29fcc6, a8cc5e3)

a8cc5e3 (README only): one rule for a stream still open at uninstrument(). It ends when closed or completed; once dropped, at the next traced call (after instrument() again) or at exit; force_flush()/shutdown() no longer end it. The README had contradicted itself, each passage wrong in one case (probe on e29fcc6).

Code, with tests in tests/test_review_round8.py that fail on the previous head 55c5672 for the reason each targets:

  • Calls in flight at shutdown or exit (major). Under register()'s SIGINT/SIGTERM handler the span of an in-flight run() / async_run() was never exported (and neither was a create in flight), while a stream at the same moment was exported as cancelled and a running wait() as OK. Each traced call whose vendor code runs is now tracked, as open iterators already were, until its span ends or a registry holds it, and end_open_iterators() (provider shutdown and exit) ends it as cancelled with the prediction it polled. A wait()/cancel() that claimed a held create span is also ended as cancelled there (it was OK with the last polled status). A create that returns after a shutdown ended its span returns the client's own prediction and holds nothing. The copy of the tracked set is one C-level list() of its weak references.
  • Ctrl-C alike for every call (minor). KeyboardInterrupt / SystemExit during run(), async_run(), a create, wait() or cancel() is now the caller stopping the call (ERROR cancelled, replicate.cancelled, no exception event), as it already was inside a stream's next(). Before, run/create/wait recorded ERROR KeyboardInterrupt with an exception event.
  • trio (minor). trio.Cancelled (what anyio raises under trio) is a caller stop in the async iterator and the async wrappers, found through sys.modules (trio is never imported by the package). Other libraries' cancellations are recorded as errors; documented.
  • Stop edges (minor). throw(GeneratorExit) / athrow(GeneratorExit) is a caller stop. A close() that the generator refuses because another thread is inside its next() (ValueError: generator already executing; RuntimeError: ... already running for an async generator) no longer ends the span: the stream goes on in that thread and ends its span there. The docs now say throw() / athrow() end it with that error except for GeneratorExit, KeyboardInterrupt, SystemExit and asyncio/trio cancellations.

CI and tests:

  • The red check at 55c5672 (major). Run 39587088349 (the one started by the round-7 description edit) failed one assertion: max(tails) < 0.5 in test_200_concurrent_streams_on_threads_end_every_span_and_leave_no_read_running measured 0.538 s on the GitHub runner; the run before it at the same SHA (39584469188) passed. The bound now follows from what the test proves: every read outlives its budget, so a loop that waited for its read would exit at least the budget after its last event. The budget there is now 2 s (the read still outlives budget + slack), every tail must be below it and the median below a quarter of it. CI status at the new head is under Verification.
  • CI skipped the httpx-instrumentation tests (minor). opentelemetry-instrumentation-httpx is now a dev dependency (CI showed 5 skipped, among them the m5 instrument_client test), and so is trio for the new test.

Docs and claims (README, CHANGELOG, docstrings; the docs page in future-agi/docs#1015):

  • M1. The README has no performance or fork-probe figures; this description and the docs page hold the measurements from git archive e29fcc6, naming the quoted blocks. The README still describes the ninth live stream's measured timings; those are that session's observations, not benchmark/fork figures.
  • M2 leftovers (major). The docs page's bold heading now reads "Flush, shutdown, exit and uninstrument() do not wait for a read by default", and its troubleshooting entry "uninstrument() never waits for a read". The README, CHANGELOG, docs page and this description were searched again for "never waits", "do not wait", "does not wait" and "waits for nothing": every remaining one is qualified (for a read; with the read off; or the memo after a timed-out wait: "the next one does not wait until the finisher has ended another span", "later forks do not wait for that thread again").
  • Dead link (minor). The README pointed to the docs URL, which 404s until docs(integrations): Replicate tracing (TH-8320) docs#1015 is published. It now points to this description's Verification section and to the docs PR.
  • fork() wording (minor). "They still end, inside the call, the spans that are due" now names force_flush(), shutdown(), uninstrument() and exit only; a fork ends no span.
  • Privacy-scan wording (minor). Views, materialized views and dictionaries were listed, not read; nothing is claimed about what they read.
  • Figures (minor). Load is given as the 1-minute and 5-minute averages, and the [D1] line on the docs page now says that 2,180 of its 2,500 completions were skips that ended the span inline.
  • h3 (minor). Worded as "most likely" with the exact counts (below).
  • Live. One more session on the round-8 code (below): the code on the default stream path changed after session 8.

Verification

  • Tests. 390 passed at e29fcc6 (369 before this round, plus 21 in test_review_round8.py). From git archives, test_review_round8.py and the updated in-flight tests run against the package code of 55c5672: 25 fail and 15 pass. The 20 round-8 tests that fail do so for the reason each targets: ERROR KeyboardInterrupt / GeneratorExit / Cancelled: cancelled due to deadline instead of cancelled (Ctrl-C in run, async_run, create and wait; trio; throw / athrow); no span exported for run, async_run and create in flight at shutdown or SIGTERM/SIGINT/exit under register(); status OK for a wait at SIGINT; and the refused cross-thread close ending the span. The one round-8 test that passes there is a control: a stream under SIGINT was already exported as cancelled. The 5 changed test_inflight_claim.py tests fail there because a wait running at shutdown or exit ended OK; they now expect cancelled. The 200-thread test passes on both; only its bound changed.
  • Runtime matrix at e29fcc6. Fresh git archive export and fresh uv venvs, Python 3.10.17 / 3.11.12 / 3.12.10 / 3.13.15 × replicate 1.0.0 / 1.0.7: all 8 cells 390 passed, 0 failed, 0 skipped (httpx 0.28.1, httpcore 1.0.9, anyio 4.15.1, OpenTelemetry SDK 1.45.1, opentelemetry-instrumentation-httpx 0.66b1, respx 0.23.1, mocket 3.15.0, sentry-sdk 2.72.0, truststore 0.10.5, trio 0.34.0). The same matrix at c5b9505, before the live run, was 8/8 with 390 passed each.
  • CI at e29fcc6. Run 39597696665, started by the code push, and run 39598488802, started by the PR-description edit; Plan, fi_instrumentation, package (replicate) and Python tests passed in both. The prior red run on 55c5672 was 39587088349; its failure is described above.
  • Live, ninth session, 2026-10-21 11:51 UTC. One paid prediction (meta/meta-llama-3-8b-instruct, max_tokens 24) through the module function replicate.stream (sync), instrument(tracer_provider=...) with no other option, the package imported from a git archive of c5b9505 (e29fcc6 changes only the README), Python 3.12.10, replicate 1.0.7, exported to the local project th8103-livefill-th8320; the same recipe as session 8. Events output x5 and done; the span ended on the caller's thread inside the next() that returned done (0.36 ms including the read of the event; end time 0.26 ms before it returned); the loop exited 0.41 ms after done. Counted at httpx's transport classes: the create and the event stream only, nothing after the stream, during shutdown() or in the 0.5 s after it; thread names unchanged by instrument(), the stream and the flush; shutdown() 0.14 ms. A dry run against loopback stand-ins came first, and a ledger stops a second paid call. fi_verify (wait 240, min spans 1) PASS: the new trace holds one replicate.stream span, LLM, OK, with the prediction id and the joined text, tokens stored as 0 and no status, predict_time or replicate.stream.* attribute. The account's prediction list since 2026-10-20 went from 15 to 16.
  • Privacy scan after the session. ClickHouse tables enumerated through system.tables / system.columns (49 read by project id, trace id, or whole when small, no table skipped; spans with and without FINAL; 17 views, materialized views and dictionaries listed, not read), plus every Postgres tracer_* table (by project id, tracer_project by id, else trace id, else whole). 99 sources read, 10 with rows, 153 rows. Counts: API token 0, its last 12 characters 0, r8_ 0, Bearer 0, authorization 0, /Users/ 0, /private/ 0, login 0, host 0, Traceback 0, File " 0, site-packages 0, exception.stacktrace 0, api.replicate.com 0; the session marker 3 (spans, spans FINAL, traces). The token is counted client-side, never sent in a query. Token counts over this round's evidence files, this description, the docs PR's description, the README, the CHANGELOG and the docs page: 0 (whole token and last 12 characters; token counts were run after this description and both PR bodies were written).
  • Performance and fork, measured from a git archive of e29fcc6 (Python 3.12.10, replicate 1.0.7, httpx 0.28.1, httpcore 1.0.9, anyio 4.15.1, OTel SDK 1.45.1; loopback stand-in; 10-core laptop shared with other work, 1-minute load average 13.35–22.02, 5-minute 16.07–17.74, 15-minute 13.14–13.49; each output block headed with the SHA, the package tree 884786e, time, load and versions). Quoted blocks:
    • [A1] asyncio.run() per call, 10 calls, 0.2 s read, read on: p50 9 ms, max 12 ms, 10/10 reads succeeded (uninstrumented in the same block p50 6 ms, max 50 ms). [A2] read off: p50 7 ms, max 10 ms, 0 reads (uninstrumented p50 7 ms, max 44 ms).
    • [D1] complete_stream() on the caller's thread, 500 threads × 5 runs, read on (2 s): p50 0.107–0.133 ms, p99 0.308–0.545 ms, max 0.63–3.45 ms per run; 2,500 spans OK, 320 with read data, 2,180 skipped (500 streams complete at once against the cap of 64), so these figures mostly measure the inline end of a skipped stream.
    • [G1] 200 concurrent async streams, one shared client, 3 runs, median done→loop end: uninstrumented 10.7–16.0 ms, read off 11.7–18.8 ms, read on 21.1–26.6 ms. [G2] a client per stream, 2 runs: 10.9–11.3 ms, 15.0–17.7 ms, 32.7–89.9 ms.
    • [S1] 160 streams at 20/s for 8 s, inline exporter 0.1 s per span, 2 s budget: 146 read data, 14 at_capacity, peak 64 pending; caller loop p50 3 ms, p99 108 ms; last span exported at 15.2 s.
    • [F1] Fork with 10 reads in flight (read on, 8 s; the stand-in answers reads after 3 s): fork() 3.3 ms in the parent; the child exited 0, started with no package thread and nothing pending, AnyIO's asyncio backend had been imported before the fork, its own async httpx call finished within 8 s, its first stream took 12 ms and its read ended succeeded 3.02 s later on the child's own finisher; the parent's 10 spans ended succeeded. [F2] Finisher stuck in an export, four forks: 5.002 s, then 0.001, 0.002 and 0.001 s.
    • F1/F2 set no_proxy: on macOS a forked child that looks up system proxies after the parent did (CPython's _scproxy, used by every httpx client with trust_env) crashes in the Objective-C runtime with or without the package (reproduced in round 7 with plain urllib.request.getproxies() and no httpx, replicate or traceai). Documented on the docs page.
    • Two earlier runs of the same package code (a local commit that differed from e29fcc6 only in the README, amended before the push) are kept with the evidence and not quoted; the first lacked [D1] because psutil was missing from the probe venv. The probe scripts, the fork probe among them, are kept with the evidence.
  • h3, httpcore.ReadError: [Errno 9] Bad file descriptor on the caller's create POST (round 7). Most likely an httpcore 1.0.9 pool race in the caller's own sync pool, not the read machinery; the evidence is circumstantial. The test that hit it passed 50 of 50 reruns, each in its own pytest process, from a git archive of e288580 in a fresh 3.12 / replicate 1.0.7 venv. In httpcore 1.0.9's _sync/connection_pool.py, _assign_requests_to_connections decides under the pool lock to close an IDLE connection when len([connection.is_idle() for connection in self._connections]) (which counts every connection) exceeds max_keepalive_connections (20), and closes it after releasing the lock; a connection just handed to another thread's request is still IDLE until that thread marks it ACTIVE, so it can be closed while that request uses it. The package does no fd-level work and never uses the caller's pool. A plain-httpx reproducer, with no traceai or replicate imported (200 threads sharing one httpx.Client, a keep-alive POST then a Connection: close GET), hit ReadError: [Errno 9] Bad file descriptor in 2 of 12 widened runs (2 of the 10 at 200 threads; one run with 1 such error, on the POST; one with 11 among 15 errors), 0 of the 2 widened runs at 100 and 150 threads; one of the two hits was an exploratory run whose console output was copied by hand, not captured by the script. Both stock runs (10 rounds each at 100 and 200 threads, 3,000 calls) had 0 errors. The strongest evidence that the read is not the cause: on the original shared-client form of the test at e29fcc6, the error appeared with the read OFF (3 of 100 runs, no package thread or pool at all), not with the read on (0 of 100), and not with replicate alone, uninstrumented (0 of 200). 3/100 vs 0/200 is only weakly significant (one-sided p ≈ 0.04), so this supports 'not the read machinery' but not 'independent of the instrumentation's timing'. The test now gives each thread its own client, so it measures the reads it is about.
  • Docs build (docs(integrations): Replicate tracing (TH-8320) docs#1015 at bf6bb646): BUILD_EXIT=0, AUDIT_EXIT=0, 0 broken nav links, 0 broken content links, 30 orphan pages: 29 on docs main plus this new page, which is not in the navigation yet (reachable by URL). The build script's output (last 25 build lines and the audit summary) is kept with the round's evidence; a full 3,726-line build + audit run at bf6bb646 also exited 0 with no page-specific warnings.

Background for m1: partial output

  • OpenTelemetry GenAI semantic conventions (open-telemetry/semantic-conventions-genai at 8538971, docs/gen-ai/client-inference.md, gen_ai.output.messages): "When an operation ends in an error, this attribute SHOULD only be set when the operation produced output, including partial streamed output. Instrumentation SHOULD NOT synthesize output content for a failed operation." Spans end "when the response is fully received or the operation is terminated due to an error or cancellation" (gen-ai-spans.md).
  • opentelemetry-instrumentation-openai-v2 (opentelemetry-python-contrib at de43f81, patch.py): BaseStreamWrapper.close() and an exception in __next__ call cleanup(), which emits the text received so far ("finish_reason": choice.finish_reason or "error") before ending the span (with handle_span_exception on error).
  • OpenInference OpenAI (Arize-ai/openinference at ca3ed46, _stream.py / _utils.py): an exception in __next__ calls _finish_tracing(status=ERROR), which writes the response accumulator's attributes (the output so far).
  • OpenLLMetry OpenAI (traceloop/openllmetry at 8d9c93f, chat_wrappers.py): _ensure_cleanup() on an exception, __exit__ or __del__ runs _record_partial_metrics(), which sets the completions "for partial streams (parity with happy path)".

This package keeps the received text with the same status rules, synthesizes nothing, and leaves output empty when nothing was received (kind CHAIN then, as for any call without output).

Limits and remaining behaviour

  • The opt-in read in its current form has been checked against stand-ins and the probes above, not the live API. Calls in flight at a shutdown or exit are checked against the stand-in and with the real register() handler in subprocesses, not live.
  • A call ended by a shutdown that does not exit (another thread calls provider.shutdown()) is recorded as cancelled although the call itself goes on; its later result ends no second span.
  • A mock that starts after a request has passed the final checks and begun sending can still see that one in-flight read; a mock below the checked HTTP and socket methods, and any stub server, is not detected.
  • The finisher wait (up to 5 s) applies to flush, shutdown, exit and uninstrument() with the read on, and timeout_millis does not shorten it. With an inline exporter, ending pending spans still costs up to 64 per-span exports.
  • After uninstrument(), worker threads stay idle until exit. A stream still open after uninstrument() ends at drop, the next traced call or exit; if the provider was shut down before that, its span is not exported.
  • A fork waits up to 5 s in all for the read loop and the finisher, once per stuck thread; after that a child can inherit a lock an exporter holds. Dispatcher, resolver and BatchSpanProcessor threads are not coordinated. On macOS, see the _scproxy note above.
  • A KeyboardInterrupt exactly inside OutputCollector.add() can leave that one chunk unrecorded (the span then ends as cancelled at drop or exit, with the chunks before it).

History (condensed)

Earlier rounds moved the read after a stream off the caller's path, then made it opt-in (round 4), restricted it to the package's own pool for plain transports and skipped HTTP mocks (round 5), and added socket-mock detection, AnyIO pre-import, fork coordination and cancellation of reads in flight (round 6). Round 7 kept a stopped stream's output, recorded Ctrl-C inside next() as a cancel, added the fork wait memo and the subclass-resolver skip, and moved every quoted figure to a measured head. The round-6 description said "12 regressions"; precisely, round 6 added 12 tests that all failed on aea2604, 10 on the behaviour they target and 2 only because a module attribute they patch or call (importlib, socket_layer_patched) did not exist there. Live sessions 1–9 are described in the README; sessions 2–5 ran earlier designs of the read, 6–7 ran with the read off on code from before rounds 6 and 7, 8 on the round-7 code, and 9 (the default path, read off) on the round-8 code.

🤖 Generated with Claude Code

nik13 added 17 commits October 3, 2026 18:58
The Node OTLP exporter streams with Transfer-Encoding: chunked and sends
no Content-Length. The Receiver read Content-Length only, so every Node
export came back 400 with zero spans; two TH-8103 children had to put a
de-chunking relay in front of it. Receiver now accepts both framings.

It also records one entry per accepted export (path, lower-cased
headers, flattened resource attributes) via requests(), so a contract
test can assert X-Api-Key/X-Secret-Key and project_name through the
shared harness instead of a private recorder.

Verified: 6 harness tests pass, and the TanStack example's real Node
exporter delivered 4 chunked exports (4 spans, collector path, both
auth headers, project_name/project_type) with no relay.

Refs: TH-8339, TH-8103
Real replicate 1.x client calls against FakeReplicate (create via version,
official-model and deployment routes, get, cancel, SSE stream, versions,
trainings) through a recording httpx transport that refuses any non-loopback
host. Covers run/async_run, create+wait (one span), create without wait,
cancel, streams and iterators, token redaction, TraceConfig hide flags,
failure isolation, uninstrument identity restore, packaging ranges, the
TH-8339 Receiver contract and the example script.

RED: traceai_replicate does not exist yet.

approved: Nikhil 2026-10-03 blanket
Refs: TH-8320
…client

ReplicateInstrumentor wraps replicate 1.x (>=1.0.0,<2):

- Client.run/async_run -> replicate.run; Client.stream/async_stream ->
  replicate.stream (one span per stream, open until it ends, is closed or
  is dropped; wrapt.ObjectProxy keeps the vendor generator type).
- Predictions/ModelsPredictions/DeploymentsPredictions/DeploymentPredictions
  create (+async) -> replicate.predictions.create. A non-terminal prediction
  keeps the span open on the returned object so wait/cancel end it (one
  span, AC-03); otherwise it ends with the create-time status and end time
  when the object is released, at uninstrument() or at exit (AC-02, J2.4).
- Prediction.wait/async_wait and cancel/async_cancel, Predictions.cancel
  (+async): continue an open create span or open one span; the client's
  poll loop never adds spans. trainings, get, list and reload are untouched.
- replicate.run & co. (bound to default_client at import) are rebound
  through the wrappers and restored by identity.

Spans are current during the vendor call so HTTP spans nest; nested
wrapped calls are suppressed with a ContextVar. Attributes: provider,
model, prediction id/status/version, predict_time, output type,
input/output via TraceConfig; no token usage or cost. The API token is
redacted wherever the client stored it; the environment is never read.
Caller cancellation -> ERROR "cancelled" + replicate.cancelled; a
canceled prediction is a terminal state, not an error (J3.3, AC-04).

Test corrections found while running against the real SDK: replicate
1.0.x drops wait= when predictions.create delegates model=, so the
Prefer-wait tests use the version route; pickling/copying are compared
with a plain Prediction (the client's httpx transport is not picklable);
uninstrument checks use identity (a FunctionWrapper == its function).
Added coverage: chained async create().async_wait(), a missing vendor
method at instrument time, a no-environment-read source check, and a
subprocess exit-without-wait export through the Receiver.

approved: Nikhil 2026-10-03 blanket
Refs: TH-8320
…1.0.0 floor

- A file handle in the prediction input is recorded as "<BytesIO>"; the
  client still reads and inlines it (file_encoding_strategy="base64").
- The example test accepts replicate 1.0.0, which sends no Prefer: wait by
  default and so polls; 1.0.7 does not. The span contract is unchanged.

approved: Nikhil 2026-10-03 blanket
Refs: TH-8320
Documents the span per call (run, stream, create held open for
wait/cancel, wait, cancel), every attribute, TraceConfig content control,
token redaction, file/data-URI handling, webhook presence-only recording,
error/cancellation semantics, and the known limits (Prediction.stream and
output_iterator not wrapped, no Cancel-After option in replicate 1.x,
env-only token before the first request).

approved: Nikhil 2026-10-03 blanket
Refs: TH-8320
One row in the root README's Python "LLM Providers" table and one in
python/README.md's "LLM Providers" table.

approved: Nikhil 2026-10-03 blanket
Refs: TH-8320
…lock

Runs in a subprocess through harness.run: a PendingPrediction released
while its thread holds the pending-span registry lock (as the cyclic GC
can do) must end its span without deadlocking. RED on the non-reentrant
lock (the subprocess times out at 30 s).

approved: Nikhil 2026-10-03 blanket
Refs: TH-8320
A PendingPrediction finalised by the cyclic GC inside a registry
operation ends its span and calls discard() on the same thread; with a
plain Lock that deadlocks.

approved: Nikhil 2026-10-03 blanket
Refs: TH-8320
…shuts down

register() installs SIGTERM/SIGINT handlers that shut the tracer provider
down and then sys.exit(0). Held create spans were ended only from an
atexit hook, which runs after the processors have stopped, so every span
held for a prediction nobody waited on was dropped (batch) or swept up
without its prediction attributes as gen_ai.span.leaked (simple).

instrument() now replaces force_flush and shutdown on the provider
instance it was given with a hook that ends every held create span (as of
create time, with the create-time status) and then calls the original.
uninstrument() puts back exactly what was on the instance before (by
identity), leaves a hook someone layered on top in place as a
pass-through, and providers without these methods (no-op, proxy) or with
read-only attributes (__slots__) are left untouched without raising. The
atexit hook stays as the fallback. A span still ends exactly once.

Tests: test_signals.py runs register() + the real OTLP exporter into the
harness Receiver in a subprocess, holds a create, and sends SIGTERM or
SIGINT (batch and simple processors). test_provider_hooks.py covers
force_flush/shutdown export without release, a later wait() getting its
own span, and identity restore.

approved: Nikhil 2026-10-03 blanket
Refs: TH-8320
…nds)

Rick accepted the held-open create span (one span for create + wait,
AC-03) as a deviation from spec J2.4, with a cap on how long delivery can
be deferred. A create span that no wait()/cancel() continued is now held
for at most max_pending_seconds (default 600). An older one is ended,
with its create-time status and create-time end timestamp, the next time
the registry is touched: any traced call (sync or async), force_flush or
shutdown. A later wait() on that prediction gets its own
replicate.prediction.wait span.

The registry keeps entries in hold order so the check stops at the first
young entry. instrument(max_pending_seconds=...) takes a positive, finite
int or float; anything else raises TypeError (type) or ValueError (value)
before anything is patched. The README states the J2.4 deviation and why
(AC-02 vs AC-03), the cap and the flush/shutdown behaviour.

approved: Nikhil 2026-10-03 blanket
Refs: TH-8320
PendingPrediction, TracedIterator and TracedAsyncIterator ended their span
inside __del__. The cyclic GC can run __del__ on any thread at any
allocation, including while that thread holds a non-reentrant lock on the
export path (register(batch=False) exports inline through the HTTP pool),
so a finaliser that runs the processors can deadlock.

__del__ now only appends (call, cancelled, release time) to a module-level
deque (deque.append is atomic). The queue is drained, and the spans ended,
at a safe point: on every traced call's entry, on registry operations
(add/get/touch/expire_all), in the hooked provider force_flush/shutdown,
at uninstrument() and at exit (the instrumentor's atexit hook, plus a
module-level drain that stays registered for a stream released after
uninstrument). Spans still end exactly once and keep their original end
timestamps: create time for a held prediction, drop time for an abandoned
stream or iterator.

Tests: test_release.py records span.end calls and shows __del__ makes
none (RED before: ['MainThread'] == []) for a held prediction, a sync
stream, a run iterator and an async stream, then drains through the next
traced call, force_flush, uninstrument and shutdown. test_exit.py and
test_signals.py gain released variants (drain at exit / at the signal).
Two existing GC tests now drain explicitly with drain_released():
test_predictions.py::test_create_without_wait_is_one_span_ended_at_create_and_never_polled
and test_streams.py::test_dropping_a_stream_mid_iteration_ends_its_span_as_cancelled.

approved: Nikhil 2026-10-03 blanket
Refs: TH-8320
…e 1.0.0/1.0.7

Rick ran the full suite on every cell of {3.10, 3.11, 3.12, 3.13} x
{replicate 1.0.0, 1.0.7} (79 passed at 370f7ef; rerun at the final head of
this round). The README said 3.10, 3.11 and 3.13 and the classifiers left
out 3.12, which poetry-core would add anyway from python = ">=3.10,<3.14".

README and classifiers now name 3.10, 3.11, 3.12 and 3.13 with replicate
1.0.0 and 1.0.7. test_packaging.py pins the classifier list to that matrix
and checks that the README sentence states it.

approved: Nikhil 2026-10-03 blanket
Refs: TH-8320
….cancelled

R5: with Prefer: wait (the default wait=True in replicate 1.0.7, an
explicit wait= in 1.0.0) the client's run() returns without polling once
the create response is no longer 'starting', and the server can answer
'processing' for a finished prediction (vendor run.py). The status row now
says a replicate.run span can show 'processing' on an OK span with output.

V1: the replicate.cancelled row said 'true when the caller cancelled',
which an explicit cancel() does not set. It now reads: true when the
caller abandoned the call (closed/dropped an iterator, cancelled a task);
an explicit cancel() is recorded as status canceled.

Docs only; no code change.

approved: Nikhil 2026-10-03 blanket
Refs: TH-8320
Verifier finding V2 (t_053f4104): LifecycleWrapper found the held create
span with registry.get(), which left it registered with its create-time
held_since. While wait() polled, a traced call on another thread or task
(the max_pending_seconds cap) or provider.force_flush() ended the span as
of create time with status 'starting'; when the wait returned, finish()
was a no-op and the final status, output and duration were lost.

wait()/cancel() now claim the span: registry.claim() pops it from the
held set under the lock and records it as claimed, so the cap, a flush
and a release of the prediction object leave it to that call, which ends
it with its result. Provider shutdown and interpreter exit (end_all) still
end a claimed span as a best effort, as of that moment with the last
polled status, so it is exported before the processors stop; the call's
later finish() ends nothing. force_flush and uninstrument() keep ending
only held spans. finish() is an atomic non-blocking test-and-set, so a
shutdown run from register()'s signal handler while this thread is inside
another finish() returns instead of deadlocking.

Tests first (tests/test_inflight_claim.py, 18 cases): cap and flush during
a running wait and cancel (sync, async, by id), a release while a cancel
by id runs, shutdown during a running wait (sync, async) and SIGTERM/exit
under register() with batch and simple processors. Against 0bcc3f5:
18 failed (13 V2 early ends, 5 shutdown ended as of create with
'starting'). Full suite after: 136 passed (Python 3.11, replicate 1.0.7).

approved: Nikhil 2026-10-03 blanket
Refs: TH-8320
README: while wait()/cancel() runs, the cap, a flush and a release leave
its create span alone; shutdown and exit end it as a best effort with the
last polled status. New 'Which provider' section: without
tracer_provider= the hooks go on the global provider at instrument()
time, so call register() before instrument() (the placeholder provider
has no force_flush/shutdown to hook). Verify-r2 t_053f4104 P3 follow-ups.

approved: Nikhil 2026-10-03 blanket
Refs: TH-8320
@nik13
nik13 requested a review from NVJKKartik October 10, 2026 08:12
nik13 and others added 2 commits October 11, 2026 21:07
Replicate's language models report metrics.input_token_count and
metrics.output_token_count. LLM spans were showing 0 tokens; record them as
gen_ai.usage.input_tokens / output_tokens / total_tokens when present
(never estimated, no cost). This reverses the earlier no-token rule, which
assumed Replicate reports none.

Also apply pii_redaction to the error status and the exception event, and
add a CHANGELOG.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The catalog rows for all TH-8103 integrations now live in one PR
(docs/th-8103-integration-catalog), so this PR no longer conflicts with
its siblings in README.md and can merge in any order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… name)

span.record_exception() always adds a Python traceback, which carries the
caller's file paths, user name and virtualenv. traceAI never exports
those, so the exception event is now built with add_event and keeps only
exception.type (module.QualName, bare for builtins, as the SDK writes it)
and exception.message (token-redacted, PII-redacted, __REDACTED__ when
outputs are hidden). Found in the 2026-10-16 re-verification: the OTLP
body sent to Future AGI carried /Users/<name>/.../site-packages frames.

tests/test_no_stacktrace.py fails on 3e6d005 (4 failed) and passes now:
ModelError (default and hidden), ReplicateError, ValueError and a stream
RuntimeError export no stacktrace, no Traceback, no site-packages path,
no test-file path and no user name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nik13 and others added 7 commits October 20, 2026 06:19
…cation notes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nal status and usage

e7e16c4's title announced dropping the placeholder version "hidden", but its
diff changed README.md only: the code kept recording it, and the rows stored in
the 2026-10-20 live run carry replicate.prediction.version="hidden". This
commit makes the change in code:

- real_version() drops "hidden" (and empty or non-string values) wherever a
  version is read: the prediction a call returns (run, async_run, stream, the
  held create snapshot, wait, cancel), a "owner/name:hidden" ref, a ref tuple,
  and a Version object. A real version id is still recorded.

Streams: the SSE stream carries no status or metrics, so stream spans had no
status, predict_time or token counts. After a completed stream (StopIteration
after "done") the prediction is now read once with predictions.get /
async_get (GET /v1/predictions/{id}: free, creates nothing), and once more
poll_interval later if it is not finished yet. New option
stream_reload_seconds (default 2.0, 0 = off) bounds it: the sync read runs on
a daemon thread joined for the budget (the client's RetryTransport honours an
unbounded Retry-After), the async read uses asyncio.wait_for. The span keeps
the stream's end time. No read after an error, close() or a drop. _write keeps
a terminal status on a stream span and still drops the create-time one.

Tests: tests/test_live_shapes.py (29 tests; 21 fail on e7e16c4) and an
official-model route in the stand-in that answers version "hidden" and lets a
stream's prediction finish at "done", as the live API did.

README / CHANGELOG: the option, the stream row, the version row, the CHAIN
kind of a failed call, 429 behaviour, and the live-verification section
restated to the stored evidence (second live session 2026-10-20 14:59 UTC:
async_run, stream with the read after done, and HTTP 422 stored).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e is completion

The read after a completed stream (GET /v1/predictions/{id}) ran inside the
caller's last next(): up to stream_reload_seconds of added latency, a
caller's asyncio.timeout tripped by it, RetryTransport retries with an
uncapped Retry-After on a thread left running past the join, an HTTP child
span starting after the parent's end, and a KeyboardInterrupt in join()
leaking the span. A caller that broke at done and closed the stream got an
ERROR "cancelled" span with no output and no read.

- A stream completes at its done event (or the end of iteration). From then
  a close(), a drop or a later exception leaves the span OK with its output.
- complete_stream() fixes the end time and returns at once; a finisher
  thread starts a reader and ends the span when the read returns or the
  budget runs out, as of the stream's end. Async streams use the same
  threads.
- The read: one GET, one more poll_interval later only when the prediction
  is not finished; sent to the transport under replicate's RetryTransport
  (no retry, Retry-After ignored); every phase given at most the time left;
  body capped at 2 MiB and dropped at the deadline; OTel instrumentation
  suppressed, so no HTTP span; only id/status/version/model/metrics/error
  kept. At most 256 reads in flight; past that a stream ends without one.
- force_flush/shutdown/uninstrument/exit (atexit) wait for reads in flight
  at most until their deadline, then end the spans; a BaseException while
  waiting ends them first.
- Tests (test_stream_read_offpath.py, 22): 21 fail on 18f7f8b. The stand-in
  transport now honours instrumentation suppression like a real HTTP
  instrumentation; the fake can send Retry-After on a refused read.
- README/CHANGELOG: the read's cost and rate-limit note, how to turn it off,
  the third live session, and the stray "before the fix below" wording.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Round-2 review of traceAI#219 (ab5f5c4). The read after a stream moves to
a new module, traceai_replicate/_stream_read.py:

- Completing a stream is one SimpleQueue.put: no thread starts on the
  caller's thread or loop (only the finisher, once per process), so the
  next() that returns done no longer pays two Thread.start handshakes.
- With replicate's own transport (no transport=/mounts=), the read goes
  through a package-owned httpx pool (16 connections) on a package event
  loop, with the client's base URL, headers and proxy options. It never
  borrows the caller's pool and never builds a sync httpx client on an
  async-only Client. The budget is an absolute deadline (an anyio cancel
  scope: a plain asyncio cancel was absorbed inside httpcore's own scopes
  under 100 concurrent trickling reads), and name lookups run on at most
  2 daemon resolver threads.
- Through a caller transport: at most 4 live reads on daemon reader
  threads (or a task on the caller's loop for an async-only transport,
  which is now read instead of silently skipped), none on an httpx pool
  under 8 connections. The cap counts reads until their I/O returns.
- No read for a non-recording span, after provider shutdown, or without
  a provider hook; calls under the caller's suppress-instrumentation
  context pass through untraced.
- force_flush(timeout_millis) waits at most half its timeout, cuts what
  is left (span ends with what is known) and passes the remaining time
  to the provider's flush; shutdown/exit wait at most the budget;
  uninstrument() does not wait. stream_reload_seconds is capped at 10.
- replicate.stream.read / done_reason / truncated; done reason canceled
  records status canceled; aborted is a final status.
- Accept-Encoding: identity and a bounded decompressor; a closed client
  is not read; a forked child never ends its parent's spans; a stream
  that is never iterated has no span (the span starts at first next()).

Tests: test_stream_read_bounds.py ports the verifiers' probes (28 of 31
fail on ab5f5c4); FakeReplicate gains trickled reads and truncated,
canceled and aborted streams.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… pool

Fourth live session (one paid prediction, meta/meta-llama-3-8b-instruct,
max_tokens 24) on 22832fd: client.async_stream on a client with
replicate's default transport. The __anext__ that returned done took
0.42 ms; the read went through the package's own connection pool and the
span ended on the finisher thread 104 ms after the loop exited; no sync
httpx client was built on the Client. Stored span: LLM, OK, succeeded,
predict_time, 39/17 tokens, the only span in its trace (trace
b3858eab-adff-4d03-8720-3d56d459f054). Eleven predictions since
2026-10-20 in the account's list; 137 stored rows scanned, the token
appears 0 times.

async_stream moves from stand-in only to live-verified; the read through
a caller's transport is stand-in only in this release.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hare name lookups

Review round 3 of the read after a stream (TH-8320):

- A cancel from the caller's loop (asyncio.run() ending) is never swallowed:
  the read ends there (timeout, or skipped if it never ran) and no second GET
  is sent. A plain httpx.HTTPTransport / AsyncHTTPTransport (no proxy) passed
  to replicate.Client is read through the package's own pool with that
  transport's TLS context, HTTP versions, local address, socket options and
  connect retries, so asyncio.run() does not wait and the caller's pool is
  never lent. Package clients dropped from the cache close only once idle.
- force_flush() no longer waits for reads in flight: their spans end at once
  (replicate.stream.read = flushed) and that flush exports them. The new
  stream_flush_wait_seconds (0 to 10, default 0) opts into a wait, never more
  than half the flush's timeout. shutdown() and exit still wait up to the
  budget. The time passed to the provider's own flush is rounded up (a 1 ms
  flush is no longer cut to 0) and replaces the caller's timeout whatever the
  original's signature; the hook keeps the original's name and signature.
- Name lookups wait for a free resolver within the read's deadline instead
  of failing when more than two overlap; reads of one name share one lookup,
  kept 5 s. A new lookup fails at once only while both resolvers have been
  stuck for 5 s.
- Completing a stream is one SimpleQueue.put (no GIL release). The worker
  threads start in instrument(); a dispatcher hands reads to the read loop in
  batches, and the finisher only ends spans, so an inline exporter no longer
  delays the start of reads. Admitted spans not yet ended count against the
  64 cap, the own pool has 64 connections (one per admitted read), and the
  caller-transport cap is 4 per transport (16 in all).
- A KeyboardInterrupt while the read is planned ends the span and frees its
  slot. A fork waits for the finisher to finish exporting a span. An ended
  span no longer stays referenced until its read's deadline.

tests/test_stream_read_round3.py ports the verifiers' probes (27 tests;
run against ee9cb13's package: 23 fail, 20 of them for the behaviour and 3
only because they use the new option or function; 4 are guards that pass
on both).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nik13 and others added 12 commits October 20, 2026 19:43
README and CHANGELOG: force_flush() does not wait for reads by default
(replicate.stream.read = flushed; stream_flush_wait_seconds opts in), a
plain httpx transport is read through the package's pool, an async-only
transport's read ends with its loop, name lookups are shared and wait for a
resolver, 64 stream spans at most wait at once (one connection each), and
4 caller-transport reads per transport (16 in all). The caller-cost figure
names its setup and load. The flush promise is qualified for inline
exporters; the 2 MiB cap for transports that read whole bodies.

Live section: five sessions, twelve predictions. The third and fourth
sessions are labelled with the code they ran (ab5f5c4, 22832fd); the
Accept-Encoding sentence no longer reads as a live observation. Session 5
(02:37 UTC on 2026-10-21, code b5df956): async_stream inside asyncio.run()
on httpx.AsyncHTTPTransport(retries=2), read through the package's pool.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ds only through the package's pool

Round 4 of the TH-8320 review. stream_reload_seconds now defaults to 0: a
stream span ends at done (or the end of iteration) on the caller's thread,
OK, with what the stream gave and no status or token counts. No request is
made after a stream, no thread is started, and force_flush, shutdown,
uninstrument, exit and fork wait for nothing.

With the opt-in (stream_reload_seconds > 0):
- the read runs only through the package's own pool, for replicate's own
  transport or a plain httpx.HTTPTransport / AsyncHTTPTransport without a
  proxy; any other caller transport is never called for it
  (replicate.stream.read=skipped, replicate.stream.skip_reason=
  custom_transport). The reader-thread and caller-loop routes are gone, so
  no read runs on the caller's thread or event loop;
- shutdown() and interpreter exit no longer wait: pending stream spans end
  at once (replicate.stream.read=shutdown); stream_shutdown_wait_seconds
  (0 to 10, default 0) opts into a bounded wait;
- uninstrument() ends pending stream spans at once (uninstrumented), so a
  shutdown right after it exports them; a stream completing after it makes
  no read (after_uninstrument);
- the name-lookup queue is bounded at 64;
- every skipped span says why (replicate.stream.skip_reason).

Fork: only the finisher thread is coordinated; docs no longer imply span
processors are fork-safe. tests/test_stream_read_round4.py ports the
round-3 verifiers' probes (mock transport, uninstrument then shutdown,
asyncio.run, dns queue) and checks the default literally (threads before
and after, in a subprocess, against an uninstrumented run).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…er is exporting

With the opt-in read, force_flush(), shutdown(), exit and uninstrument() end
pending stream spans at once (d610ede). A span the finisher thread was
exporting at that moment was not waited for, so a process exit right after a
flush could cut it off: the round-4 fork probe lost 2 of 100 child spans
that way. That one export (the span has already ended) is now waited for, at
most 5 s. With the read off there is no finisher and nothing waits.

test_with_the_read_on_a_span_the_finisher_is_exporting_is_not_cut_off
(shutdown, force_flush, uninstrument) fails without the fix. The per-call
asyncio.run() test now uses a 2.5 s read and checks the median, so machine
load cannot be mistaken for a wait.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… (default settings)

README: the opt-in read's caller cost, burst and slow-exporter figures,
re-measured with the round-3 verifiers' probes on a git archive of f1866eb
(load average 11 to 33; outputs in livefill/TH-8320/ev/r4); with an inline
exporter, shutdown and exit still take the export time of the spans waiting
(at most 64), which is export, not a wait for a read. A pointer near the top
says stream spans carry no status or token counts by default.

Live, session 6 (05:15 UTC on 2026-10-21, code f1866eb): sync
replicate.stream with the default settings. The span ended on the caller's
thread inside the next() that returned done (end time 0.27 ms before it
returned), no request after the stream (counted at httpx's transport
classes), no thread started, shutdown() 0.15 ms; the stored span is OK with
no status, tokens or replicate.stream.* attribute, alone in its trace.
Thirteen predictions in all; 145 stored rows scanned.

CHANGELOG: the inline-export note for shutdown and exit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…HTTP mock or a changed transport

Default path:
- A text chunk that starts with https://, http:// or data: (or the FileOutput
  replicate 1.0.7 makes of it) stays in output.value, in order, for streams,
  run iterators and list outputs. Output is files only when every chunk is a
  URL or a file; a text stream reports no file count.
- A KeyboardInterrupt while the span's attributes are written, or while the
  done event's data is read, still ends the span.
- send()/throw() and asend()/athrow() step a stream through the same span
  logic as next()/__anext__().

Opt-in read (stream_reload_seconds > 0):
- No read while httpx's or httpcore's send path is replaced on the class
  (respx, pytest-httpx, VCR.py, mock.patch): skip_reason patched_http, checked
  when the stream completes and again before each request.
  opentelemetry-instrumentation-httpx's wrapt wrappers are allowed.
- Only replicate's own RetryTransport class is looked through, not a
  subclass; a transport or pool with handle_request set on the instance, and
  a client with proxy=, get custom_transport.
- respx is a dev dependency for the new tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…igures and a seventh live session

- CHANGELOG: the default-off bullet leads Added, then the opt-in and its
  limits (patched_http, the stricter custom_transport rule, the exit bound).
- README: the opt-in's skip rules and HTTP-mock check; the exit cost with an
  inline exporter stated as up to 64 times the per-span export time, with
  figures re-measured at 4767825 (and the slow-exporter rate with each
  skip's reason); a stream dropped after uninstrument(); a flush's inline
  export of dropped streams; URL-like text chunks; send()/throw().
- Live session 7 (code 4767825): async_stream with the default settings;
  requests counted through shutdown() and after it. The opt-in read as it is
  now is marked as stand-in only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…it memo; subclass resolver skip

- A stream or run iterator that stops before it completes (close, drop,
  cancelled task, Ctrl-C, an exception, or still open at shutdown/exit)
  keeps the output it had yielded (subject to TraceConfig), so a text
  stream stays LLM; status unchanged. Follows the OTel GenAI conventions
  on partial streamed output.
- KeyboardInterrupt / SystemExit inside next() / __anext__() is recorded
  as the caller stopping the stream (replicate.cancelled), as one in the
  loop body is.
- A fork whose wait for the finisher's export or the read loop ran out
  does not make later forks wait again until that thread has moved on.
- _read_routing never calls a _transport_for_url overridden by a subclass
  of httpx's client: no read, custom_transport.
- Tests for each (failing first on e288580), for Sentry's
  HttpxIntegration (patched_http), truststore.inject_into_ssl
  (patched_socket) and instrument_client (custom_transport); the
  100-thread trickling-read test gives each thread its own client
  (httpcore 1.0.9 pool race on a shared client, not the read).
- README/CHANGELOG: uninstrument() waits for no read but, like flush,
  shutdown and exit, up to 5 s for the finisher's export; skip-reason
  table and stub-server note; no evidence paths or worktree figures.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…mns privacy scan

The eighth session streamed one prediction through replicate.stream with
the read off, importing the package from a git archive of f0376ad: span
ended at done on the caller's thread, nothing sent after the stream, no
package thread, LLM/OK with no status or tokens. README only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ller stops alike everywhere

- A traced call still running (run, async_run, a create, wait or cancel) at
  provider shutdown, including register()'s SIGTERM/SIGINT handler, or at
  interpreter exit now ends then as cancelled, with what it knows, so it is
  exported before the processors stop. An in-flight run()/async_run() or
  create span was lost there before, and a running wait() ended OK.
- Ctrl-C/SystemExit during run(), async_run(), a create, wait() or cancel(),
  a trio cancellation, and throw(GeneratorExit) are recorded as the caller
  stopping the call (replicate.cancelled, no exception event), as inside a
  stream's next().
- A close() the generator refuses because another thread is inside next()
  no longer ends the span; that stream ends it.
- Tests: test_review_round8 (21, 20 fail on 55c5672 for the reason each
  targets; the stream-under-SIGINT case is a control). The 200-thread tail
  bound is now every tail below the read budget (2 s) and the median below
  a quarter of it, after a 0.54 s outlier on CI run 39587088349.
- Dev dependencies: opentelemetry-instrumentation-httpx (CI skipped those
  tests) and trio.
- README/CHANGELOG: fork() ends no span; dead docs link replaced by the PR
  links; the privacy scan listed views and dictionaries, did not read them;
  the h3 cause is worded as most likely.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… after it

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The README said in one place that such a stream ends at the next traced call
or at exit, and in another that a dropped one ends only at exit. Both were
wrong in one case (probe on e29fcc6): a held stream is not ended by the next
traced call, and a dropped one is, once instrument() runs again. Both
passages now say: it ends when closed or completed; once dropped, at the next
traced call (after instrument() again) or at exit; force_flush() and
shutdown() no longer end it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@nik13
nik13 merged commit ab1ef6f into dev Oct 26, 2026
5 of 8 checks passed
@nik13
nik13 deleted the feat/th-8320-replicate branch October 26, 2026 09:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant