Skip to content

docs(integrations): Replicate tracing (TH-8320) - #1015

Merged
nik13 merged 15 commits into
mainfrom
feat/th-8320-replicate-docs
Nov 6, 2026
Merged

nik13 merged 15 commits into
mainfrom
feat/th-8320-replicate-docs

Conversation

@nik13

@nik13 nik13 commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Document the released traceAI-replicate==0.1.0 package with fi-instrumentation-otel==1.2.3 and replicate==1.0.7, from installation through viewing a prediction in Future AGI.
  • Preserve default-off post-stream reads, optional metadata costs and limits, cancellation behavior, and content/error privacy controls.
  • Replace the public verification and benchmark diary with task-focused guidance, add Replicate to the shared integration catalog, and add page-contract coverage to existing CI.

Resolves TH-8320. The required packages are published on PyPI; no separate navigation PR is needed.

🤖 Generated with Claude Code

Page for the traceAI-replicate package (future-agi/traceAI#219): install,
configure, what is captured (run/stream/create/wait/cancel as one span per
call), privacy defaults and the FI_HIDE_* opt-out, tested versions
(replicate 1.0.7 and 1.0.0), troubleshooting and limits. No navigation.ts
edit; the nav entry comes with the TH-8103 nav PR.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nik13 and others added 13 commits October 20, 2026 06:22
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The page claimed a live 422 and 402, async_run stored, and the "hidden"
version not recorded, none of which the stored rows showed at cf52206. A
second live session (2026-10-20 14:59 UTC, traceAI#219 at 18f7f8b) stored
async_run, a stream with its final status and token counts, and an unknown
version answered HTTP 422; this page now says what each session stored, what
is still stand-in only (async_stream, create + wait, cancel, deployments, run
iterators, image and file outputs), and that trainings are not traced.

Also: stream_reload_seconds and the read after done, the "hidden" version,
the CHAIN kind of a failed call, 429 behaviour and troubleshooting (402 is
not claimed), and the unit-suite matrix (3.10-3.13 x replicate 1.0.0/1.0.7).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ne is completion

traceAI#219 ab5f5c4 moves the read after a stream (GET /v1/predictions/{id})
onto a background thread: never retried, no HTTP span, bounded by
stream_reload_seconds, and the span ends when it returns or the budget runs
out. A loop that stops at done and closes the stream is a completed stream.

- New section "The read after a stream": free but counted against the
  account's rate limit for non-create requests, how to turn it off
  (stream_reload_seconds=0), off the caller's path, bounded, no span, when
  the span ends, flush/exit waits, 256 in flight, what is kept.
- Stream row, option text, retry and cancellation bullets, a
  troubleshooting entry for a stream span without tokens.
- Live section: session 3 (one prediction, loop stops at done; 10 in all),
  fi_verify 8 traces, privacy scan of 121 rows.
- Tested versions: the cells the matrix actually ran.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s timeout

Matches traceAI#219 at ee9cb13 (round-2 review):

- The read uses the package's own connections with the default transport
  (never the client's pool), at most 4 through a transport you pass and
  none on an httpx pool under 8 connections; a hard deadline on the own
  pool; at most 64 reads at once (replicate.stream.read=skipped past
  that); no read for a span that is not exported.
- force_flush(timeout_millis) waits at most half its timeout and never
  longer than asked; shutdown and exit wait at most the budget;
  uninstrument() does not wait; stream_reload_seconds is at most 10.
- The caller's cost stated as measured, not "no latency".
- replicate.stream.read / done_reason / truncated, aborted, a done with
  reason canceled, a stream never iterated has no span, suppressed calls
  are not traced; troubleshooting and limitations updated.
- Live: fourth session, async_stream through the package's own pool
  (0.42 ms for the done event, span ended 104 ms after the loop, 39/17
  tokens); eleven predictions in all; 137 stored rows scanned.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
traceAI#219 round 3 (b5df956, head 86915f8): force_flush() does not wait
for reads by default (replicate.stream.read=flushed; the new
stream_flush_wait_seconds opts in); a plain httpx transport is read through
the package's own connections; an async-only transport's read ends with
its event loop; name lookups are shared and wait for a resolver; at most
64 stream spans wait at once; 4 reads per transport of yours, 16 in all.
The caller-cost figure names its setup; "never longer than asked" is
qualified for inline exporters; the 2 MiB cap for transports that read
whole bodies. Troubleshooting lists every skipped cause, Limitations
carries the README's two bullets, dates are UTC. Live: session 5,
async_stream inside asyncio.run() on httpx.AsyncHTTPTransport(retries=2),
twelve predictions in all.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he read after a stream is opt-in

traceAI#219 round 4 (d610ede, f1866eb, README 5ca9a1c). stream_reload_seconds
now defaults to 0: a stream span ends at done on the caller's thread, OK,
without status, predict_time or token counts; no request after the stream,
no thread, and flush, shutdown, uninstrument, exit and fork wait for nothing.
A Warning right after the instrument() options says so, how to opt in, and
the opt-in's limits.

"The read after a stream" becomes "The opt-in read after a stream": only
through the package's own pool, for replicate's default transport or a plain
httpx one (any other transport: skipped, custom_transport); flush, shutdown,
exit and uninstrument() do not wait by default (flushed / shutdown /
uninstrumented; stream_flush_wait_seconds and the new
stream_shutdown_wait_seconds opt in); forks: only the package's finisher is
coordinated; a table of every replicate.stream.skip_reason. The measured
cost is from probes at f1866eb. Troubleshooting and Limitations follow.

Live: session 6 (05:15 UTC on 2026-10-21, traceAI f1866eb), sync
replicate.stream with the default settings; thirteen predictions in all;
sessions 2 to 5 are labelled as run with the read on. The fifth session's
"looked up once" is restated as what was measured (one cache entry).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…for HTTP mocks and changed transports; seventh live session

- The opening paragraph says stream spans carry no status or token counts by
  default; the description no longer promises status and tokens for streams.
- The opt-in read's rules as in traceAI#219 round 5: only replicate's own
  RetryTransport over a plain httpx transport without a proxy and unchanged on
  the instance; no read while an HTTP mock patches httpx or httpcore
  (patched_http); the exit cost with an inline exporter.
- Default path: a flush's inline export of dropped streams; URL-like text
  chunks stay in the output; send()/throw().
- Live: the opt-in read in its current form is stand-in only; session 7
  (async_stream, default settings); fourteen predictions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…aits only for the finisher's export; final-head figures; eighth live session

- A stream stopped before done keeps what it received (LLM for text);
  Ctrl-C recorded alike inside next() and in the loop body.
- uninstrument() never waits for a read; like flush, shutdown and exit
  it waits up to 5 s for the span the finisher is exporting;
  stream_shutdown_wait_seconds also applies at exit; force_flush(0)
  still exports held and dropped spans inline.
- Skip reasons: Sentry HttpxIntegration (patched_http),
  truststore.inject_into_ssl (patched_socket), instrument_client and a
  subclass's _transport_for_url (custom_transport); stub servers see
  the read. Truncated-stream status only with the read on.
- Figures measured from a git archive of traceAI 55c5672 (probe blocks
  named); fork memo and the macOS _scproxy fork hazard; no evidence paths.
- Eighth live session (traceAI f0376ad) and a system.columns-driven scan.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…celled; caller stops alike; figures at traceAI e29fcc6; ninth live session

- Calls still running at a shutdown (including register()'s SIGTERM/SIGINT
  handler) or at exit end as cancelled; Ctrl-C/SystemExit during run(), a
  create, wait() or cancel(), trio cancellations and throw(GeneratorExit)
  are the caller stopping the call; a refused cross-thread close ends nothing.
- "do not wait" / "never waits" now say "for a read" everywhere on the page;
  fork() ends no span.
- Figures re-measured from a git archive of traceAI e29fcc6, with the
  1-minute and 5-minute load and the [D1] skip proportion.
- Ninth live session (traceAI c5b9505); the privacy scan listed views and
  dictionaries without reading them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A stream still open at uninstrument() ends when closed or completed; once
dropped, at the next traced call (after instrument() again) or at exit.
The troubleshooting entry now carries the 5 s finisher-export clause like
the rest of the page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ackage version (TH-8320)

The page merges only after traceAI-replicate 0.1.0 is on PyPI, so the note saying it is not there yet would be wrong the day the page goes live. Names traceAI#219 instead of 'the current pull request' / 'this release candidate'.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Merge main and replace the verification diary with an actionable guide for the released Replicate instrumentor. Preserve default-off stream reads, rate-limit and lifecycle limits, cancellation semantics, and privacy controls for the current core release. Register the page in the shared catalog and protect the customer-facing contract in CI.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@nik13
nik13 marked this pull request as ready for review November 6, 2026 05:59
@nik13
nik13 merged commit 1f1836f into main Nov 6, 2026
8 checks passed
@nik13
nik13 deleted the feat/th-8320-replicate-docs branch November 6, 2026 07:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant