Skip to content

docs(auto): make zero-code Node tracing easy to start - #617

Merged
nik13 merged 3 commits into
devfrom
docs/auto-readme
Nov 8, 2026
Merged

nik13 merged 3 commits into
devfrom
docs/auto-readme

Conversation

@nik13

@nik13 nik13 commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Make the released @traceai/auto package understandable and usable from GitHub in two minutes, without changing runtime behavior.

  • Lead with a pinned 0.1.0 install and a single Node preload; cover ESM, CommonJS and NODE_OPTIONS.
  • Add a prominent zero-code entry point to the root README and distinguish auto setup from manual fi-core instrumentation.
  • Generate the 48-entry library table from the runtime registry, distinguishing import hooks, optional enable entries and manual-only integrations.
  • Move detailed capture/privacy internals into docs/ARCHITECTURE.md; add complete configuration and framework references.
  • Give content capture, redaction defaults, external-exporter policy, bounded capture and unsupported transports explicit treatment.
  • Correct two misleading statements: custom exporters do not require FI credentials, and global-fetch instrumentation does not cover direct Undici/node-fetch calls.
  • Add documentation regression checks and a disposable-consumer harness that executes code blocks directly from the Markdown.

Why

The old 736-line README led with implementation details, obscured the first successful request, omitted settings and mixed setup instructions with internal test/release notes. The root README did not expose the zero-code package at all.

The structure follows task-first patterns from OpenTelemetry zero-code, OpenLLMetry JS, OpenInference JS, Langfuse, Sentry and Datadog, while taking every product claim from traceAI's own code.

Verification

All execution used E2B with synthetic credentials, a loopback OTLP receiver and a mocked OpenAI-compatible provider. No real model calls or customer/Cloud data were used.

Final exact-head run fu-auto-readme-head-f5dd0c53-1108e completed successfully; fetched source-head.txt matches f5dd0c53557bc122be17b9a9731c455214016a1f. Earlier full recipe run: fu-auto-readme-recipes-1108c.

Final results:

  • 17 recipe/behavior checks passed for the public npm @traceai/auto@0.1.0 consumer and the packed workspace consumer, 34 total.
  • Both consumers exported real OTLP spans for ESM, CJS, NODE_OPTIONS, the root quickstart, a custom exporter without FI credentials, content-hidden TOML, esbuild, Vite SSR, Rollup, webpack, two warm Lambda invocations, and a real Next production build/start.
  • Next 15.5.29, OpenAI 4.104.0, esbuild 0.27.2, Vite 7.3.1, Rollup 4.56.0, webpack 5.104.1, Node 22.18.0.
  • Doctor reported accepted; --no-check and bootstrap sent no test spans. The edge observer did not create an implicit exporter.
  • Documentation checks: 3 failures with the original README/spec restored, then 5/5 passing with the rewrite restored.
  • fi-core: 59 suites passed, 1,728 tests passed, 13 existing skips.
  • Both installed consumers also passed checks for inherited OTel batch/span-limit defaults, equality of all 48 registry entries with source, and invocation-parameter hiding's derived error privacy.
  • Full auto suite: 32 suites passed; 1,282 tests passed, 12 existing skips, one failed registry-count assertion, plus the existing safety.test.ts Buffer/BodyInit TypeScript compilation failure. Both failures reproduce on unchanged base 00c0313790f6493830cae6b0cd536f40da791fc1 in fu-auto-readme-base-full-1108d; no runtime/test suppression is included. That base run additionally exposed a build-order catalog byte-identity failure, absent from the final packed-consumer sequence.
  • Legal-notice tests pass. The README retains the redistribution notice instead of weakening its existing assertion.
  • Inspected the rendered public GitHub README, including code blocks, privacy/configuration tables and the expanded 48-row coverage disclosure. Public docs-site targets resolve.
  • All E2B sandboxes were deleted; the runner verified get_info returned 404.

CI and existing base failures

The required Python tests check is green. At the completed qualification snapshot: 103 passed, 15 failed, 3 skipped, 0 pending. The 15 failed checks are existing compatibility failures, not new documentation regressions. Every remaining failure was compared against a completed dev run at 2c3a6e8700a0e76b6c8c0c6b820017cffe0c4804:

Failed checks Matching base run Matching failure
Google GenAI SDK matrix (8) 42469073559 Generation modality cost-input retention tests 57/59
Legacy Google SDK (2) 42469075799 Directional output-hiding / metadata OTLP assertions
OpenAI SDK 7.47.0 42469078088 Five existing openai.test.ts snapshots
Vercel gateway / OpenAI 5.23.2 42469080353 The same five OpenAI snapshots
LangChain core 1.3.2 42469085152 README contract expects project name in service.name, gets unknown_service:/usr/local/bin/node
Codex SDK + integration aggregate (2) 42469053220 contract/otlp_wire.mjs:57 expects session.id=thread-contract-1

Transient GitHub HTTP 504 download failures were retried. VoltAgent now passes; the integration retry progressed to the Codex assertion, which was then independently reproduced by retrying the base integration workflow. No unrelated assertions, snapshots, workflows or runtime behavior were changed to hide these failures.

This remains a draft as requested. Engineering qualification uses the task's explicit allowance for proven pre-existing non-required CI failures; it is not a claim that all checks are green. A human approval is still required before merge.

Reproduction entry points:

  • From typescript/packages/traceai_auto: node scripts/generate-readme.cjs --check and pnpm exec jest --runInBand test/docs.test.ts.
  • In E2B, install the documented packages and test bundlers in a disposable consumer, then run node <checkout>/typescript/packages/traceai_auto/test/docs-consumer.cjs <consumer> <receipts-directory>. This writes fixtures and replaces that disposable consumer's dist/ directory.
  • Workspace build: pnpm --filter @traceai/auto... --filter @traceai/vercel... install --frozen-lockfile --ignore-scripts, pnpm run build:deps, build @traceai/genkit and @traceai/vercel, then build @traceai/auto; pack core, semantic conventions, OpenAI and auto into a separate clean consumer.

Receipt root on the task owner's machine: /Volumes/nik-ext-1/tmp/traceai-release-20261104/e2b/runs/. Each run records stdout/stderr, source, exit status and sandbox deletion verification. Consumer assertions are committed in this PR; no private credentials are needed to reproduce them.

Scope and rollout

Documentation, generated coverage and documentation tests only. No runtime source, package versions, dependencies, lockfiles, default flags or APIs changed. No migration or deployment is needed; revert the documentation commits to roll back. The published npm README will only change on a later authorized package publication; this PR does not publish anything.

The docs-site clone has no dedicated @traceai/auto page. Its existing tracing/integration pages describe manual instrumentors rather than this process preload. That separate docs-site follow-up is intentionally not included here. Existing SDK guide screenshots are not presented as an auto-instrumentation demo; no screenshot was fabricated.

No Linear ticket was supplied for this documentation follow-up; none has been invented. Repository review/approval requirements still apply. No merge, admin override or package publication is requested by this PR.

Follow-up commit (d85da9c)

  • Real trace screenshot. "What you get" opens with the README's own OpenAI example as it appears in Future AGI: a real run of @traceai/auto 0.1.0, OpenAI SDK 7 and openai/gpt-4.1-mini through OpenRouter. It shows the model, provider, 13/3/16 tokens, cost, 885 ms, and the request and response. The trace went from an E2B run to a loopback collector and was forwarded to Cloud with process identity stripped, matching fi-core 1.2.1. No placeholder imagery.
  • No version pins. The installs are now npm install @traceai/auto and npm install openai. The quick start was verified in E2B against @traceai/auto from npm with OpenAI SDK 5, 6 and 7.53.0. Each of --import, NODE_OPTIONS and CommonJS --require gave one LLM span with output and usage. @traceai/openai hooks ^4–^7.
  • fi-core 1.2.1 is now on npm (chore(release): @traceai/fi-core 1.2.1 (process identity privacy fix) #621), so a fresh install no longer exports process command-line arguments or the OS user.

🤖 Generated with Claude Code

nik13 and others added 3 commits November 8, 2026 01:50
Replace the implementation-led README with quick start, source-derived coverage, privacy defaults, deployment recipes and focused references. Preserve capture details and correct custom-exporter authentication and wire-transport claims. Add documentation regression checks and an executable consumer recipe harness.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
- "What you get" opens with a screenshot of the README's own OpenAI
  example as it appears in Future AGI: a real run of @traceai/auto 0.1.0
  with OpenAI SDK 7 and openai/gpt-4.1-mini through OpenRouter (model,
  provider, 13/3/16 tokens, cost, 885 ms, request and response).
- `npm install @traceai/auto` and `npm install openai` instead of pinned
  versions. The quick start was verified in E2B with OpenAI SDK 5, 6
  and 7.53.0 through --import, NODE_OPTIONS and CommonJS --require (one
  LLM span with output and usage each); @traceai/openai hooks ^4-^7.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@nik13
nik13 marked this pull request as ready for review November 8, 2026 11:15
@nik13
nik13 merged commit 8744cef into dev Nov 8, 2026
118 of 135 checks passed
@nik13
nik13 deleted the docs/auto-readme branch November 8, 2026 11:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant