Skip to content

feat(agents): one-command coding-agent tracing (fi-agent CLI, 14 adapters, VS Code extension) - #609

Merged
nik13 merged 71 commits into
devfrom
feat/agents-integration
Nov 8, 2026
Merged

nik13 merged 71 commits into
devfrom
feat/agents-integration

Conversation

@nik13

@nik13 nik13 commented Nov 8, 2026

Copy link
Copy Markdown
Contributor

Summary

One-command tracing for coding agents: npx @traceai/agents setup <agent> --project <name> installs a maintained plugin, validates Future AGI keys, saves them owner-only, and exports one trace per prompt with model generations, token usage (counted once), tools, subagents and sessions. Content capture is on by default with one switch (--no-capture-content / FI_AGENT_CAPTURE_CONTENT=false) to turn it off.

New packages (first release 0.1.0)

  • @traceai/agents: the fi-agent CLI (setup, status, doctor, enable, disable, update, uninstall, run, open; --user-only, --scope, --project-dir).
  • @traceai/agent-core: shared tracer, privacy and redaction, delivery, lifecycle, and a conformance kit that every adapter passes (14 adapter paths, 422 checks including mutation guards).
  • Plugins: @traceai/claude-code, @traceai/codex-plugin, @traceai/cursor-plugin (IDE + CLI), @traceai/copilot (CLI + VS Code), @traceai/vscode-extension, @traceai/antigravity-plugin (CLI + macOS desktop app), @traceai/grok-plugin, @traceai/opencode-plugin, @traceai/opencode-v1-plugin, @traceai/goose, @traceai/hermes-agent, @traceai/openclaw, @traceai/pi-plugin, @traceai/oh-my-pi-plugin.
  • Held surfaces (Muse, Gemini CLI, Kiro, Augment, Devin Cascade, Cursor Cloud, Antigravity IDE profile) stay private/optional and report their qualification reason instead of installing.

Version bumps

Real-host qualification

Real agents on a real Mac and in E2B, read back from Future AGI Cloud: Claude Code, Codex, Cursor IDE + CLI, Copilot CLI, VS Code Copilot (1.143.1 and 1.145.2: setup, Restricted Mode, agent/ask turns, disable/enable without reload, content off, two-window relay takeover, uninstall restore), Antigravity desktop app 2.28.2, Grok Build, OpenCode, Hermes, Goose, OpenClaw, Pi, Oh My Pi.

Merge from dev

Merges dev (zero-code release, @traceai/fi-core 1.2.0):

  • fi-core now merges OpenTelemetry's process detector into provider resources, so @traceai/agent-core strips host.* / process.* before export (agents never send local process identity; fi-core follow-up fi-core 1.2.0 exports process command-line arguments and OS user on every span #607).
  • fi-core masks credential-shaped keys with <redacted>; tests accept either marker while still requiring secret absence and retained public content.
  • fi-core's admission privacy pass omits a structured (JSON) value that agent-core had to shorten past 20,000 characters (<omitted: incomplete audio>). Nothing leaks; readable truncation of oversized JSON is agent-core: oversized structured content is omitted after truncation (fi-core 1.2.0) #608 (three attempts were rejected in adversarial review because cutting before scrubbing can split a secret).
  • Cursor: bounded concurrent tombstone scans keep the terminal hook inside its deadline (stop hook 4.9 s → 1.0 s with 256 stored turns).

Validation (E2B)

On head f024eb6 (run release-gate-f024): frozen-lockfile install and builds, all agent suites (1,256 tests, 0 failed, 28 conditional skips), conformance kit for all 14 adapter paths, Antigravity 5x repeat, type consumer, package pack and unchanged lockfile. Reviewer privacy probes: environment-credential echo masked, FI_HIDE_EMBEDDING_VECTORS and FI_PII_REDACTION effective, 4/4 boundary assertions, and the 57-case adversarial matrix identical to the merged-dev baseline. VS Code extension: real VS Code under xvfb, 10/10 consecutive passes (earlier run on the same extension sources). Release plan dry run lists the new/bumped packages above.

Follow-ups

🤖 Generated with Claude Code

nik13 and others added 30 commits November 6, 2026 01:27
Add shared private-profile configuration, authenticated empty-OTLP validation, lifecycle/status APIs, persistent disable generations, and a unified fi-agent CLI framework.

Use a private FI provider with bounded finalized OTLP delivery, source-bound identities, explicit usage ownership and content-on capture with strict privacy opt-outs. Preserve the separate preview contracts and their backend/content holds. Keep adapter registration fail-closed for the downstream host implementation lanes.

Validated in E2B: 38 core tests, 3 CLI tests, 70 release-script tests, release inventory/audit checks, and clean-cache packed CJS/ESM/CLI installation. No Cloud/model calls, pushes or publication.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Adds the attribute names Future AGI already reads but the conventions did
not define, so coding-agent traces show cost, TTFT, tool names and token
totals:

- gen_ai.cost.total / gen_ai.cost.input / gen_ai.cost.output (read by
  fi-collector pkg/adapter costTotalKeys and tracer/utils/otel.py).
- gen_ai.server.time_to_first_token. Future AGI's dashboard reads it as
  milliseconds; the OTel GenAI spec and the agentcc gateway use seconds.
  The mismatch is documented on the constant as a backend follow-up.
- fi.usage.scope / fi.usage.contribution / fi.cost.kind, which the trace
  detail totals already honour, so turn-level usage counts exactly once.
- The fi.agent.* provenance group used by the coding-agent core
  (fidelity, timing, outcome, usage plan/additivity, cost source, gaps,
  transport, source call id, non-additive summaries, tags, events).
- SpanAttributes.TOOL_CALL_ID alias in Python.

Same keys in TypeScript, Python, Java, C# and the five Go modules, each
with a wire-contract test. No existing constant is renamed or removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Consolidate all lane core contracts on the reviewed Cursor foundation. Preserve scope-aware approval, protected locking, bounded policy-fenced delivery and canonical single-owner usage. Retain all fork regressions with explicit compatibility expectations.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Import the Goose adapter without shared-core or release metadata changes. Preserve recoverable state writes and structured redaction, bind diagnostics to scoped delivery receipts, and qualify cached input usage and writer recovery against real Goose.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Import the CLI adapter and gated IDE profile without changing the frozen core. Use scoped delivery receipts and shared structured redaction. Correlate prompt boundaries to native user records, retain delayed subagent continuations, and retry crashed event leases from the finite exit observer. Add real-binary delayed-child and fresh SIGKILL lock regressions.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Port the OpenCode V2 plugin and CLI adapter without modifying the frozen core. Link shared native dispatches across participating prompt roots, preserve their final response, and keep generations and additive usage single-owned.\n\nDisable credential echo before displaying prompts, exercise the scheduling race in a PTY, reuse canonical redaction before truncation, and emit native tool-call IDs.\n\nValidated 380 core, 40 plugin and 11 CLI tests plus the packed OpenCode 2.2.5 native scenarios in E2B (int-opencode-unified-r3).

Co-Authored-By: Claude Code <noreply@anthropic.com>
A setup operation advances the shared capture epoch, so status must not reuse an earlier export as evidence for a changed privacy policy.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Import the adapter-only OpenCode V1 lane and align project fallback diagnostics, pre-truncation redaction, source tool-call IDs and retry recovery with the frozen core. Add actual OTLP regression coverage for canonical usage and failure privacy.

E2B agents-int-opencode-v1-r2: 380 core, 16 CLI and 36 plugin tests passed; packed plugin loaded by real OpenCode 1.19.0 with loopback-only model and OTLP scenarios.

Co-Authored-By: Claude Code <noreply@anthropic.com>
…cture

Move the native Codex plugin and its CLI registration onto unified core 76a2427. Redact embedded serialized shell content before truncation, revalidate pending Stop snapshots, preserve structural events and usage under bounded journal pressure, and emit canonical tool IDs with observed hook timing.

Validate in E2B: 380 core tests, 38 Codex tests, 3 CLI tests, fresh packed consumers, and five real Codex 0.167.0 scenarios covering privacy, verbose turns, nested/reused children, resume, interrupt, fatal provider errors and disable.

Shared CLI carryovers for assembly: current-directory status/doctor, masked credential replacement after rejection, and TRUST_REQUIRED exit status. Core, root manifests and shared lockfile remain unchanged.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Preserve native CLI privacy, lifecycle and real-binary coverage; keep VS Code tracing qualification-gated. Align scoped receipts and semantic usage, tool and error outcomes with the frozen core. Include CLI setup/uninstall support and protected-project privacy regressions.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Preserve every coalesced Pi prompt with linked roots, count aggregate compaction usage once, honor native extension selection, and surface host failures. Upgrade unchanged setup-owned OMP registrations and qualify FIFO fallback and native installs against the frozen core.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Import the Claude hooks adapter and its CLI registration only. Scope terminal evidence per prompt, preserve late transcripts, diagnose disabled host hooks, and use unified scoped receipts and no-follow locks. Fence pending capture across setup changes and verify shared redaction and destination protection. Keep shared CLI setup-default and docs publication prerequisites explicit in validation.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Port the Grok adapter without changing frozen core or shared release metadata. Adopt reconstructed generations, canonical usage ownership, scoped receipts, pre-truncation redaction and error-context privacy; cover host updates and broken-runtime uninstall.

Validated in E2B: 20 Grok tests, 380 core tests, 3 CLI tests, and packed Grok 1.0.77 native scenarios producing 4 roots and 15 spans.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Preserve late-transcript generations and finalized usage ownership while merging the separate Cursor profiles. Reuse the core secure leases, keep Cloud capture lightweight and environment-only, redact before truncation, and remove unchanged owned hook files. Add integration, privacy, lifecycle and concurrency regressions without changing the frozen core or shared CLI implementation.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Preserve private qualification gates, merge Kiro CLI and IDE packaging, adopt bounded event leases and scoped receipts, pin the Kiro IDE interpreter, and use reconstructed Cascade generations. Add grouped E2B qualification and regressions for tool IDs, concurrent recovery, bounded context, and external-volume fallback.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Import the Hermes companion and Python runtime without changing the frozen core. Align lifecycle, destination approval, diagnostics and compatibility tests with the unified implementation.

Reject malformed project transport layers with safe fallback, retain bounded native delegation ancestry across automatic continuations, and preserve unrecovered provider failures on terminal roots. Qualify packed packages and real interactive Hermes against loopback model and OTLP services in E2B.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Import the OpenClaw adapter without shared-core or release metadata changes. Normalize canonical cache and tool-call semantics, preserve early-failure roots, scope delivery receipts, align host discovery and safe diagnostics, and cover unified privacy behavior.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Merge the eight assigned adapter groups, retain held surfaces, cover all
TypeScript packages in ordered builds, and add assembly regressions.
Use explicit epoch timestamps in the core ownership test. The lockfile
was regenerated once in E2B and copied back byte-for-byte.

Release remains held: the unified CLI imports private adapter packages
and its release dependency closure must be separated before publication.

Co-Authored-By: Claude Code <noreply@anthropic.com>
…aults

Core:
- Redact the whole YAML block scalar under a sensitive key (literal and
  folded, chomping/indent indicators, raw or JSON-escaped newlines). The
  Pi group reproduced `access_token: |` leaking its value through OTLP.
- Load protected file locks without evaluating deprecated fs getters
  (silent on Node 24), with a CJS/ESM import-silence test.

CLI:
- Every command resolves the current directory as the project by
  default, like the hosts' own workspace discovery; --user-only skips
  the repository layer; --project-dir is made absolute.
- Environment-only Cloud surfaces default to project scope and never
  prompt for or read saved credentials.
- run/update follow the scope of the resolved profile instead of
  assuming user scope.
- Hidden key entry disables echo and attaches every handler before the
  prompt is shown, and restores the terminal on end/error/exit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	typescript/packages/traceai_agents/package.json
#	typescript/packages/traceai_agents/src/cli.ts
#	typescript/packages/traceai_agents/src/registry.ts
#	typescript/packages/traceai_agents/test/cli.test.cjs
# Conflicts:
#	typescript/packages/traceai_agents/package.json
#	typescript/packages/traceai_agents/src/registry.ts
# Conflicts:
#	typescript/packages/traceai_agents/package.json
#	typescript/packages/traceai_agents/src/registry.ts
nik13 and others added 25 commits November 6, 2026 19:40
Label completed native roots, preserve additive root-only usage, and recover missing terminal parents at stream EOF with explicit incomplete provenance. Keep bounded structural metadata and native trace IDs without retaining prompt content. Exercise EOF in the shared driver and add native, privacy, nested-usage and launcher regressions. Assert real-binary outcomes and auth rejection fencing without depending on batch timing.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Use interrupted for SIGINT/SIGTERM and incomplete for abrupt SIGKILL exits, preserving SIGKILL through the local launcher. Omit mixed serialized history under output/error privacy switches so earlier assistant replies and tool failures cannot leak through model input. Add signal, env-alias, malformed-history and real-host resume regressions.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Redact recognizable credentials and structured secret fields before truncation. Apply output and error policies to request history and constrain private exit reasons to known categories. Match the native session context in the shared conformance driver without weakening workspace-bound delivery evidence. Add privacy regressions and packed native status/doctor and credential-redaction assertions.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Wait for late SQLite responses and tool results within the existing hook budget, without holding a read transaction or inflating Stop duration. Recheck collection policy before export and preserve explicit user cancellation over incomplete or failed tools.

E2B uni-conform-goose-r3-final: 26 conformance checks, 24 adapter tests, 419 core tests, 51 CLI tests and the packed Goose 1.54.0 native suite passed.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Use route-scoped activation evidence to clear stale reload diagnostics. Find owned native registrations independently of configure-only profile scope, while preserving manual and edited registrations.

Add adapter regressions and real-host lifecycle assertions; align the acceptance harness with shared config cleanup and credential prompts without suppressing shared CLI failures.

Co-Authored-By: Claude Code <noreply@anthropic.com>
A real run of Antigravity 2.28.2 on macOS showed the app runs our
plugin's hooks with the CLI's payload shape, but the plugin dropped every
event: it only accepted ~/.gemini/antigravity-cli/ paths and setup was
Linux-only.

- Accept the app surface (~/.gemini/antigravity/) next to the CLI, record
  the surface on the trace, and keep IDE-only paths gated.
- The transcript reader accepts the app's transcript with the same
  ownership, symlink and traversal checks; PLANNER_RESPONSE rows become
  generations with their tokens counted once.
- Setup and detection support macOS.
- Uninstall removes the empty plugin_data directory agy leaves behind.
- Golden test from the real (sanitized) app payloads and transcript:
  root -> 2 invocations -> run_command tool + 2 generations
  (20,437 input / 64 output tokens), outcome success.

Review (E2B ag-app-review-r2-0348): plugin 70 pass (8 gated), real agy CLI
8/8, core 405/405, CLI 51/51, packed consumer 16/16; adversarial privacy
and path probes 9/9.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Preserve sessionEnd model and validated usage, and let core-scoped delivery evidence determine activation. Enable the qualified CLI shared-hook route while retaining the separate print-stream hold.

Canonicalize Cursor CLI and IDE lifecycle state so setup, disable, delivery receipts and uninstall control the same native hooks. Add terminal, privacy-fencing, activation and shared-ownership regressions.

Validated in E2B: 54 conformance checks; 420 core, 110 Cursor and 51 CLI tests; clean-package goldens and real Cursor binary inspection.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Use the shared sensitive-key policy for all relay attributes with numeric-only semantic counter exceptions. Redact complete YAML plain scalars while preserving CRLF boundaries. Treat tool output as opaque content and honor adapter outcomes. Preserve intervening shared-settings edits across repeated setup and uninstall.

E2B uni-fix-full-r2-1107: frozen install/build, 1258 tests passed, 27 skipped, zero failures; 369 conformance checks across 14 adapter paths; five Antigravity repeats; type consumer and package pack. All four defect suites fail on baseline and pass on fixed sources (17 focused tests).

Co-Authored-By: Claude Code <noreply@anthropic.com>
Merges feat/agents-antigravity-app (95aa348): the Antigravity 2.28.2 app runs
the agy plugin hooks, so the antigravity adapter accepts the app surface and
macOS. Qualified on a real Mac with a Gemini turn read back from Cloud.
Keeps the conformance privacy tests and the app's cache-read detail test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
VS Code opens a newly cloned folder in Restricted Mode. The extension declared
untrustedWorkspaces unsupported and returned early when the workspace was not
trusted, so on a real Mac setup reported REGISTERED while no relay ran and
Copilot's spans were refused. The Electron test hid this by passing
--disable-workspace-trust.

The relay reads only Future AGI's user-level installation record, never
workspace files or settings, so it now runs regardless of workspace trust and
in virtual workspaces. A transient lifecycle error during a tick no longer
closes the listener; only a changed or removed installation stops it.
A manifest test checks the shipped VSIX, because extension test hosts always
trust their workspace.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Keep native Copilot startup settings stable across relay disable/enable, suppress Cursor sessions with no observed work, and enforce conformance exit codes, privacy conservation, failed-tool outcomes, and per-generation attribution with mutation guards.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Qualified on a real Mac with VS Code 1.143.1 and 1.145.2 (Default profile):
setup keeps keys out of VS Code settings; the extension relay runs in
Restricted Mode; agent and ask turns read back from Cloud; disable/enable
leave settings byte-identical without a reload prompt and the disabled turn
never reaches Cloud; --no-capture-content removes prompt and response text;
a second window takes over the relay; uninstall restores settings exactly.
copilot-vscode now loads like other qualified adapters.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The turn root's output was the final generation's serialized content-block
array, so Future AGI displayed raw JSON where other agents show the reply.
The root now carries the final response text; generations keep their
structured content.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in the zero-code release (fi-core 1.2.0, @traceai/auto) and #605.
- build:consumers keeps every agent package and adds @traceai/auto; the
  lockfile is regenerated from the merged manifests.
- @traceai/fi-semantic-conventions 1.1.0: the published 1.0.0 lacks the
  coding-agent attributes from #605 that agent-core imports.
- @traceai/pi-coding-agent 0.2.0: adds the ./content export the Pi plugins use.
- agent-core strips host.*/process.* resource attributes before export:
  fi-core 1.2.0 merges OpenTelemetry's process detector (command arguments,
  OS user) into every provider (fi-core follow-up: #607).
- Tests accept fi-core's <redacted> marker alongside [REDACTED].

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Read independent pending metadata under eight concurrent protected leases and reuse the hook's scanned IDs instead of repeating a full scan in the runtime. Claims still revalidate the current envelope and policy before export; stale handed-off IDs cannot replay claimed delivery.

Co-Authored-By: Claude Code <noreply@anthropic.com>
fi-core 1.2.0's admission privacy pass omits a truncated JSON prefix it
cannot classify (as '<omitted: incomplete audio>') instead of keeping the
truncated text. Secrets never reach export either way. Tests now accept
that omission for structured values over the 20,000-character content limit
and fi-core's '<redacted>' marker, while still requiring secret absence and
retained public content wherever the value is kept. Readable truncation of
oversized JSON is a tracked follow-up; three attempts to shorten before
fi-core's pass were rejected in review because cutting before scrubbing can
split a secret.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Hermes Python package and its plugin were bumped to 0.1.1 without the
release-please manifest, which fails the release plan's manifest check. Also
make the FIFO project-config test assert that PROJECT_CONFIG_IGNORED is
reported (and print the issues) instead of requiring it to be the only
issue, which differs on GitHub's runner.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GitHub runners set XDG_CONFIG_HOME, which the child inherited, so it read
enablement from the runner's home instead of the test configuration and
built no tracer provider. The child now inherits only non-XDG/FI/Hermes
variables plus the test's own configuration.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@nik13
nik13 merged commit 9104fa9 into dev Nov 8, 2026
118 of 150 checks passed
@nik13
nik13 deleted the feat/agents-integration branch November 8, 2026 07:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant