Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion .oxfmtrc.jsonc
Original file line number Diff line number Diff line change
Expand Up @@ -9,5 +9,15 @@
// migration-snapshot diff.
// lib/llm and lib/thinking-orbs are vendored upstream source in prettier style;
// reformatting them would turn every future upstream sync into a whole-file conflict.
"ignorePatterns": [".context", "deploy", "skills", "packages/db/drizzle", "lib/llm", "lib/thinking-orbs"],
// ai-registry/registry.json is a generated artifact copied verbatim from trace-capture
// (see its README); formatting it would make every resync a false diff.
"ignorePatterns": [
".context",
"deploy",
"skills",
"packages/db/drizzle",
"lib/llm",
"lib/thinking-orbs",
"packages/domain/src/ai-registry/registry.json",
],
}
108 changes: 108 additions & 0 deletions packages/domain/src/ai-registry/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
# AI vendor registry (generated artifact)

`registry.json` is the compiled **per-span AI/agent telemetry classification registry**: the data
that lets Maple look at a single OTLP span and decide _which AI framework emitted it_ and _what
session it belongs to_, without cross-span joins.

It contains:

- **`vendors`** — one entry per AI framework, each with **matchers** (every one carrying a unique
integer priority) and **`session_candidates`** describing which attribute keys carry a session/run
identity and how to validate them. Vendors also carry `decoy_keys` / `decoy_values` (keys that look
identifying but are not), `caveats`, and `variants`.
- **`unknown_tier`** — the fallback bucket for spans that are recognisably AI telemetry but match no
known vendor: `unknown:genai`, `unknown:openinference`, `unknown:other`.
- **`algebra`** — the predicate contract the matchers are written in: ops `present`, `eq`,
`key_prefix`, `value_prefix`, plus the attribute-canonicalization rule
(`AnyValue -> String`; bools as `true`/`false`, numbers as decimal strings, array/kvlist as JSON;
duplicate keys resolve first-occurrence-wins among registry-referenced keys). There is deliberately
**no negation, no conjunction, no event access and no JSON traversal** — the algebra is kept small
enough to compile to both SQL and Rust.
- **`session_state_enum`** — the ladder a span lands in (1 = vendor has no session-key rules … 6 =
resolved at session granularity), reduced as `max` over candidates with the hash taken from the
winning candidate, ties broken by candidate order.

## What is generated and what is not

`registry.json` is **generated — never hand-edit it.** Edits made here are lost on the next sync and,
worse, silently diverge from the wire-verified seeds that justify every matcher. Treat it as a build
output that happens to be checked in.

`README.md` and `UPSTREAM.json` are maple-side and hand-maintained — the upstream compiler emits
`registry.json` and nothing else.

## Provenance

Generated by `scripts/compile-registry.ts` in the companion repo **trace-capture**
(`https://github.com/MapleTechLabs/trace-capture` — private), compiled from the wire-verified
per-framework seeds at `frameworks/<name>/registry-seed.yaml`.

`UPSTREAM.json` next to this file is the pin: the trace-capture commit that last produced
`registry.json`, the sha256 of the artifact so drift is detectable without a diff, and the date it
was synced.

"Wire-verified" means each seed was derived from real OTLP captures of that framework, not from
reading its source or docs; the seed format and the review that produces it are specified in
trace-capture's `frameworks/REVIEW_IMPLEMENTATION.md` (§6 is the seed format itself).

## How to update

Updates are expected to be **rare** — a new framework, or a vendor changing its wire format.

1. In the trace-capture repo, edit or add `frameworks/<name>/registry-seed.yaml` following
`frameworks/REVIEW_IMPLEMENTATION.md` §6.
2. Run `bun scripts/compile-registry.ts` there to recompile `registry.json`.
3. Keep `bun scripts/verify-seed.ts --all` green there.
4. Copy the regenerated `registry.json` here **verbatim**, and update `UPSTREAM.json` with the new
trace-capture commit, sha256 and `syncedAt`.

Treat trace-capture as the source of truth; never patch the artifact in maple to fix a
classification bug.

## Who reads it in maple

Two implementations, with different jobs:

- **`apps/ingest` (Rust)** classifies every span at write time and stamps the `AiVendor` /
`AiSessionKey*` columns. This is the only path that classifies live traffic.
- **`packages/domain` (TypeScript)** does _not_ classify on the read path — dashboards and queries
read the stamped columns. It exists for the two jobs Rust cannot do: compiling the same rules to
ClickHouse SQL so a registry fix can re-derive vendors for spans **already on disk** (rollup
rebuilds cannot trust the column being fixed), and backing the differential test that holds the
Rust and SQL evaluators to the same answer span-for-span.

## Reading the vendor list

Two things in here regularly surprise people:

- **There is no `langgraph` vendor — LangGraph spans classify as `langchain`.** LangGraph is
instrumented by LangChain's own tracer, so an individual span carries no evidence separating the
two. Keeping both as vendors would mean two entries matching the same span with one winning
arbitrarily, so the compiler merges them into the wider one; the entry keeps
`renamed_from: "langgraph"`.
- **A single trace can carry spans from several vendors, and usually does.** Classification is
per span, never per trace. `openinference-openai` is the clearest case: that instrumentor carries
the token/model/prompt payload for _any_ framework driving the OpenAI SDK, so in the captures
roughly half the spans of a CrewAI trace classify as `openinference-openai` and the rest as
`crewai`. Anything reading vendors has to aggregate over a trace's spans rather than expect one
label per trace.

Those are decisions D2 and D3 in the artifact's own `decisions` array. The other two are internal to
the compiler and only matter if you are changing it in trace-capture: **D1** adds a fourth predicate
operator (`value_prefix`, allowed only on scope/span-name pseudo-keys), **D4** assigns priorities in
bands so first-match-wins is deterministic.

## Validation

The **capture corpus is deliberately not vendored.** trace-capture holds the OTLP captures every
seed was derived from; they are large, contain raw third-party payloads, and are only useful next to
the replay tooling that reads them.

Consequently:

- **Corpus-replay verification against seed goldens is an on-demand local gate**, run from the
trace-capture repo — not from maple, and not in CI.
- **Maple's CI relies on synthetic differential and property tests** over this artifact: the TS and
Rust evaluators must agree span-for-span, and the algebra's invariants
(unique priorities, band ordering, `value_prefix` pseudo-key restriction, session-state reduction)
are asserted directly against `registry.json`.
10 changes: 10 additions & 0 deletions packages/domain/src/ai-registry/UPSTREAM.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"repo": "https://github.com/MapleTechLabs/trace-capture",
"ref": "main",
"sha": "cb29d8434a5919fd106278947ed3e262456407e5",
"generatedBy": "bun scripts/compile-registry.ts",
"compiledFrom": "frameworks/<name>/registry-seed.yaml (20 seeds + 1 synthesized vendor)",
"artifact": "registry.json",
"sha256": "226031079a9b273a3f799998ad8152c727a62f1c0a45593d1b884f52e2718595",
"syncedAt": "2026-08-11"
}
Loading
Loading