Extract @codeburn/core: one engine, CLI as first customer (tracking)

[iamtoruk]

Tracking issue for the @codeburn/core extraction (accepted direction of RFC #796, amended). Work happens on the long-lived integration branch feat/core-extraction; phase PRs target that branch, never main. Main merges once, when Phases 0-7 plus the dual-registry bridge are complete and green.

End state

One implementation: @codeburn/core owns deterministic decoding -> content-minimized observations -> pure detection. The CLI is Core's first customer: it keeps all side effects (discovery, SQLite, caches, pricing, aggregation, UI, apply) and delegates every parse/detect call. No parser or detector exists in both places once migrated.

Decisions (final)

• D1 Fingerprint privacy key: required, caller-supplied; the CLI auto-generates and stores a per-install key.
• D2 Classification stays host-side (D2-A), scope explicitly includes the user-corrections scanner (workflow-insights) — no free text crosses the Core boundary.
• D3 Pricing stays fully host-side for the preview.
• D4 Detector redesign may re-baseline savings numbers, with documented diff + changelog entry.
• D5 Core findings are advisory (D5-A); the host re-derives concrete fix payloads; act/apply stays CLI-side.
• D6 Preview scope: Claude + Codex decoders only.
• D7 Previous-major retention promise deferred until a real external consumer exists.
• D8 Publish as 0.x at preview; 1.0 only after the provider long tail is migrated.

Gates (every phase PR)

1. Parity: byte-identical CLI report output on the golden fixture corpus vs the pre-migration path.
2. Perf (Phases 3/4/8): cold and warm parse within 5% of pre-migration on the corpus — the scanner and incremental-append machinery must survive extraction.
3. Full suite green; no working path deleted until its replacement passes both gates; every phase reversible (opt-in module or move-with-shim).

Phases

0. Pricing lift-out (no package split): decoders emit token counts + basis (measured/derived/estimated); a single host-side pricing pass produces costUSD/savings exactly as today. Removes calculateCost from all provider files + parser.ts. Determinism test with network stubbed.
1. Workspaces + mechanical move: packages/{core,cli} via git mv, no logic change.
2. Schema + invariants + harness: ObservationEnvelope, diagnostics, HMAC resource fingerprints, content-smuggling fixture, import-smoke harness. Rich-capture fields (loc deltas, interruptions, branch) are first-class in the observation schema from day one.
3. Claude decoder carve-out from parser.ts (parity + perf gated).
4. Codex decoder + explicit serializable incremental state (round-trip resume test).
5. Detector redesign over fingerprints (evidence, confidence, algorithmVersion).
6. Classifier + corrections relocation per D2-A.
7. act/apply reconciliation per D5-A.
8. Dual-registry bridge + long tail (~35 providers, one PR each) — bridge lands on the branch; tail migrates after the main merge as ordinary PRs.
9. Publish 0.x preview (npm pack installable without CLI/native deps; import-smoke + content-smuggling green in a scratch install; export-coverage test keeps the CLI a real caller of every export).

Workflow

• Weekly main -> branch syncs (Phase 0 touches ~30 provider files; keeping the conflict window short matters).
• Every phase PR gets maintainer review before landing on the branch.
• Full plan with test strategy, risk register, and effort estimates is maintained internally; this issue tracks phase status.

Phase status: [x] 0 [x] 1 [x] 2 [x] 3 [x] 4 [x] 5 [x] 6 [x] 7 [x] 8-bridge [ ] main merge [ ] 9

[iamtoruk]

Phase 0 complete on feat/core-extraction (PRs #810-#819, 10 landed). Every provider decoder now emits token counts + costBasis; the single host-side pricing pass (src/pricing-pass.ts) computes cost before any cache write. Two add-only seam fields emerged from real shapes: pricingModel (display model != priced model: antigravity, cursor) and fallbackCostUSD (table-preferred with provider-figure fallback: codebuff, session-message, sqlite-session-parser). Byte-identical throughout: zero parse-version or cache-version bumps; full suite green at every step.

Documented residuals (all in-code): codex.ts x2 (results-cache persists decoder output verbatim — owned by Phase 4), copilot shutdown site (prices output=0 by design), mistral-vibe allocation branch. parser.ts Claude sites are Phase 3 scope by design.

Next: Phase 1 (npm workspaces + mechanical packages/cli move).

[iamtoruk]

Phase 1 complete (#820): npm workspaces, CLI moved wholesale to packages/cli via 334 history-preserving renames, @codeburn/core typed skeleton in place and depended on. Byte-identical CLI behavior verified (--help diff, payload keys, stage-cli closure superset); 2252 tests green from the workspace root; the appx workflow passed its first windows-latest run on the new layout. One commit, single-revert reversible. Next: Phase 2 — Core schema, invariants, conformance harness, fingerprints.

[iamtoruk]

Phase 2 complete (#821): @codeburn/core foundation — strict 0.1.0 ObservationEnvelope (structurally content-free, rich-capture numerics first-class), versioned decode/detect contracts, domain-separated HMAC fingerprints (required privacy key), diagnostics, drift-tested JSON Schemas, and both guardrail harnesses (import-smoke with non-vacuity check; content-smuggling). zod-only runtime. 91 core tests; root suite untouched at 2252. Next: Phase 3 — Claude decoder carve-out behind byte-parity + perf (±5%) gates.