diff --git a/.claude/board/INTEGRATION_PLANS.md b/.claude/board/INTEGRATION_PLANS.md index 9d17dbc69..994846357 100644 --- a/.claude/board/INTEGRATION_PLANS.md +++ b/.claude/board/INTEGRATION_PLANS.md @@ -1,3 +1,13 @@ +## 2026-11-02 — D-TGL-PUB: G10 × G11 atomic publication lab spec → `.claude/plans/g10-g11-atomic-publication-lab-v1.md` + +**Status:** SPEC, lab only; nothing implemented. **Board:** [STATUS_BOARD](STATUS_BOARD.md) D-TGL-PUB; [entry](entries/2026-11-02-d-tgl-release-fence-and-g10-g11-boundaries.md). +**Confidence:** boundaries read in code (dt64 `route`/`RouteLog`, `persist_sink.rs`); the release-profile fence result was measured. + +## 2026-11-02 — D-TGL: the GitHub-analogue Thought-Git lifecycle as a shared contract → `.claude/plans/thought-git-lifecycle-contract-v1.md` + +**Status:** CONTRACT, docs only; Codegraph lab F-LAND / F-DISCARD shipped (crate-private). **Board:** [STATUS_BOARD](STATUS_BOARD.md) D-TGL-0 … D-TGL-G12, D-TGL-EXO148; [decisions + lab](entries/2026-11-02-p4-operator-decisions-lifecycle-lab.md); [P4 review](entries/2026-11-02-p4-thought-git-codegraph-dt64-review.md). +**Confidence:** lab measured (tests + 2 disables); production gates G9–G12 and EXO-148 open. + ## 2026-11-02 — D-DT64-AUTH: DT64 as the one method authority, census + ratchet → `.claude/plans/d-dt64-method-authority-census-v1.md` **Status:** PROPOSED, plan only (operator decision P3 of the architecture review). **Board:** [STATUS_BOARD](STATUS_BOARD.md) D-DT64-AUTH; [review](entries/2026-11-02-codegraph-dr3-dt64-ruff-architecture-review.md); [decisions](entries/2026-11-02-codegraph-review-operator-decisions.md). Same day: `d-codegraph-0-binary-lanes-v1.md` gained Amendment A (P2: provenance header, `(artifact, FunDeclId)`, `INVOKES_WORD`). diff --git a/.claude/board/LATEST_STATE.md b/.claude/board/LATEST_STATE.md index beaa9c50f..83cb4cc88 100644 --- a/.claude/board/LATEST_STATE.md +++ b/.claude/board/LATEST_STATE.md @@ -1,3 +1,13 @@ +## 2026-11-02 — main after #1922 and #1923 (merged `2e19a5adc`) + +- `lance-graph-codegraph` opens stores of any size on a multi-tile xlog cube. + - `refresh_crate` is atomic: a refused refresh changes nothing. + - The codegraph suites run in `rust-test.yml`. +- xlog: wide closure, multiset delta log with merge on read, `Cube::stage` / `commit`; binds survive an extending rebase. +- dt64 `dr`: a versioned append-only tile index; a frame reads its own layout; `map_at(RouteVersion)`. +- egglog pinned to one commit in `egglog-r2il-probe`, `dr-math-binding-probe`, `xlog-relcomp-probe`. +- Board: architecture review and decisions, P4 review, plans for binary lanes (amended) and the DT64 authority census. + ## 2026-11-02 — main after #1920 and #1921 (merged `4ab93bf3e`), ruff #130–#132 - New workspace-excluded crate `lance-graph-codegraph` (#1921, D-CODEGRAPH-0 slice C): reads the `ruff_charon` rows v2 artifacts by column name and answers callers, callees, references and impact as confirmed / possible / unknown masks over one xlog cube; diagnostics dead function, unresolved call, dangling call, unused import (NOT_RUN). Limits surfaced as typed refusals: closure > 64 positions, tile > 256 symbols. diff --git a/.claude/board/PR_ARC_INVENTORY.md b/.claude/board/PR_ARC_INVENTORY.md index 5a4403500..1a3bd5ccd 100644 --- a/.claude/board/PR_ARC_INVENTORY.md +++ b/.claude/board/PR_ARC_INVENTORY.md @@ -1,3 +1,37 @@ +## 2026-11-02 — lance-graph PR #1923 (merged `2e19a5adc`, branch `claude/clone-repositories-71a5sw`) — D-CODEGRAPH-0 wide closure, atomic refresh, versioned DR index (D-DYNBAG-4) + +- **Added:** + - dt64 `dr`: `MapTable::locate`, `LocalMap::extends` / `view`, `Axis::for_survivors`, versioned append-only tile index (`map(frame)`, `map_at(RouteVersion)`, shared `VersionClock`). + - xlog: + - `closure_wide` (multi-word rows, block presence); + - `Cube::new_tiled`; + - multiset delta log with merge on read; + - `refresh_closure`, `refold`, `rebase`; + - `Cube::stage` / `stage_closure` / `commit`; + - `Node::Bind` keeps its global id. + - codegraph: multi-tile stores, interned lanes, digest skip, atomic `refresh_crate` (staged interner, positions, tile map, batch, closures). + - CI line for the codegraph suites; egglog pinned to one commit in three crates. +- **Locked:** + - positions never renumber; + - a refused refresh changes nothing; + - an old frame replays its own layout; + - a bind to an unmapped id is placed by an extending rebase or commit. +- **Deferred:** + - binary lanes (D-CODEGRAPH-0-BIN, amended with PROV / `(artifact, FunDeclId)` / `INVOKES_WORD`); + - Quack counting; + - prefetch; + - cross-tile ℕ-compose; + - the DT64 authority census (D-DT64-AUTH, plan only). +- **Docs:** + - architecture review (Codegraph × DR³ × DT64 × Ruff) and operator decisions; + - P4 review (Thought-Git × Codegraph × DT64); + - plans `d-codegraph-0-wide-closure-v1`, `d-dynbag-4-dr-versioned-mapping-v1`, `d-codegraph-0-binary-lanes-v1`, `d-dt64-method-authority-census-v1`. +- **Confidence:** + - suites and clippy green; atomic-refresh and bind falsifiers with 7 disable runs red; full CI green on the merged head. + - mask-risc: impact 286 ns, callers 29 µs, references 45 µs. + - An unchanged refresh takes 1.5–1.7 ms (failed gate, TSV read). +- **Stranded:** two commits made after the merge point (prepare/commit split with F-LAND / F-DISCARD; the D-TGL lifecycle contract). They are re-offered in a follow-up PR from the same branch, with `main` merged in (no history rewrite). + ## 2026-11-02 — lance-graph PR #1921 (merged `4ab93bf3e`, branch `claude/clone-repositories-71a5sw`) — D-CODEGRAPH-0 slice C: `lance-graph-codegraph` - **Added:** `crates/lance-graph-codegraph` (workspace-excluded): rows v2 reader keyed by schema names; one xlog cube (CALLS, NODE, MAY, IMPL, RCONF, RPOSS) and position masks; `callers` / `callees` / `references` / `impact` graded confirmed / possible / unknown; diagnostics dead function, unresolved call, dangling call, unused import NOT_RUN; store fixtures rev1, rev2, mixed, fnref; `examples/timing.rs`. diff --git a/.claude/board/STATUS_BOARD.md b/.claude/board/STATUS_BOARD.md index 9c42384e9..8880b04e2 100644 --- a/.claude/board/STATUS_BOARD.md +++ b/.claude/board/STATUS_BOARD.md @@ -41,7 +41,7 @@ Plan: `.claude/plans/d-dynbag-holographic-bags-v1.md`. Plan only. | D-DYNBAG-1 | ℕ-compose probe: `matmul_i8_to_i32` over two DR64² tiles for `p(X, Z)`; B projection = `relcomp::compose` | In PR: exact vs both oracles (300 cases), projection = compose; MEASURED ~21 µs/tile on the AVX-512 VNNI tier (no AMX) vs 50 ns–2.4 µs for B (plan §7.6) | D-DYNBAG-0 | | D-DYNBAG-2 | semiring parameter K (B, ℕ) in relalg `Rel` + Quack surface; typed refusal per (op, K) | Queued | D-DYNBAG-1 | | D-DYNBAG-3 | missing ℕ words backend-first (i64 masked select, min, truncated minus, compare-to-mask, masked sum; maybe bit-packed product) | Queued | D-DYNBAG-2 | -| D-DYNBAG-4 | DR{dynamic}: width from survivor count, multi-tile map, cross-tile fold/compose | In PR (partial): `Axis::for_survivors` (width rule), `MapTable::locate` (global → (tile, local)), versioned append-only index (`map(frame)` replays the frame's layout, `map_at(RouteVersion)`, shared `VersionClock`), xlog `Cube::new_tiled` + `closure_wide` (cross-tile closure). Plans `d-codegraph-0-wide-closure-v1.md`, `d-dynbag-4-dr-versioned-mapping-v1.md`. Open: cross-tile ℕ-compose (§7.4) | DT64 `dr` owner | +| D-DYNBAG-4 | DR{dynamic}: width from survivor count, multi-tile map, cross-tile fold/compose | Shipped (partial, #1923 merged `2e19a5adc`): `Axis::for_survivors` (width rule), `MapTable::locate` (global → (tile, local)), versioned append-only index (`map(frame)` replays the frame's layout, `map_at(RouteVersion)`, shared `VersionClock`), xlog `Cube::new_tiled` + `closure_wide` (cross-tile closure). Plans `d-codegraph-0-wide-closure-v1.md`, `d-dynbag-4-dr-versioned-mapping-v1.md`. Open: cross-tile ℕ-compose (§7.4) | DT64 `dr` owner | | D-DYNBAG-5 | `sql-quack-probe` modelled on `lance-graph-cypher-quack` | Queued | parser decision | | D-DYNBAG-6 | scoreboard vs DuckDB / PostgreSQL, ns units, losses reported | Queued | D-DYNBAG-5 | | D-DYNBAG-7 | second Datalog frontend (Scallop) through the same path | Queued | D-DYNBAG-2 | @@ -240,8 +240,17 @@ Plan: [d-codegraph-0-resident-codegraph-v1.md](../plans/d-codegraph-0-resident-c | **D-CODEGRAPH-0-A** | ruff `ruff_charon`: LLBC → Kythe facts with a witness status per call, for one real crate | SHIPPED: ruff #129 (merged `9913e90946`), lance-graph #1918 (merged `c11e4262e`): calls byte-identical to `llbc_calls.py` on the fixture and `mask-risc`; receipt conserved (16,149); Kythe `/xrefs` 3/3 call pins | rust-analyzer oracle (`sag-ra-probe GRAPH=1`) and rustc on `fixtures/codegraph`; `POSSIBLE` never counted as a confirmed caller | | **D-CODEGRAPH-0-B** | resident rows via the ruff sink; per-file digests; incremental invalidation | LANDED (offline form): ruff #130 (merged `95be2e480b`; offline store in `ruff_charon`; `RefinedTruthSink` not reused, typed to R2IL). Incremental == clean on the fixture, byte-identical; skipped dependent leaves exactly one dangling edge. Follow-up fixes in ruff #131 (open). Lance-graph SoA residency still open | clean import == incremental import | | **D-CODEGRAPH-0-C** | callers / callees / impact / diagnostics; tiled closure; body → R2IL design check | SHIPPED (fixture scale): ruff #132 (merged `ed8fd4194f`, rows v2) + lance-graph #1921 (merged `4ab93bf3e`), `crates/lance-graph-codegraph` (queries as confirmed / possible / unknown masks over the xlog cube; dead function, unresolved call, dangling call; unused import NOT_RUN). 27 tests, 12 disable runs red. Tiled closure deferred: refusals `ClosureWidth` (> 64) and `TooMany` (> 256) are surfaced as typed UNKNOWN | each guard disable-verified; unused import proven only without macros, traits, globs, re-exports | -| **D-CODEGRAPH-0-C2** | wide closure (multi-word rows, block presence), multi-tile cube, merge-on-read refresh, interned lanes | IN PR: lance-graph branch `claude/clone-repositories-71a5sw`. Impact over any size; mask-risc store (695 symbols, 3 tiles): impact 286 ns, callers 29 µs, references 45 µs; refresh of an unchanged crate 1.5–1.7 ms = FAILED gate (reading the 2.5 MB text file), fixed by D-CODEGRAPH-0-BIN. `refresh_crate` atomic (prepare → validate → commit: staged interner, positions, tile map, `Cube::stage`, closure; commit has no error path); codegraph suites now in `rust-test.yml` | narrow vs wide bit-identical ≤ 64; BFS reference on 694 positions; refresh == clean open; a refused refresh changes nothing (NeedsReopen, injected stage and closure refusals; no version issued); retry == the refresh that never failed, call-site multiplicity included; 7 disable runs red | +| **D-CODEGRAPH-0-C2** | wide closure (multi-word rows, block presence), multi-tile cube, merge-on-read refresh, interned lanes | SHIPPED: #1923 (merged `2e19a5adc`). Impact over any size; mask-risc store (695 symbols, 3 tiles): impact 286 ns, callers 29 µs, references 45 µs; refresh of an unchanged crate 1.5–1.7 ms = FAILED gate (reading the 2.5 MB text file), fixed by D-CODEGRAPH-0-BIN. `refresh_crate` atomic (prepare → validate → commit: staged interner, positions, tile map, `Cube::stage`, closure; commit has no error path); codegraph suites now in `rust-test.yml` | narrow vs wide bit-identical ≤ 64; BFS reference on 694 positions; refresh == clean open; a refused refresh changes nothing (NeedsReopen, injected stage and closure refusals; no version issued); retry == the refresh that never failed, call-site multiplicity included; 7 disable runs red | | **D-CODEGRAPH-0-BIN** | ruff_charon binary row lanes + header digest; codec9 small codebooks; Quack counting; no strings in the hot path; provenance header + per-row `(artifact, FunDeclId)`; `INVOKES_WORD` to a registered DT64 word | QUEUED: plan `d-codegraph-0-binary-lanes-v1.md` (operator: graph-wide u32 ids, TSV kept as debug `open_tsv`; Amendment A §8: PROV section, FUNDECL/ARTREF lanes, `INVOKES_WORD`, F9–F12). Ruff writer PR first, then the codegraph reader. OPEN: source of the "function realises word" association | unchanged refresh ≤ 20 µs; changed-crate open ≤ 1 ms; binary == TSV row for row | +| **D-TGL-0** | GitHub-analogue Thought-Git lifecycle as a shared contract (branch → change → diff → PR → checks → review → rebase/squash → land → receipt/replay; rework / discard) for Codegraph, DT64, XLOG | CONTRACT (docs): plan `thought-git-lifecycle-contract-v1.md`; PR states are readings of existing records, no workflow engine | each step expressible with listed primitives | +| **D-TGL-LAB** | Codegraph lab: F-LAND / F-DISCARD | SHIPPED (lab, crate-private): `prepare_refresh` / `commit_refresh` with a generation fence (`RefreshRefusal::Stale`); second branch recomputed against current W0 == clean open incl. call-site counts; discard leaves W0 unchanged; 2 disables red (without the fence only a `debug_assert` catches the collision) | not a production land | +| **D-TGL-G8** | bag semantics: merge source revisions, never raw edge ADDs | DECIDED (contract §4) | 5 call sites from two sources after both land (F-LAND) | +| **D-TGL-G9** | lifecycle gate | OPEN (production) | D-TGA-1/3/4/6, D-TGB-2/3/5 delivered against the contract table | +| **D-TGL-G10** | atomic publication: prepare all, durable commit first, then infallible in-memory apply from the land record, replay after crash | OPEN (production); named gap: no prepare/validate route API on `BindStore` | a `Fenced` leaves no published partial state | +| **D-TGL-G11** | branch-local routing; no branch-minted DR position as durable routing key | OPEN (production) | a branch route leaves `BindStore::current_version()` unchanged until land | +| **D-TGL-G12** | identities apart: W-slot, generation, ThoughtCommitId, DurableRef, RouteVersion, Lance version, source SHA (provenance only) | OPEN (production) | joins only through receipt / CommitWitness / MetaKey / land record | +| **D-TGL-PUB** | G10 × G11 lab: prepared land (Codegraph + DT64 route intent), fenced durable commit through a test `WalSink`, crash recovery from the durable record, branch-local routing | SPEC: plan `g10-g11-atomic-publication-lab-v1.md`; boundaries B1–B5 located; needs `BindStore::check_route` (read-only, missing) | L1–L6, each with a disable; L4 shows route re-application ticks `RouteVersion` without a watermark | +| **D-TGL-EXO148** | `DurableRef` + never-reused `ThoughtCommitId` before any durable land, replay or PR record | OPEN (production, EXO-148) | reuse test (D-RST-1) | | **D-DT64-AUTH** | DT64 / F### as the one method authority: census of production words with a two-way exception ratchet; parallel identities (`MethodSignature`, tract `Registry`, XLOG `Word`, ogar-loco `FnIndex`) stay as listed `LAB` adapters | QUEUED: plan `d-dt64-method-authority-census-v1.md` (operator decision 2026-11-02); egglog pinned to one commit in `egglog-r2il-probe`, `dr-math-binding-probe`, `xlog-relcomp-probe` | undeclared production word → red; stale exception → red; current tree green | | **D-CODEGRAPH-0-D** | local stdio MCP over C | QUEUED | only after B and C gates | | **D-CODEGRAPH-0-E** | token measurement with and without the Codegraph | QUEUED | same revision, questions, model; nothing claimed before measured | diff --git a/.claude/board/entries/2026-11-02-d-tgl-release-fence-and-g10-g11-boundaries.md b/.claude/board/entries/2026-11-02-d-tgl-release-fence-and-g10-g11-boundaries.md new file mode 100644 index 000000000..3686c79e3 --- /dev/null +++ b/.claude/board/entries/2026-11-02-d-tgl-release-fence-and-g10-g11-boundaries.md @@ -0,0 +1,30 @@ +# D-TGL: the generation fence holds in release; G10 × G11 boundaries located (2026-11-02) + +**Status:** MEASURED (release run) + SPEC (G10 × G11 lab, plan [`g10-g11-atomic-publication-lab-v1.md`](../plans/g10-g11-atomic-publication-lab-v1.md), D-TGL-PUB). +**Source:** operator review of #1924 (2026-11-02). Production Thought-Git land stays unapproved; EXO-148 stays locked. + +## Git hygiene +#1923 was merged at `83ac4b3a8`. The two commits after that point (`ab9fb36f6`, `4025292ea`) are in #1924, which contains current `main` (`2e19a5adc`, merged in, no history rewrite, no force push). Nothing was lost. + +## Release profile (MEASURED) +- `cargo test --release --lib` in `crates/lance-graph-codegraph`: 7 passed, including `lifecycle::f_land_*` and `f_discard_*`. +- **Release disable (fence removed):** F-LAND fails on its own assertion: `a stale preparation must be refused, got Ok(RefreshStats { edges_added: 6, symbols_appended: 1, .. })`. + - With `debug_assert` compiled out, the stale preparation is **accepted silently**. + - The generation fence is the only guard, and it refuses correctly on its own. + - The debug-build catch (`artifact.rs:186`) was a second net, not the guard. + +## G10 × G11 boundaries (read in code) +- **DT64 `route`:** + - It refuses only `UnknownClass` / `OutsideReservation`. + - The class registry is insert-only and the route log push-only, so a checked route intent never goes stale. + - **Gap:** no read-only `check_route` exists. + - **Re-applying `route` ticks `RouteVersion` again (not idempotent).** +- **Codegraph:** re-applying a source revision is a no-op (digest skip). DT64 recovery therefore needs a watermark; Codegraph recovery does not. +- **Writer (`persist_sink.rs`):** + - The refusals and outcomes are `Fenced` / `HashConflict` / `InvalidArtifact` / `Io` / `Ambiguous` → `Reconciled`. + - `scan_sealed` returns landings without payload, so land records must be read from the image. + - `recover_and_apply` is idempotent by watermark, not by phase. +- **Five crash boundaries:** B1 before commit; B2 refused; B3 ambiguous; B4 committed but not published; B5 process loss. Plan §1.4. +- **The smallest common falsifier set is L1–L6.** Each has its own disable. The decisive one is L4: without the watermark, a second recovery ticks `RouteVersion` while the Codegraph stays unchanged. + +OPEN: `check_route`, the lab `WalSink`, and the L1–L6 test are three separate reviewable steps (plan §4). None is implemented. diff --git a/.claude/board/entries/2026-11-02-p4-operator-decisions-lifecycle-lab.md b/.claude/board/entries/2026-11-02-p4-operator-decisions-lifecycle-lab.md new file mode 100644 index 000000000..3cfe5fd23 --- /dev/null +++ b/.claude/board/entries/2026-11-02-p4-operator-decisions-lifecycle-lab.md @@ -0,0 +1,38 @@ +# P4 operator decisions: GitHub-analogue Thought-Git lifecycle; F-LAND / F-DISCARD lab result (2026-11-02) + +**Status:** lab DONE (Codegraph, crate-private, no Thought-Git wiring); contract written; production gates OPEN. +**Source:** [P4 review](2026-11-02-p4-thought-git-codegraph-dt64-review.md); plan [`thought-git-lifecycle-contract-v1.md`](../plans/thought-git-lifecycle-contract-v1.md) (D-TGL). + +## Decisions (operator, 2026-11-02) +- **North Star:** Thought-Git maps Git/GitHub processes onto DR³ and DT64. + - W0 is main; W1..63 are generation-qualified reusable slots; `ThoughtCommitId` is slot-independent. + - Lifecycle: `branch → change → diff → PR → checks → review → rebase/squash → merge/land → receipt/replay`. A refusal leads to rework or discard. + - Shared by Codegraph, DT64, XLOG and later participants. +- **F-LAND / F-DISCARD:** approved as isolated Codegraph lab tests. +- **DRed exception:** conditional. Replace-and-recompute is allowed only for Codegraph facts that are fully and deterministically reconstructible from immutably identified source artifacts. It does not apply to derived XLOG facts, incremental conclusions or DT64 routing. +- **G8:** merge source revisions, never raw edge ADDs. The rules are in plan §4. +- **New production gates:** + - G9: lifecycle as a contract; + - G10: atomic publication (prepared changes, production fence, controlled W0/DT64 publication, replay or reconciliation after a crash); + - G11: branch-local routing; + - G12: identities kept apart. + +## Lab result (MEASURED) +- `refresh_crate` is now `prepare_refresh` (`&self`; returns an owned `PreparedRefresh`, or `None` for an unchanged file) followed by `commit_refresh`. + - `commit_refresh` first checks the graph generation the change was prepared against, and refuses `RefreshRefusal::Stale` before changing anything. + - The generation is bumped by every commit and by `refold`. It is local and in memory: not an identity, not persisted. +- **F-LAND** (`graph.rs` `mod lifecycle`): + - Setup: two branches over one W0 (source `m` gains `m::x`; source `n` gains `n::y`). Both preparations start their new symbol at the same free position (asserted). + - After A lands, B's old preparation is refused `Stale` and the full-state snapshot is unchanged. + - B is then recomputed against the current W0 and lands at a distinct position. + - Every symbol's callers, call-site count, callees and impact equal a clean open of both revised sources. `m::b` has 5 call sites from two sources. +- **F-DISCARD:** dropping a prepared branch leaves the snapshot unchanged and issues no map-table version. The snapshot covers interner, positions, `last_owner`, lanes, digests, table version and history, cube frame and logs, masks, cached closures and generation. A twin that commits the same change shows the snapshot does move. +- **Disable runs (red):** dropping the generation check, or not bumping the generation, turns F-LAND red. + - **Refusal finding:** in both disables the stale change is caught only by the interner's `debug_assert` "interner changed under the stage" (`artifact.rs:186`). A release build would apply the colliding positions silently. + - The fence is therefore the load-bearing guard, not a convenience. +- **Gates:** codegraph 7 lib tests + all suites green; clippy and fmt clean. + +## Limits stated +- The Codegraph has no `BindStore`, so F-DISCARD's "global `RouteVersion` unchanged" holds by construction. DT64 routing staging is open (G10/G11: no prepare/validate route API on `BindStore`). +- `PreparedRefresh` / `commit_refresh` are crate-private. Nothing publishes, persists or routes. +- Lab results do not satisfy G9–G12 or EXO-148. diff --git a/.claude/board/entries/README.md b/.claude/board/entries/README.md index 26d5435d2..699ed8730 100644 --- a/.claude/board/entries/README.md +++ b/.claude/board/entries/README.md @@ -25,11 +25,13 @@ index row, (3) no duplicate entry id. Checks 1 and 2 are deliberately opposite directions; the stranding this convention prevents shows up in exactly one of them, never both. -548 entries, 2026-08-06 .. 2026-11-02. +550 entries, 2026-08-06 .. 2026-11-02. | date | entry id | finding | file | |---|---|---|---| | 2026-11-02 | `p4-thought-git-codegraph-dt64-review` | | [2026-11-02-p4-thought-git-codegraph-dt64-review.md](2026-11-02-p4-thought-git-codegraph-dt64-review.md) | +| 2026-11-02 | `p4-operator-decisions-lifecycle-lab` | | [2026-11-02-p4-operator-decisions-lifecycle-lab.md](2026-11-02-p4-operator-decisions-lifecycle-lab.md) | +| 2026-11-02 | `D-TGL` | | [2026-11-02-d-tgl-release-fence-and-g10-g11-boundaries.md](2026-11-02-d-tgl-release-fence-and-g10-g11-boundaries.md) | | 2026-11-02 | `D-CODEGRAPH-0` | | [2026-11-02-d-codegraph-0-gap-map.md](2026-11-02-d-codegraph-0-gap-map.md) | | 2026-11-02 | `codegraph-review-operator-decisions` | | [2026-11-02-codegraph-review-operator-decisions.md](2026-11-02-codegraph-review-operator-decisions.md) | | 2026-11-02 | `codegraph-dr3-dt64-ruff-architecture-review` | | [2026-11-02-codegraph-dr3-dt64-ruff-architecture-review.md](2026-11-02-codegraph-dr3-dt64-ruff-architecture-review.md) | diff --git a/.claude/plans/g10-g11-atomic-publication-lab-v1.md b/.claude/plans/g10-g11-atomic-publication-lab-v1.md new file mode 100644 index 000000000..9317c15de --- /dev/null +++ b/.claude/plans/g10-g11-atomic-publication-lab-v1.md @@ -0,0 +1,87 @@ +# Lab spec: G10 × G11 — prepared land, fenced durable commit, crash recovery, branch-local routing (2026-11-02) + +**D-id:** D-TGL-PUB ([STATUS_BOARD](../board/STATUS_BOARD.md)). +**Status:** SPEC, lab only. No production integration, no workflow engine, no extra semantic store, no automatic merge. +**Parent:** [`thought-git-lifecycle-contract-v1.md`](thought-git-lifecycle-contract-v1.md) §5 G10/G11. +**Fence:** EXO-148 stays locked for real durable thought landings. This lab lands a Codegraph source revision and DT64 route intents through a test `WalSink`; it mints no `ThoughtCommitId` or `DurableRef`. + +## 1. Code boundaries (read 2026-11-02) + +### 1.1 DT64 routing +- `BindStore::route(position, class)` (dt64 `lib.rs`) refuses in only two cases: + - `UnknownClass`, when the class is not registered; + - `OutsideReservation`, when `position >= extent.identity.reserved()` (`RouteLog::route`, `dr.rs:255`). +- Otherwise it pushes an event and ticks the shared `VersionClock`. +- `BindStore::classes` is insert-only (`lib.rs:622`, no remove) and `RouteLog` is push-only. A route intent that passes the check therefore stays valid forever. A prepared route cannot go stale. +- **Gap (the one read-only addition needed):** `BindStore` has no check that applies `route`'s two refusals without pushing. + - Reaching `extent` today requires `resolve(view, …)`, which needs a `ClassView`. + - Smallest fix: `BindStore::check_route(&self, position, class) -> Result<(), Refusal>`, the same two conditions with no write. It is a pure function of the insert-only registry. +- **Not idempotent:** applying the same `route(p, c)` twice ticks `RouteVersion` twice and appends a second event. Codegraph re-application is idempotent through the digest skip; route re-application is not. Recovery idempotence for DT64 therefore needs the watermark of §1.3. +- **G12 caveat:** the Codegraph `MapTable` has its own clock (`MapTable::new`). The lab keeps the two clocks separate. Joining them through `with_clock` would make Codegraph map growth tick DT64 `RouteVersion`. + +### 1.2 Codegraph +- `prepare_refresh(&self)` returns a `PreparedRefresh`, or `None` for an unchanged digest. +- `commit_refresh` refuses `Stale` before any change; it has no error path after that check (#1924). +- Re-applying the same source revision is a no-op (digest skip). Multiplicities follow G8: a source's contribution is replaced, never added twice. + +### 1.3 Durable writer (`lance-graph-planner/src/persist_sink.rs`) +- `WalSink::commit_cycle(DetachedCycleBatch)` checks against `batch.frame.base_version`. + - `Fenced { current_head }` (nothing written); + - `HashConflict` (fail closed); + - `InvalidArtifact` (permanent); + - `Io` (nothing published); + - `Ambiguous` (outcome unknown; reconcile by re-submitting the same frozen batch). + - Success is `Committed { version, cycle, batch_hash }`, or `Reconciled { current_head, … }` on a re-submit. +- `scan_sealed(after_cycle)` returns landings **without payload**; durable payloads live only in the coalesced image. Recovery must read land records from the image, never from `scan_sealed`'s slots. +- `recover_and_apply(owner, sealed, applied_through)` replays in canonical `(cycle, stream_position)` order above a durable watermark. **The watermark, not phase equality, is the idempotence key.** On error it returns the partial watermark with the error. +- Production payload ABI: `LanceCycleWriter` requires 512-byte artifacts (`InvalidArtifact`). The lab uses a test `WalSink` and must not claim that ABI. + +### 1.4 Crash and replay boundaries + +| boundary | durable state | in-memory W0 / DT64 | recovery | +|---|---|---|---| +| B1: crash before `commit_cycle` | unchanged | unchanged | none; the branch is discardable | +| B2: `Fenced`, `HashConflict`, `InvalidArtifact`, `Io` | unchanged | unchanged | regenerate against the current head (Fenced/Io), or stop (HashConflict/InvalidArtifact) | +| B3: `Ambiguous` | unknown | unchanged | re-submit the same batch → `Reconciled` or `Committed` | +| B4: committed, crash before publication | land record durable | unchanged | read records above the watermark from the image; re-prepare; apply; advance the watermark | +| B5: process loss during publication | land record durable | lost | rebuild W0 from the durable base, then replay all records once in canonical order | + +The in-memory publication is one infallible step (Codegraph `commit_refresh` + the checked routes + the watermark). Inside one process there is therefore no half-published state. B5 is the process-loss case, and it rebuilds instead of patching. + +## 2. The land record (payload of one landing) + +`{ source: (crate, file digest, rows content), routes: [(position, class)], base: DatasetVersion }` +- It contains no positions: W0 assigns them at apply (G11, F-LAND). +- It contains no `RouteVersion`: the version is issued at apply. +- **Lab encoding:** a fixed little-endian layout inside the test `WalSink` payload. + +## 3. Falsifiers (the smallest common set) + +**Fixture:** +- **Codegraph:** the F-LAND sources `m` / `n`. +- **DT64:** a `BindStore` with one registered derived class `C'` and a free position `p` in its reservation. +- **Writer:** a test `WalSink` with a head and an injectable `Fenced`. +- **Branch:** revises source `m` and carries the route intent `(p, C')`. + +| # | case | must | disable that must turn it red | +|---|---|---|---| +| L1 | prepare: Codegraph `prepare_refresh` + `check_route` | W0 snapshot, `BindStore::current_version()` and `routed_at(·, p)` unchanged | let prepare call `route` instead of `check_route` | +| L2 | `commit_cycle` returns `Fenced` (base behind) | nothing published: W0 snapshot, `RouteVersion`, `routed_at` unchanged; the branch can re-prepare against the new head | publish before the durable commit (the order G10 forbids) | +| L3 | commit succeeds, crash injected before publication | recovery from the durable record restores exactly the confirmed state: snapshot equals a clean open of the landed sources (call-site counts included); `routed_at(current, p) == Some(C')`; `RouteVersion` is exactly one tick past the base | recover from `scan_sealed` slots (no payload) instead of the image | +| L4 | second recovery with the persisted watermark | no change: multiplicities, snapshot and `RouteVersion` identical to after L3 | drop the watermark: `RouteVersion` ticks again (route is not idempotent) while the Codegraph stays unchanged (digest skip), proving the watermark is load-bearing for DT64 | +| L5 | branch route before land | `BindStore` shows no trace of `(p, C')` (version, `routed_at`, `claimed`) until L3 publishes; after a discard, still none | apply branch routes eagerly at prepare | +| L6 | B5: rebuild from the durable base + replay of all records | equals L3's state; one replay per record | replay records in physical order instead of canonical `(cycle, stream_position)` | + +**Anti-vacuity:** +- In L3 the snapshot and `RouteVersion` actually move, compared with L1. +- In L4 the watermark-less twin really ticks. + +## 4. What has to exist before the lab runs (each its own reviewable step) + +1. `BindStore::check_route` (read-only; the two refusals of `route`, no write). Its own falsifiers: + - it agrees with `route` on accept and refuse for every case; + - it leaves `current_version()` unchanged. +2. A lab-only test `WalSink` beside the Codegraph test (dev-dependency on `lance-graph-planner`; test `WalSink` impls exist in ten places, for example `lance-graph-planner/examples/blw_fusion.rs` `MemWal`). No production edge from Codegraph to the planner. +3. The lab test itself, in-crate, because `prepare_refresh` / `commit_refresh` are crate-private. + +Out of scope: `LanceCycleWriter` (needs the 512-byte ABI and real Lance), `ThoughtCommitId`/`DurableRef` (EXO-148), MergeQueue, review gates. diff --git a/.claude/plans/thought-git-lifecycle-contract-v1.md b/.claude/plans/thought-git-lifecycle-contract-v1.md new file mode 100644 index 000000000..8972d12d2 --- /dev/null +++ b/.claude/plans/thought-git-lifecycle-contract-v1.md @@ -0,0 +1,127 @@ +# Plan: the GitHub-analogue Thought-Git lifecycle as a shared contract (2026-11-02) + +**D-id:** D-TGL ([STATUS_BOARD](../board/STATUS_BOARD.md) rows D-TGL-0 … D-TGL-G12). +**Status:** CONTRACT, docs only. Nothing here is wired; no workflow engine is built. +**Sources:** +- P4 review [entry](../board/entries/2026-11-02-p4-thought-git-codegraph-dt64-review.md); +- operator decisions and lab result [entry](../board/entries/2026-11-02-p4-operator-decisions-lifecycle-lab.md); +- `thought-git-api-v1.md` (D-TGA) and `thought-git-basics-v1.md` (D-TGB); +- `crates/thought-git-probe`. + +DECISION (operator, 2026-11-02): Thought-Git maps the processes of Git/GitHub onto DR³ and DT64. It does not store physical Git branches. + +- W0 is the canonical main. +- W1..63 are generation-qualified, reusable working slots. +- A durable `ThoughtCommitId` is independent of the slot. +- **Lifecycle:** `branch → change → diff → PR → checks → review → rebase/squash → merge/land → receipt/replay`. +- **On refusal:** `rework` or `discard`. +- **Scope:** one shared process semantics for Codegraph, DT64, XLOG and later participants. +- **Exclusions:** no 63 graph copies, no extra semantic store, no new method authority. +- **REVISIT WHEN:** EXO-148 lands its types, or a participant cannot express a step with the primitives below. + +## 1. Steps as existing primitives + +A participant (Codegraph, DT64, XLOG …) supplies four things: +- what a change is; +- how it is checked; +- how it is recomputed against a newer W0; +- how it is applied. + +Thought-Git owns the rest: identity, slot, provenance, admission and order. + +| step | Thought-Git primitive (owner) | participant hook | state today | +|---|---|---|---| +| branch | `SlotPool::branch` → `Slot{index, generation}`; identity `ThoughtCommitId` | none | slot pool shipped (lab, D-TGB-1); identity MISSING (EXO-148) | +| change | thought write (D-TGA-1) | Codegraph: a source revision; DT64: a route overlay (G11); XLOG: a delta under its own rules | Codegraph lab shipped (input = source rows); others open | +| diff | `diff` mask (D-TGA-1) | Codegraph: `PreparedRefresh` (the checked op batch) | Codegraph lab shipped | +| PR | enqueue in `MergeQueue` (D-TGA-6). A PR record = `ThoughtCommitId` + base `DurableRef` + receipt plan | none | MISSING (D-TGA-6, EXO-148) | +| checks | deterministic, rerunnable gates, each a named refusal (§2) | participant validation | partly shipped (lab + code) | +| review | authorization, never computation: MUL/Rubicon gate and revision gate G1–G7 | none | gate exists (probe `the_mul_gate_decides_whether_the_write_crosses`); wiring open | +| rebase | D-TGA-3 (DRed) for derived facts; replace-and-recompute only under §3 | participant recompute against the current W0 | Codegraph lab shipped (F-LAND); DRed MISSING | +| squash | `squash_gate` / `protected_squash` (D-TGA-5/5D), under G8 for bag cells | none | value cells shipped (lab); bag cells contract only (§4) | +| merge/land | D-TGA-4 (up to date before landing) + D-TGB-3 (`LanceCycleWriter`), published per G10 | participant apply (infallible after its fence) | Codegraph apply + fence shipped (lab); durable land MISSING | +| receipt/replay | D-TGB-4 `Receipt`, `MetadataTable`, `CommitWitness` | participant program id + provenance (Codegraph: PROV, `binary-lanes` §8) | lab shipped; durable MISSING (EXO-148) | +| rework | same slot, new thought commit whose parent is the refused one; back to `change` | none | contract only | +| discard | `SlotPool::free`; the generation moves on and old handles go `Stale` | none; W0 untouched (F-DISCARD) | slot pool + Codegraph F-DISCARD shipped (lab) | + +**PR states are readings, not a stored state machine.** Each one follows from records that already exist: +- `Draft`: the slot is live and the thought is not enqueued. +- `Open`: the thought is enqueued. +- `Checking` / `ChecksFailed`: the check folds are absent or refused. +- `Approved`: the review gate passed. +- `Behind`: the base is not the current W0. +- `Landed`: a sealed metadata row exists. +- `Discarded`: the slot was freed with no landing. + +No additional table holds these states. + +## 2. Checks (each a named refusal, never an empty answer) + +- **Oracle:** a contradiction with a given, or a divergence between landings (`next_version`). +- **Squash gate:** R1 and R2; a protected reader the retained data cannot answer (`ProtectedRefusal`). +- **Codegraph:** + - `prepare_refresh` refusals: intake, NeedsReopen, a cube stage refusal, a closure stage refusal; + - `commit_refresh` → `Stale`. +- **DT64:** + - `admit` (method, `sig_rev`, guard); + - the D-DT64-AUTH census for production words; + - `route` reservation (validated in prepare, G11). +- **Writer:** `Fenced`, `HashConflict`, `InvalidArtifact` (`persist_sink.rs`). +- **Receipt:** every `receipt::Refusal` arm. + +## 3. Replace-and-recompute instead of DRed (conditional exception) + +DECISION (operator, 2026-11-02): allowed for Codegraph facts that can be reconstructed completely and deterministically from source artifacts with an immutable identity (§8 of `d-codegraph-0-binary-lanes-v1`: PROV with LLBC digest and source revision). + +**Not** allowed for: +- derived XLOG facts; +- incremental conclusions; +- DT64 routing. + +These keep D-TGA-3 (four masks, DRed). + +## 4. G8: bag semantics across thoughts + +DECISION (operator, 2026-11-02): Thought-Git merges **source revisions**, never raw edge ADDs. For a source S, its previous contribution is removed entirely and its new contribution is recomputed entirely. + +- Identical revisions of the same source count once (the second lands as a no-op: equal digest). +- Different sources may contribute the same edge several times. +- Several real call sites stay counted several times. +- Competing revisions of the same source need an explicit conflict decision. A checked three-way merge comes later. +- Squash loses no real multiplicity and applies no identical contribution twice. + +The bag algebra stays with XLOG/DR (multiset delta log, merge on read). Thought-Git manages identity, provenance and admission. + +**Pinned in the lab:** F-LAND's oracle counts 5 call sites of `m::b` from two sources after both branches land. + +## 5. Production gates (all OPEN; lab results never satisfy them) + +- **G9 Lifecycle.** Every step in §1 is expressible with the listed primitives, and the PR states are readings. Closes when D-TGA-1/3/4/6 and D-TGB-2/3/5 are delivered against this table. +- **G10 Atomic publication.** "Codegraph commit → `BindStore::route` → `LanceCycleWriter`" is NOT a valid production land: a later `Fenced` would leave published partial state. The required order: + 1. **Prepare everything.** + - Codegraph: `PreparedRefresh`. + - DT64: route overlay validated against the `BindStore`: the class is registered and the position is inside its reservation. **`BindStore` has no prepare/validate route API today; that is a named gap.** + - The land record and its batch hash. + 2. **Durable commit first:** `commit_cycle` with the land record. + - `Fenced` / `HashConflict` → nothing applied, regenerate. + - `Committed` / `Reconciled` → continue. + 3. **Apply W0 and DT64 in memory from the durable land record,** with no error path. The participant's own fence cannot fire, because the writer is W0's sole mutator. + 4. **Crash between 2 and 3:** replay from the durable land record at start-up (the `recover_and_apply` shape in `persist_sink.rs`). +- **G11 Branch-local routing.** A branch never calls `BindStore::route` and never uses a DR position minted in the branch as a durable routing key. + - Routing changes live in the branch's route overlay; registration of a derived classid is additive and global, but stays unreachable until a land routes to it. + - Positions are assigned by W0 at land (the Codegraph analogue is F-LAND). +- **G12 Identities stay separate:** W-slot, slot generation, `ThoughtCommitId`, `DurableRef`, `RouteVersion`, Lance version, source Git SHA. + - For Codegraph the source Git SHA is provenance (PROV), not a thought identity. + - Joins go only through explicit records: receipt, `CommitWitness`, `MetaKey`, land record. + - The Codegraph generation fence is local and in memory, and is none of these. +- **EXO-148.** `DurableRef` and a never-reused `ThoughtCommitId` are required before any durable land, replay or PR record. + +## 6. Order + +1. F-LAND and F-DISCARD (done, lab). +2. G8 recorded (this plan §4, board). +3. This contract. +4. G9–G12 + EXO-148 registered as production gates. +5. Only then D-TGB-7: the shared crate (`lance-graph-thought-git`), with Codegraph as its first participant. + +No automatic merge. No production land on the strength of lab tests. diff --git a/crates/lance-graph-codegraph/src/graph.rs b/crates/lance-graph-codegraph/src/graph.rs index 2a9412734..41c7ab1c0 100644 --- a/crates/lance-graph-codegraph/src/graph.rs +++ b/crates/lance-graph-codegraph/src/graph.rs @@ -11,7 +11,7 @@ use lance_graph_xlog::cube::{Cube, CubeRefusal, EdgeOp, RelId, StagedEdges}; use lance_graph_xlog::wide::{WideRows, WideStats}; use crate::artifact::{ - self, Interner, Lanes, SymbolStage, Symbols, NO_SPAN, P_CALLS, P_CRATE, P_DEFINES, + self, Interner, Lanes, StagedNames, SymbolStage, Symbols, NO_SPAN, P_CALLS, P_CRATE, P_DEFINES, P_IMPLEMENTS, P_MAY, P_PUBLIC, P_REFERENCES, P_SYNTHETIC, P_UNRESOLVED, }; @@ -245,6 +245,10 @@ pub struct Codegraph { masks: Masks, rconf: OnceCell<(WideRows, WideStats)>, rposs: OnceCell<(WideRows, WideStats)>, + /// Bumped by every committed refresh and every refold: the fence a + /// [`PreparedRefresh`] is checked against. Local and in memory only; not + /// an identity, a version or a persisted counter. + generation: u64, /// Test-only: make the next refresh's validation refuse at a chosen /// point, so the falsifiers can show a refusal changes nothing. #[cfg(test)] @@ -493,6 +497,30 @@ pub enum RefreshRefusal { /// Why growth in place is impossible. reason: &'static str, }, + /// A prepared refresh was offered to a graph that changed since it was + /// prepared. Nothing was changed; prepare again against the graph as it + /// is now. + Stale { + /// The graph generation the change was prepared against. + prepared_at: u64, + /// The graph's generation now. + current: u64, + }, +} + +/// A refresh prepared and validated against one graph generation, not yet +/// applied ([`Codegraph::refresh_crate`] is prepare + commit). Owns +/// everything it needs, so the graph is not borrowed while it is held. +pub(crate) struct PreparedRefresh { + generation: u64, + names: StagedNames, + positions: StagedPositions, + grown: Option<(TileId, LocalMap)>, + staged: StagedEdges, + closures: [Option>; 2], + kid: u32, + fresh: Lanes, + stats: RefreshStats, } impl Codegraph { @@ -557,6 +585,7 @@ impl Codegraph { masks: m, rconf: OnceCell::new(), rposs: OnceCell::new(), + generation: 0, #[cfg(test)] inject: None, }) @@ -621,6 +650,21 @@ impl Codegraph { dir: &Path, krate: &str, ) -> Result { + match self.prepare_refresh(dir, krate)? { + None => Ok(RefreshStats::default()), + Some(prepared) => self.commit_refresh(prepared), + } + } + + /// Prepare and validate a refresh of `krate` without changing the graph: + /// `None` when the file is unchanged (digest skip), otherwise the + /// complete, checked change as an owned [`PreparedRefresh`]. Dropping it + /// discards the change; nothing was written. + pub(crate) fn prepare_refresh( + &self, + dir: &Path, + krate: &str, + ) -> Result, RefreshRefusal> { let text = artifact::read_crate_text(dir, krate).map_err(RefreshRefusal::Intake)?; // An unchanged file is no delta. The digest recorded when the crate // was last read (at open or at a successful refresh) decides, before @@ -629,7 +673,7 @@ impl Codegraph { let file_digest = artifact::digest(text.as_bytes()); if let Some(k) = self.symbols.get(krate) { if self.lanes.digest_of(k) == Some(file_digest) { - return Ok(RefreshStats::default()); + return Ok(None); } } @@ -741,14 +785,52 @@ impl Codegraph { } } } - let names = syms.into_names(); - let positions = pos.finish(); + Ok(Some(PreparedRefresh { + generation: self.generation, + names: syms.into_names(), + positions: pos.finish(), + grown: grown.map(|m| (last, m)), + staged, + closures, + kid, + fresh, + stats, + })) + } + + /// Apply a [`PreparedRefresh`]. The one check comes first and changes + /// nothing: the graph must still be at the generation the change was + /// prepared against ([`RefreshRefusal::Stale`]); a change prepared + /// against an older graph would hand out positions the graph has since + /// given to other symbols. After the check there is no error path. + pub(crate) fn commit_refresh( + &mut self, + prepared: PreparedRefresh, + ) -> Result { + if prepared.generation != self.generation { + return Err(RefreshRefusal::Stale { + prepared_at: prepared.generation, + current: self.generation, + }); + } + let PreparedRefresh { + names, + positions, + grown, + staged, + mut closures, + kid, + fresh, + mut stats, + .. + } = prepared; // ---- commit: no error path below ---- + self.generation += 1; self.symbols.absorb(names); self.pos.absorb(positions); let frame = match grown { - Some(map) => { + Some((last, map)) => { let rev = self.table.revisions(); self.table .advance(rev.source + 1, rev.relation_map, vec![(last, Some(map))]); @@ -807,6 +889,9 @@ impl Codegraph { /// nothing calls this on a threshold). Cached closures stay valid: the /// base now equals the effective edges they were computed from. pub fn refold(&mut self) -> Result<(), CubeRefusal> { + // A prepared refresh staged its closures against the logs; folding + // them into the base makes it stale. + self.generation += 1; for rel in 0..6 { self.cube.refold(rel)?; } @@ -989,7 +1074,7 @@ mod atomic { /// positions (incl. `last_owner`), lanes and digests, crate statuses, /// table version, revisions and per-tile history, the cube's frame and /// every delta log, masks, and the cached closures. - fn snapshot(g: &Codegraph) -> String { + pub(super) fn snapshot(g: &Codegraph) -> String { let names: Vec<&str> = (0..g.symbols.len() as u32) .map(|i| g.symbols.name(i)) .collect(); @@ -1000,7 +1085,8 @@ mod atomic { .collect(); let logs: Vec<_> = (0..6).map(|r| g.cube.delta_log(r).cloned()).collect(); format!( - "{names:?}\n{:?}\n{:?}\n{crates:?}\n{:?} {:?} {tiles:?}\n{:?}\n{logs:?}\n{:?}\n{:?}\n{:?}", + "gen {}\n{names:?}\n{:?}\n{:?}\n{crates:?}\n{:?} {:?} {tiles:?}\n{:?}\n{logs:?}\n{:?}\n{:?}\n{:?}", + g.generation, g.pos, g.lanes, g.table.current_version(), @@ -1167,3 +1253,229 @@ mod atomic { assert_eq!(got.sites, want.sites, "bag multiplicity"); } } + +/// P4 lab falsifiers F-LAND and F-DISCARD (operator decision 2026-11-02): +/// two branches over one W0 base, each a revision of one source; landing +/// recomputes against the current W0; discarding leaves W0 untouched. Lab +/// only: no Thought-Git wiring, no persistence, no routing. +#[cfg(test)] +mod lifecycle { + use super::*; + use std::collections::BTreeMap; + use std::fs; + use std::path::PathBuf; + use std::sync::atomic::{AtomicU32, Ordering}; + + const HEAD: &str = "#version 2\n#schema predicate\tsubject\tobject\tgrade\tfile\tstart\tend\n"; + + fn dir(tag: &str) -> PathBuf { + static NEXT: AtomicU32 = AtomicU32::new(0); + let n = NEXT.fetch_add(1, Ordering::Relaxed); + let d = std::env::temp_dir().join(format!("cg_lifecycle_{}_{tag}_{n}", std::process::id())); + let _ = fs::remove_dir_all(&d); + fs::create_dir_all(&d).expect("temp dir"); + d + } + + /// Rows of one source: `defs` defined, `calls` as (caller, callee) sites. + fn rows(file: &str, defs: &[&str], calls: &[(&str, &str)]) -> String { + let mut s = String::from(HEAD); + for (i, f) in defs.iter().enumerate() { + s.push_str(&format!( + "DEFINES\t{f}\t{f}\tCOMPILER\t{file}\t{i}\t{}\n", + i + 1 + )); + } + for (i, (a, b)) in calls.iter().enumerate() { + let at = 100 + i; + s.push_str(&format!( + "CALLS_RESOLVED\t{a}\t{b}\tCOMPILER\t{file}\t{at}\t{}\n", + at + 1 + )); + } + s + } + + // W0: m::a calls m::b at two sites; n::c calls m::b once. + fn m_v1() -> String { + rows( + "m.rs", + &["m::a", "m::b"], + &[("m::a", "m::b"), ("m::a", "m::b")], + ) + } + fn n_v1() -> String { + rows("n.rs", &["n::c"], &[("n::c", "m::b")]) + } + // Branch A revises source m: a new symbol m::x calling m::b. + fn m_v2() -> String { + rows( + "m.rs", + &["m::a", "m::b", "m::x"], + &[("m::a", "m::b"), ("m::a", "m::b"), ("m::x", "m::b")], + ) + } + // Branch B revises source n: a new symbol n::y; n::c now calls it too. + fn n_v2() -> String { + rows( + "n.rs", + &["n::c", "n::y"], + &[("n::c", "m::b"), ("n::c", "n::y"), ("n::y", "m::b")], + ) + } + + fn store(tag: &str, m: &str, n: &str) -> PathBuf { + let d = dir(tag); + fs::write(d.join("m.rows.tsv"), m).expect("write"); + fs::write(d.join("n.rows.tsv"), n).expect("write"); + d + } + + fn names(g: &Codegraph, mask: &[u64]) -> Vec { + let mut v: Vec = (0..mask.len() * 64) + .filter(|&p| mask[p / 64] >> (p % 64) & 1 == 1) + .filter_map(|p| g.symbol(p as Pos).map(str::to_string)) + .collect(); + v.sort(); + v + } + + /// Every symbol's callers (names + call-site count), callees and + /// impact, by name, so two graphs with different positions compare. + fn answers(g: &mut Codegraph) -> BTreeMap { + let all: Vec = (0..g.words() * 64) + .filter_map(|p| g.symbol(p as Pos).map(str::to_string)) + .collect(); + let mut out = BTreeMap::new(); + for name in all { + let p = g.pos(&name).expect("pos"); + let c = g.callers(p).expect("callers"); + let e = g.callees(p).expect("callees"); + let i = g.impact(p).expect("impact"); + out.insert( + name, + format!( + "callers {:?}/{:?}/{:?} sites {} callees {:?}/{:?}/{:?} impact {:?}/{:?}/{:?}", + names(g, &c.graded.confirmed), + names(g, &c.graded.possible), + names(g, &c.graded.unknown), + c.sites, + names(g, &e.graded.confirmed), + names(g, &e.graded.possible), + names(g, &e.graded.unknown), + names(g, &i.confirmed), + names(g, &i.possible), + names(g, &i.unknown), + ), + ); + } + out + } + + /// F-LAND. Branch A (source m) and branch B (source n) are prepared + /// against the same W0; both mint a new symbol, so both claim the next + /// free position. After A lands, B's old preparation is refused (it would + /// hand A's position to another symbol) and B is recomputed against the + /// current W0. The result equals a clean open of both revised sources, + /// call-site counts included. Red if a stale preparation is applied + /// (disable: drop the generation check in `commit_refresh`). + #[test] + fn f_land_recomputes_a_second_branch_against_the_current_w0() { + let w0 = store("w0", &m_v1(), &n_v1()); + let branch_a = store("a", &m_v2(), &n_v1()); + let branch_b = store("b", &m_v1(), &n_v2()); + let mut g = Codegraph::open(&w0).expect("open W0"); + let base_positions = g.pos.len(); + + let a = g + .prepare_refresh(&branch_a, "m") + .expect("A") + .expect("A changes"); + let b_old = g + .prepare_refresh(&branch_b, "n") + .expect("B") + .expect("B changes"); + // The collision the fence exists for: both preparations start their + // new symbols at the same free position. + assert_eq!(a.positions.base_len, base_positions); + assert_eq!(b_old.positions.base_len, base_positions); + assert!(!a.positions.ids.is_empty() && !b_old.positions.ids.is_empty()); + + g.commit_refresh(a).expect("land A"); + let x = g.pos("m::x").expect("m::x landed"); + assert_eq!( + x as usize, base_positions, + "A's new symbol took the free position" + ); + + let before = super::atomic::snapshot(&g); + match g.commit_refresh(b_old) { + Err(RefreshRefusal::Stale { .. }) => {} + other => panic!("a stale preparation must be refused, got {other:?}"), + } + assert_eq!( + super::atomic::snapshot(&g), + before, + "the refusal changed nothing" + ); + + let b = g + .prepare_refresh(&branch_b, "n") + .expect("B again") + .expect("B changes"); + g.commit_refresh(b).expect("land B"); + let y = g.pos("n::y").expect("n::y landed"); + assert_ne!(x, y, "distinct symbols, distinct positions"); + + let both = store("oracle", &m_v2(), &n_v2()); + let mut clean = Codegraph::open(&both).expect("clean open"); + assert_eq!(answers(&mut g), answers(&mut clean)); + // Anti-vacuity: the oracle sees the real multiplicities. + let pb = clean.pos("m::b").expect("m::b"); + assert_eq!( + clean.callers(pb).expect("q").sites, + 5, + "2 + 1 + 1 + 1 call sites" + ); + } + + /// F-DISCARD. A prepared branch that is dropped leaves W0 exactly as it + /// was: interner, positions, `last_owner`, lanes, digests, the map + /// table's version and history, the cube's frame and logs, masks, + /// cached closures and the graph generation. The codegraph holds no + /// `BindStore`, so DT64 routing is untouched by construction. + #[test] + fn f_discard_leaves_w0_unchanged() { + let w0 = store("w0", &m_v1(), &n_v1()); + let branch_a = store("a", &m_v2(), &n_v1()); + let g = Codegraph::open(&w0).expect("open W0"); + g.closure(RCONF).expect("rconf"); + g.closure(RPOSS).expect("rposs"); + let before = super::atomic::snapshot(&g); + let version = g.table.current_version(); + + let a = g + .prepare_refresh(&branch_a, "m") + .expect("A") + .expect("A changes"); + // Anti-vacuity: the discarded change was real (a new symbol, edges). + assert!(!a.positions.ids.is_empty() && !a.staged.is_empty()); + drop(a); + + assert_eq!(g.table.current_version(), version, "no version issued"); + assert_eq!(super::atomic::snapshot(&g), before); + + // Twin: committing the same change does move W0, so the snapshot + // comparison above can fail. + let mut twin = Codegraph::open(&w0).expect("open twin"); + let a = twin + .prepare_refresh(&branch_a, "m") + .expect("A") + .expect("A changes"); + twin.commit_refresh(a).expect("commit"); + assert_ne!( + super::atomic::snapshot(&twin), + super::atomic::snapshot(&Codegraph::open(&w0).expect("w0")) + ); + } +}