Skip to content

codegraph: read rows.bin; header-only unchanged refresh (D-CODEGRAPH-0-BIN PR B, chunks 1-2) — changed-crate timing gate NOT met - #1929

Merged
AdaWorldAPI merged 4 commits into
mainfrom
claude/codegraph-bin-reader
Nov 2, 2026
Merged

AdaWorldAPI merged 4 commits into
mainfrom
claude/codegraph-bin-reader

Conversation

@AdaWorldAPI

@AdaWorldAPI AdaWorldAPI commented Nov 2, 2026 •

Copy link
Copy Markdown
Owner

Summary

This is the reader half of plan d-codegraph-0-binary-lanes-v1.md, chunks 1 and 2. It reads the <crate>.rows.bin files that ruff_charon writes (ruff #133).

Chunk 1, artifact_bin.rs:

  • Header::parse and Header::read validate the fixed 64 bytes without touching the body.
  • CrateFile::parse verifies the whole file in the writer's own check order. Each malformed shape is refused with its own BinRefusal; IntakeRefusal::Bin carries it.
  • Every accessor reads the one owned buffer through little-endian lenses.
  • artifact::digest now folds the length as little-endian bytes. Before, it went through Hasher::write_u64, which folds native-endian bytes. The result is unchanged on little-endian hosts and now matches the writer everywhere.
  • The golden tests/data/golden/cg_lib.rows.bin and .rows.tsv are copied byte-identical from ruff (writer commit 6c3d1bf8da).

Chunk 2, intake:

  • read_dir takes each crate from <crate>.rows.bin when present, else from .rows.tsv.
  • read_bin_as fills the existing Lanes from the id lanes. It calls the interner once per distinct symbol, predicate, grade and file per crate file. It parses no text and splits no strings.
  • prepare_refresh reads only the 64-byte header of a binary file and returns before touching the body when the digest matches the recorded one.
  • The interner itself is unchanged. Replacing it is chunk 3.

Timing gate (release, mask-risc store, 17,155 rows, N = 41 fixed in advance, 4 CPUs at load ≈ 0.3)

examples/bin_timing.rs (committed) produced these numbers:

measurement median range gate
unchanged refresh_crate (rows.bin) 2.58 µs 2.53–14.92 µs ≤ 20 µs: met
changed-crate intake from rows.bin 2.54–2.84 ms 2.35–5.98 ms ≤ 1 ms: NOT met
— fs::read 0.16 ms
— CrateFile::parse (verify) 0.35 ms
— read_bin_as (interning + lanes) 2.04 ms
baseline: the same crate from rows.tsv 5.08–7.41 ms —

The unchanged path improves about 600× over the TSV path (1.5–1.7 ms before). The changed-crate intake is about 2.6× faster than the TSV path, but it fails the plan's ≤ 1 ms gate.

Per the plan, a failed gate is reported, not shipped as passing. The remaining cost is Symbols::intern, about 2 ms of the 2.5 ms: string hashing plus an allocation for each new name. That is the interner replacement the plan already assigns to chunk 3:

  • a global map from name hash (SYM_HASH) to global id, with label locators into the retained file buffers;
  • no intern at open or refresh;
  • falsifiers F4 and F5.

Whether to merge chunks 1–2 now, as a strict improvement with the gate open, or to hold them for chunk 3 is the operator's decision.

Test Plan

  • tests/bin_intake.rs (15 tests):
    • the ruff golden parses and matches its TSV row for row;
    • the digest golden vectors match those asserted in ruff;
    • Header::read requests exactly 64 bytes;
    • one refusal test per malformed shape;
    • provenance and per-row FunDeclId.
  • tests/bin_refresh.rs (5 tests):
    • F1: the binary and TSV stores open to the same graph, label for label (rows, symbols, crate status, and callers, callees, impact and references for every symbol).
    • F2: an unchanged refresh with a garbled body returns default stats without reading the body.
    • A changed digest with a garbled body is refused as Bin(Digest).
    • The binary file is preferred over the TSV.
    • A TSV-only store still opens.
  • Disable checks. Each was applied with an asserted anchor, run, and reverted; each turned the intended tests red:
    • subject range check removed;
    • digest check skipped;
    • unknown required tag accepted;
    • header-digest skip removed: the F2 test fails;
    • binary file ignored in read_dir: 4 tests fail.
  • Crate gates: all codegraph suites pass; cargo clippy --all-targets -- -D warnings and fmt are clean.

Not to be merged automatically.

🤖 Generated with Claude Code

https://claude.ai/code/session_01X6y3drwKSE2zSgoexheLFX

…PR B, chunk 1)

artifact_bin.rs reads the binary rows format ruff_charon writes (ruff
#133): Header::parse/read checks the fixed 64 bytes without touching the
body (the unchanged-refresh path), CrateFile::parse verifies the whole file
in the writer's check order and refuses each malformed shape with its own
BinRefusal, and every accessor is a lens over the one owned buffer. The
golden cg_lib.rows.bin copied from ruff parses and matches its TSV row for
row. artifact::digest now folds the length as little-endian bytes, matching
the writer on every host. Not yet wired into Codegraph::open/refresh.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X6y3drwKSE2zSgoexheLFX
@coderabbitai

coderabbitai Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Important

  • Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 65480a9b-b10e-48c2-91de-64d8bccd8d48
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-02T17:02:18.126493Z b1fa299 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

… only the header (D-CODEGRAPH-0-BIN PR B, chunk 2)

read_dir takes each crate from `<crate>.rows.bin` when present, else from
`<crate>.rows.tsv`. read_bin_as fills the existing Lanes from the id lanes:
one interner call per distinct symbol, predicate, grade and file per crate
file, no text parsing, no string splitting. prepare_refresh reads the 64-byte
header of a binary file and returns before touching the body when its digest
matches the recorded one. The interner itself is unchanged (chunk 3).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X6y3drwKSE2zSgoexheLFX
… a per-stage split

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X6y3drwKSE2zSgoexheLFX
@AdaWorldAPI AdaWorldAPI changed the title codegraph: zero-copy rows.bin parser and verifier (D-CODEGRAPH-0-BIN PR B, chunk 1) codegraph: read rows.bin; header-only unchanged refresh (D-CODEGRAPH-0-BIN PR B, chunks 1-2) — changed-crate timing gate NOT met Nov 2, 2026
…us met; changed 2.5 ms not met, interner)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X6y3drwKSE2zSgoexheLFX
@AdaWorldAPI
AdaWorldAPI merged commit 4fadef6 into main Nov 2, 2026
14 of 15 checks passed
AdaWorldAPI pushed a commit that referenced this pull request Nov 2, 2026
…e open, B3 next)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X6y3drwKSE2zSgoexheLFX
AdaWorldAPI added a commit that referenced this pull request Nov 2, 2026
…1a5sw

board: /coresearch XLOG / Soufflé / Crepe / DR³ exploration map + Persistent Relational Grammar extension; #1929 recorded as a plateau
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants