Skip to content

Stage D part 1: one vector copy, batch search and binary snapshot in the other five ports - #3

Open
maheshvaikri-code wants to merge 7 commits into
mainfrom
stage-d-ports
Open

maheshvaikri-code wants to merge 7 commits into
mainfrom
stage-d-ports

Conversation

@maheshvaikri-code

Copy link
Copy Markdown
Contributor

Stage D, part 1: port the proven Rust pieces to the other five ports

Carries three things from the reworked Rust crate (isongraph-vector-rs/DESIGN.md) to Python, C#, TypeScript, JavaScript and C++. HNSW for C++/C# is part 2.

1. One copy of every vector. Each store keeps each vector once in a chunked float32 table; records and the IVF index refer to its rows. Pending rows fold into IVF at write time (Python, C#, C++; TS/JS keep the search-time fold). Steady RSS at 100k x 384 (raw 146 MB): Python 437 -> 211 MB, TS/JS 435 -> 197 MB, C# IVF store 409 -> 215 MB, C++ IVF store 329 -> 167 MB. At 1M x 384 (one run each): 1.12-1.25x raw.

2. Batch / concurrent search. Python: matrix-matrix batch (about 8x one-at-a-time). C#, C++: concurrent searches plus parallel batch, bit-identical to single-query. TS/JS: none (single-threaded).

3. Optional binary snapshot (<graph>.embeddings.snap, spec in Rust DESIGN.md section 11). All six ports write and read it; the JSON sidecar stays the portable default and the graph file, sidecar bytes and digest are unchanged. A snapshot is used only when it provably belongs to the graph and sidecar on disk. 3-5x smaller and roughly 13-45x faster than JSON at 100k x 384; the only way to persist 100k rows in JS.

Verification

  • Per port locally (Windows): C# 280, TS 285, JS 284, Python 259, C++ 214 (+37 sqlite) tests pass; each agent compared old vs new implementations (tens of thousands of comparisons, no real differences apart from the documented Python scoring change).
  • Golden fixture written by Rust (tests/fixtures/snapshot/) is loaded from the snapshot by every port; C#, C++ and TS re-save it byte for byte.
  • CI cross-port job extended: every port reads every other port's snapshot (source must be snapshot), a stale snapshot must not mask a mismatch error, and loads without the .snap must match. 108/108 locally.

Breaking changes

  • get/getEmbedding return a copy in C#, C++ (std::optional<EmbeddingRecord>), TS and JS; index stats report 0 vector bytes.
  • TS/JS store vectors as float32 (inputs not float32-exact read back rounded).
  • Python in-memory stores have no SQLite table; Python exact top-k scores are now canonical (can differ in the last float32 bit and reorder near-ties; store.canonical_scores = False restores the old behaviour).
  • C# Add copies the caller's array.

Not done / limits

  • Not tested locally: Linux, macOS, GCC 15, aarch64, Node 20/22, numpy 1.x (CI will).
  • C# keeps its own summation order (scores can differ from Rust in the last bits, as before); its single-query IVF is 8-25% slower.
  • The snapshot format has no id kind; ports with integer ids restore it from the graph (a v2 should add a bit).
  • Fixes the C# competitive bench's recall_before 0.000 (a benchmark bug).

🤖 Generated with Claude Code

maheshvaikri-code and others added 7 commits November 17, 2026 07:53
…r other ports' output

snapshot_fixture writes tests/fixtures/snapshot/golden.* (graph, JSON sidecar,
binary snapshot) with the awkward cases in it; every port must load it from the
snapshot. snapshot_check loads a graph saved by any port and reports whether the
snapshot was accepted and whether the store equals a JSON-only load.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…snapshot

VectorTable holds each vector once in 1024-row chunks; the IVF index keeps only
cell lists and follows table compaction; pending rows fold at write time.
Searches share a ReaderWriterLockSlim and run concurrently;
SimilaritySearchBatch runs queries in parallel with results bit-identical to
the single-query path. Optional .embeddings.snap per Rust DESIGN.md section 11
(reproduces the Rust golden snapshot byte for byte). SaveEmbeddings streams
instead of building one giant string. Fixes the competitive bench's
recall_before 0.000 (a bench bug: reference recall unset when the sweep never
reached 0.95).

Breaking: Get/GetEmbedding return a copy of the vector, Add copies the caller's
array, AnnStats.VectorBytes is 0, pending rows fold below 256 at write time.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ary snapshot

VectorTable replaces the record map plus ScanCache: float32 rows in 1024-row
chunks, norms/type codes/models in parallel columns, tombstones compacted at
25%, IVF reads rows from the table. The exact scan scores four rows at a time
with independent per-row sums, so scores are unchanged and the scan is faster.
Optional .embeddings.snap per Rust DESIGN.md section 11 (reproduces the Rust
golden snapshot byte for byte; the only way to persist 100k x 384 in JS, since
the JSON export exceeds V8's string limit). New browser-safe snapshot module
with injected SHA-256; file I/O in node.ts / node.js.

Breaking: vectors are stored as float32 (values widen to double on read),
get()/getEmbedding() return a copy, AnnStats.cacheBytes is always 0.
Not done: write-time IVF fold (kept the search-time fold; a fold in a write
would block it for seconds), batch search (single-threaded).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… snapshot

In-memory stores no longer keep a SQLite copy next to the scan matrix; the
contiguous float32 matrix is the table (pure-Python path: one array('f') per
row, patched in place with tombstones and compaction). IVF pending rows fold
at write time (256). similarity_search_batch does one matrix-matrix product
per group of queries. Optional .embeddings.snap per Rust DESIGN.md section 11
(save(snapshot=True), load_with_report; verified accepted by the Rust reader).

Bounded exact search now returns canonical scores: BLAS only narrows the field
and every row within a derived error margin of the k-th score is rescored with
one kernel that gives the same bits whatever else is in the call, which is what
makes batch equal single bit for bit. Scores can differ from the previous
release in the last float32 bit and near-ties can reorder;
store.canonical_scores = False restores the old behaviour.

Breaking: in-memory stores have no SQLite table (_get_conn is empty), IVF
pending is below 256 between writes.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…, optional binary snapshot

The index no longer copies vectors; ChunkedMatrix (1024-row chunks) is the only
copy and records reference rows. One out-of-line scoring routine
(detail::cosineRows, noinline/noclone) is used by the scan, batch, IVF rerank
and snapshot-loaded stores. Searches share a shared_mutex; similaritySearchBatch
scores tiles of 16 queries per chunk across bounded worker threads. Pending
rows fold at write time (256). Optional .embeddings.snap per Rust DESIGN.md
section 11 with a streaming SHA-256; re-saving the Rust golden store reproduces
its snapshot byte for byte.

Breaking: get()/getEmbedding() return std::optional<EmbeddingRecord> (a copy),
EmbeddingRecord lost ann_slot, AnnStats::vectorBytes is 0, equal scores order by
arrival, pending rows fold at the 256th write.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…larifications

DESIGN.md section 11: row order is the writing port's sidecar order (readers must
not rely on it), repeated-key and non-UTF-8 rejection spelled out, count limit
for ports with bounded arrays, short-file fingerprint windows, and the fact that
ids are text (ports with integer ids restore the kind from the graph; a v2 should
carry an id-kind bit).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Every port writes the snapshot, every port must accept every other port's
snapshot (source == snapshot), a stale snapshot beside an edited graph must not
mask the mismatch error, and a copy without the .snap must load through the
JSON with the same observation. 108/108 locally. Also repairs the C++ helper for
getEmbedding returning std::optional.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant