Skip to content

ANN in every port, server package, real-data benchmarks, Rust engine rework (HNSW, int8, snapshot) - #2

Merged
maheshvaikri-code merged 22 commits into
mainfrom
feature/ann-ports-and-server
Nov 17, 2026
Merged

maheshvaikri-code merged 22 commits into
mainfrom
feature/ann-ports-and-server

Conversation

@maheshvaikri-code

@maheshvaikri-code maheshvaikri-code commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

What this adds

Approximate search in every port. Opt-in IVF-flat in Python (already), C#, C++, TypeScript, JavaScript and Rust. Exact search stays the default; the index is never persisted and changes nothing in graph files, sidecars or the graph digest.

Other fixes for 1.0.0. Async HttpEncoder path in TS/JS, tested libcurl transport for C++, and a new isongraph-vector-server package (tenants, quotas, snapshots, sharding router, read replicas; alpha, no TLS/failover).

Rust engine rework (isongraph-vector-rs/DESIGN.md): concurrent and batch search on a persistent pool, one copy of each vector (1.12x raw at 1M x 384, was 2.41x), folds at write time (first query after 30k writes into 1M ~8 ms, was ~600 ms), plus opt-in HNSW, int8 scan with exact rerank, and a binary snapshot beside the JSON sidecar.
Breaking (Rust): EmbeddingStore::get / SemanticGraph::get_embedding return an owned record; use get_vector to borrow.

Benchmarks, reproducible: benchmarks/multihop/ (HotpotQA/2Wiki/MuSiQue, graph built without gold annotations, shuffled-label controls, independent PPR baseline), benchmarks/competitive/ (vs faiss, hnswlib, usearch, chromadb, lancedb, sqlite-vec, numpy), benchmarks/rust-stage-a/, benchmarks/rust-stage-c/. Most of the diff size is raw benchmark output.

What the evidence supports (and does not)

  • Does not support "best vector store". Not a vector-database replacement.
  • Graph-aware retrieval beats plain vector search on multi-hop QA when the graph carries real structure; an independent PPR baseline matches it, and it hurts comparison questions.
  • Rust HNSW is level with faiss HNSW / hnswlib single-thread at recall 0.95 (1M cells are single runs). Rust exact scan is at parity with numpy / faiss Flat, not ahead (an earlier 6-33% claim was refuted by Stage C). IVF is 0.71-0.76x faiss IVFFlat; IVF+int8 beats it.
  • Weak: ANN index not persisted (HNSW rebuild ~150 s at 1M), HNSW single-call inserts, the other ports are still 2-4x heavier on memory and have no batch search.
  • Benchmark machine was not fully quiet during Stage C; differences under ~15% should be read as parity.

Test status

Rust: all suites pass with --all-features, clippy -D warnings and fmt clean. C# 250, TS 255, JS 254, server 146 pass / 1 skip, C++ 188 main (sqlite and curl suites not re-run after the last change).

Not verified

New CI jobs on real runners (cpp-curl.yml, server tests), the server Dockerfile (never built), non-x86_64 Rust paths. C# at GloVe 1M shows recall 0.000 before writes in the mutation table: likely a harness glitch, not investigated, do not quote C# 1M figures.

🤖 Generated with Claude Code

maheshvaikri-code and others added 19 commits November 15, 2026 21:57
Adds AnnConfig on EmbeddingStore / SemanticGraph (set_ann): a lazily built,
seeded IVF-flat index kept current under add/update/remove, never persisted,
with exact override, build_ann_index, ann_stats and recall_check. Exact search
now scans a contiguous f32 cache with AVX2/FMA kernels and threads, and equal
scores are ordered deterministically instead of by HashMap order.

Graph files, sidecars and the graph digest are unchanged with ann on.
SimilarityOptions gains an `exact` field.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
AnnMode.Ivf on EmbeddingStore / SemanticGraph builds a lazily trained, never
persisted IVF-flat index with the same defaults and update policy as the
Python reference, an exact override, and RecallCheck. In-memory store only.
Includes a benchmark console project outside the solution.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…r real

enableAnn / AnnOptions add the same IVF-flat index as the other ports (SIMD
kernels, threaded seeded k-means). The libcurl HttpEncoder transport is now
built and tested against real libcurl with a loopback server, which found and
fixed two bugs: Expect: 100-continue stalling large bodies, and a misleading
error for over-cap responses. Adds the cpp-curl workflow, unproven until CI
runs.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A separate package that wraps the library as an HTTP service: per-tenant API
keys stored as salted hashes, isolated graphs, quotas and rate limits; search,
multi_hop, path, subgraph and similar endpoints with filters and weights;
atomic checksummed snapshot generations with graph-digest verification; a
stateless tenant-sharding router; snapshot read replicas that refuse a
snapshot whose digest does not match. Stdlib-only apart from the library.

Not provided: TLS, automatic failover, resharding, a write-ahead log,
cross-tenant queries. The Dockerfile has not been built.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
ann: 'ivf' on EmbeddingStore, SemanticGraph and loadGraph adds the same
lazily built, never persisted IVF-flat index as the other ports, with an exact
override, buildAnnIndex, annStats and recallCheck. Plain JavaScript builds are
slow (about 54 s at 100k, about 8 minutes at 1M) and recall at 1M is 0.94 at
the default probe; call buildAnnIndex() at startup. See bench/results.md.

AsyncEncoder plus ...Async methods on the store and graph let HttpEncoder
auto-embed without a prefetch. Failures are atomic, model metadata and key
redaction are unchanged, and the sync interface still works. HttpEncoder gains
a concurrency option and shares identical in-flight requests.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…rary comparison

benchmarks/multihop: HotpotQA, 2Wiki and MuSiQue with two encoders, graphs
built without the gold annotations, negative controls, and BM25, hybrid,
cross-encoder and personalised-PageRank baselines. Graph-aware retrieval beats
plain vector search on all three, but an independent PPR re-ranker matches it
and the gain depends on graph quality.

benchmarks/competitive: identical-vector comparison against faiss, hnswlib,
usearch, chromadb, lancedb, sqlite-vec and numpy on text, GloVe and SIFT
embeddings at 100k and 1M. The results do not support calling this the best
vector store: it trails the HNSW libraries on query speed and has no batched
search. Raw console dumps and the discarded unpinned first run are not kept.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Rust example, C++ executable, C# console project and JS script that read the
shared binary vector files and report recall and latency in the format
benchmarks/competitive expects. No library source changes.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…rk findings

CHANGELOG, README, DESCRIBE and ROADMAP now describe approximate search in
every in-memory port, the async JS encoders, the tested libcurl transport,
the server package and the two real-data benchmarks. The limits state plainly
that the ports are slower per query than HNSW libraries and that the supported
claim is graph-aware retrieval on multi-hop questions.

Adds scripts/js/gen_index_js.py (with --check) so isongraph-vector-js/src/
index.js can be regenerated from the TypeScript source from a fresh checkout,
and the type stripper used for the JS tests.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…write time

Searches take &self on a persistent pool; batch search is bit-identical to
single-query search; vectors live once in a chunked matrix (1.12x raw at
1M x 384, was 2.41x); pending rows fold in groups of 256 so the first query
after 30k writes into 1M takes ~8 ms (was ~600 ms). Breaking: get() and
get_embedding() return owned records. See isongraph-vector-rs/DESIGN.md and
benchmarks/rust-stage-a/RESULTS.md.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
ScanPrecision::Int8 keeps a per-row symmetric int8 copy in the chunked
table (0.25x the raw matrix, maintained by every put, update, delete and
compaction). Full scans and IVF probed cells score rows with an integer dot
(AVX2 maddubs kernel behind an assert!, portable fallback), keep the best
k * rerank_factor, and re-score those with the exact f32 kernel, so returned
scores are exact cosines and batch answers equal single-query answers bit for
bit. int8_recall_report measures recall against the f32 scan per rerank factor.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
save_with_snapshot writes <graph>.embeddings.snap after the sidecar: little
endian, versioned header (magic, dim, count, graph digest, sidecar identity),
blocks of metadata plus f32 rows in the sidecar's order, SHA-256 trailer,
atomic temp+rename. load/load_with use it only when its digest and sidecar
identity match and the checksum verifies, else the JSON is read exactly as
before; load_with_report says which path ran and why a snapshot was skipped.
Graph files, sidecar bytes and the digest are unchanged. The IVF index and the
int8 copy are not stored and rebuild.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Slot ids and links only, over the single stored matrix; scores come from the
canonical kernels so a hit's score equals the exact scan's. Inserts on write,
updates relink, deletes tombstone and compaction bridges over them. The build
is a deterministic batched parallel insert (same graph for any thread count).
Selective filters widen ef then fall back to the scan; batch answers equal
single-query answers. Never persisted; IVF and the default exact path are
unchanged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Each probed cell's int8 rows are scored once against every query that probes
it, into per-query k*rerank_factor heaps, then each query is reranked exactly;
answers stay identical to the single-query path (tested with a rare-type filter
that forces the short-query fallback). IVF int8 single-query threading now uses
the same thresholds as f32.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Exact quantisation rule, search/rerank contract, invariant, and measured
results with the honest limits; byte-exact snapshot layout with a worked
example, identity rules, fallback rules and what is deliberately not stored.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…rch default

examples/hnsw_bench.rs sweeps ef_search (recall@10 against QPS, build time,
memory, write cost) on the benchmarks/competitive data or synthetic clusters,
with an optional IVF sweep on the same data. DESIGN.md section 10 specifies the
index so the other ports can implement it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

# Conflicts:
#	isongraph-vector-rs/DESIGN.md
…ment HNSW, int8 and snapshot

Rust exact search is at parity with numpy/faiss Flat, not ahead. HNSW matches
faiss HNSW and hnswlib single-thread; int8 and the binary snapshot help as
measured. Document that int8 is not used by HNSW graph search.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
maheshvaikri-code and others added 3 commits November 16, 2026 22:24
Rust: the loop-index and chunks_exact lints added in 1.98 (library and the
example binaries). C++: keep one out-of-line cosineSimilarity so the exact
scan, the approximate re-score and the tests cannot be optimised into
last-bit-different copies (5 ann_* tests failed on Ubuntu 26.04 / GCC 15.2
only; macOS, Windows and GCC 13 passed).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Linux and macOS clippy (-D warnings) flagged Pmc as dead code. Checked
against the Linux and macOS targets with Rust 1.98.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The Apple Silicon runner flagged an unused variable and field on the portable
path. Clippy is now clean for aarch64 and x86_64 on Linux and macOS (Rust 1.98).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@maheshvaikri-code
maheshvaikri-code merged commit a7a2330 into main Nov 17, 2026
30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants