Make SavedGraph.Save portable so the module builds on Windows - #26
Open
suykerbuyk wants to merge 1 commit into
Open
suykerbuyk wants to merge 1 commit into
suykerbuyk wants to merge 1 commit into
Conversation
renameio.TempFile is built only on !windows, so any program importing github.com/coder/hnsw failed to compile for GOOS=windows. Write the graph to a temporary file in the same directory with os.CreateTemp, sync and close it, then os.Rename it over the target; remove the temporary file on any error. Drops the github.com/google/renameio dependency.
suykerbuyk
pushed a commit
to suykerbuyk/vibe-palace
that referenced
this pull request
Oct 3, 2026
… and an HNSW implementation Engine indexes are now a VectorIndex (Build, Insert, Search, Delete, Len, Close) built by newIndex, which refuses an unknown kind. Brute force is unchanged in its results; both implementations meet one contract test: duplicate ids in Build keep the last, k=0 is empty and k<0 an error, unusable vectors (zero, non-finite, or a float32 squared norm outside [2^-60, 2^60]) are skipped on store, delete on upsert and are an error as a query, a dimension mismatch is an error, an identical re-insert is a no-op, stored vectors are copies, and the brute-force ghost left by a duplicate-then-delete is gone. The HNSW implementation wraps a vendored coder/hnsw (third_party/coder-hnsw = upstream 36cab60 + vp.patch, which drops the Windows-breaking SavedGraph; upstream PR coder/hnsw#26 makes it portable): - graph keyed by internal ids behind an RWMutex, with tombstones; - a NaN-guarded cosine registered in init(); - searches widen until k live results, capped at the graph size; - a tombstone rebuild at 20% (minimum 64) on its own goroutine from a snapshot, catching up on concurrent writes off-lock and replaying at most 64 under the lock; a rebuild that cannot catch up is abandoned without a swap and backs off by x4 in writes; - Build cancels and Close cancels and waits for a running rebuild. The engine closes a replaced index after releasing its lock, on both the replace and the empty-corpus path, and Close closes every index before the embedder. Production still constructs brute force; HNSW is selected by a later task. make hnsw-check (goreleaser cross-build, vendor drift check, slow recall and cost tests that must PASS by name) runs in a new CI job; make fmt skips third_party. README carries the vendored code's CC0 notice.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SavedGraph.Savecallsrenameio.TempFile, which renameio builds only on!windows(renameio v1.0.1, tempfile.go). As a result, any program thatimports
github.com/coder/hnswfails to compile forGOOS=windows, even ifit never calls
SavedGraph:This PR replaces
renameio.TempFilewith the standard library. The graph iswritten to a temporary file in the same directory (
os.CreateTemp), flushed,synced and closed, and then renamed over the target (
os.Rename). On any errorthe temporary file is removed. Without a crash it behaves as before:
Saveeither replaces the file or leaves it untouched. It also drops thegithub.com/google/renameiodependency.Verified at 36cab60:
go vet ./...andgo test ./...pass on linux, includingTestSavedGraph.GOOS=windows go vet ./...andGOOS=windows go build ./...pass.gofmt -l .is clean.