Conversation
Refactors core engine paths to use atomic MemTable pointers for lock-free reads, improves WAL/SSTable durability and error propagation, and adds stricter SSTable/Bloom validation constants. Updates network protocol/server behavior with clearer packet validation (new magic byte), safer request/response writes, configurable server startup flags, and graceful shutdown that stops the listener before closing the DB.
Adds `go vet ./...` to the build workflow to catch idiom and bug-prone issues earlier, and introduces a Go-focused `.gitignore` for binaries, coverage outputs, workspace files, and local WAL/SST artifacts. Updates engine tests to match current APIs/semantics: Bloom decode now checks returned errors, WAL tests use `OpPut` for realistic recovery paths, and corruption recovery assertions are tightened around torn-write/CRC behavior. The chaos script is also aligned with the current wire protocol by switching the request magic byte to `0xA1`.
Update `TestParseHeader` to check expected error types with `errors.Is` instead of direct equality. This makes the test resilient to wrapped errors while preserving the same failure expectations.
Adds stronger safety checks across the engine and network path. MemTables are now marked frozen during flush rotation, and DB Put/Delete use a lock-free retry loop so concurrent writes move to the new active table instead of writing into a table being flushed. SSTable/compaction write paths now propagate header/key/value write errors instead of ignoring them. The request handler also validates payload length against the buffer to prevent out-of-bounds reads on malformed input.
Tightens engine durability and concurrency behavior by surfacing write failures during SSTable/compaction finalization and by failing on truncated bloom-filter payloads instead of silently accepting partial data. It also fixes memtable flush rotation safety by introducing a lock-based `Freeze()` path, re-checking frozen state under lock in `Put`, and copying inserted keys to avoid aliasing issues. Tests and WAL recovery call sites were updated to handle `Put`’s error return explicitly.
This change fixes race-prone lifecycle handling in both storage and networking layers. Version refs now use a CAS-based acquire/purge protocol (including a purge sentinel) so readers cannot attach to a version being deleted, and DB reads retry until a safe acquire succeeds. The TCP server now guards listener access with a mutex and uses a stable local listener in the accept loop to avoid Stop/Start races. It also improves reliability by surfacing or logging previously ignored I/O errors: memtable flush on shutdown, WAL fsync, WAL recovery writes, SSTable flush/fsync, and protocol header read failures (while preserving raw EOF behavior for disconnect handling).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pre Release Cleaning