Repository navigation
Soak at full scale: text vs bigint ids, 10M nodes / 100M edges (M10) - #62
Merged
Merged
Conversation
Case a at 10M nodes and 100M edges with IRIs and with bigint ids, every stage included, on an otherwise idle machine. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012Rxfe2L3iVjSwSBPdUu1Ke
Replaces the half-scale paragraph's 'still to do' with the 10M-node run beside its bigint pair, and explains the components and minimum_spanning_forest peaks: every row's text datum is built up front in the call's multi-call memory context, 72 bytes an id. Describes the lazy-rows fix without building it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012Rxfe2L3iVjSwSBPdUu1Ke
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Case a at 10M nodes / 100M edges with
--ids textand--ids biginton the same machine, every stage included (recursive CTE baseline, second version with diff/distance, the library, louvain and leiden).bench/results/soak/a-text-m10.json,a-m10.jsonand the README provenance.componentsandminimum_spanning_forestpeak higher with text (every row's text datum is built in the SRF's multi-call context, about 72 bytes per id), with the fix described (return rows lazily) but not implemented.Text costs 1.0–1.4× on most stages, about 2× on
bfsandcomponents, and 4.6× onsimilarity. Targets missed:reachable_set(13M / 19M edges/s against 50M),bfsvs the CTE (5.2× / 10.5× against 20×), and textcomponents(10.6 s against 10 s).Generated by Claude Code