Skip to content

[perf] Vermeer: compact triangle-count adjacency exchange - #32

Closed
imbajin wants to merge 2 commits into
masterfrom
fix/vermeer-triangle-memory-20261105
Closed

imbajin wants to merge 2 commits into
masterfrom
fix/vermeer-triangle-memory-20261105

Conversation

@imbajin

@imbajin imbajin commented Nov 5, 2026 •

Copy link
Copy Markdown

背景

triangle_count 当前把每个顶点的低 ID 邻居保存为 Go map,并在 step 1 后复制到每个 worker。大图上会形成“每个 worker 一份整图 map 对象”的高额对象数、桶空间与 GC 压力;step 2 结束后这些邻接结构还会一直保留到输出完成。

方案

Before
全局:[]map[neighbor]struct{}   (每个 worker 复制整图 map)
本地:[]map[neighbor]struct{}
step 2 后仍保留两套邻接工作区

After
全局:[]SliceUint32             (仅交换低 ID 紧凑邻接)
本地:[]map[neighbor]struct{}   (只为本地 membership lookup 保留)
step 2 后立即释放邻接工作区
  • 合并 in/out edges,并去除 self-loop 与平行重复边;
  • 只把低 ID 邻居作为紧凑 SliceUint32 跨 worker 交换;
  • 本地顶点继续使用 hash set 做 O(1) membership lookup;
  • 保持原有计数语义:对顶点 v 的每个邻居 f,检查 N(v) ∩ lower(f),每个 incident triangle 对 v 恰好计数一次;
  • 防御非法远端 vertex ID;
  • step 2 后主动释放全局低邻接和本地 hash set,避免输出阶段继续占用工作内存。

回归覆盖

  • 单个三角形的三个顶点均输出 1;
  • open wedge 不误计数;
  • in/out 混合方向仍按无向邻接识别三角形;
  • self-loop、平行边去重;
  • compact lower-slice membership;
  • 非法远端 ID 防御。

范围

后续性能边界

本 PR 优先消除确定性的 map 复制和 GC 放大,同时保持两步计算与原结果语义。按 degree/rank 定向可进一步降低高阶图交集次数,但需要额外 superstep 或新的交换结构,需在真实图上测量后单独设计。

ASF 上游将使用同一 head 分支同步提交。

ASF upstream: apache#379

@coderabbitai

coderabbitai Bot commented Nov 5, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 17 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 8ee20a82-98c3-43e6-bfaa-2f5222a00e29
Commits

Reviewing files that changed from the base of the PR and between d06a23b and 794c3f2.

Files selected for processing: 2

  • vermeer/algorithms/triangle_count.go
  • vermeer/algorithms/triangle_count_test.go
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-05T15:07:30.463122Z 794c3f2 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 794c3f2561

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +84 to +85
for _, candidate := range lowerNeighbors {
if _, exists := neighbors[candidate]; exists {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid scanning the larger adjacency for every intersection

For graphs with high-degree hubs, always scanning lowerNeighbors introduces a quadratic runtime regression. Consider a star with n leaves whose hub has the largest vertex ID: each leaf now scans all n entries in the hub's lower-neighbor slice, making step 2 perform Θ(n²) membership checks even though there are no triangles. The previous findCross scanned the smaller set, so the same graph required only Θ(n) checks. Preserve a smaller-side intersection strategy, for example by sorting the exchanged slices and using binary searches when the local neighbor set is smaller, so compact exchange does not make sparse hub-heavy graphs impractical to process.

Useful? React with 👍 / 👎.

imbajin commented Nov 5, 2026

Copy link
Copy Markdown
Author

已完成与 #30 的逐文件、算法语义和回归测试对比,保留 #30 作为主方案:

因此关闭本 PR,避免两套 triangle-count 实现并行维护;没有丢失独有测试或有效语义。

@imbajin imbajin closed this Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant