diff --git a/.agents/AGENTS.md b/.agents/AGENTS.md deleted file mode 100644 index d96cd03698..0000000000 --- a/.agents/AGENTS.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -name: codegraph-orchestrator -description: Global coordination and structural codebase navigation via CodeGraph MCP -trigger: auto -paths: - - "**/*" ---- - -# CodeGraph System Instructions - -This project is backed by a custom **CodeGraph MCP server**. CodeGraph maintains a local Tree-sitter knowledge graph encompassing every symbol, edge, boundary, and file within this workspace. Reads operate at sub-millisecond speeds and deliver accurate structural insights that traditional text-based tools (like grep) cannot match. - ---- - -## 🚨 CRITICAL CONSTRAINTS (Read First) - -- **NEVER use generic text-search, grep, or file-reading tools** if a symbol, reference, or definition can be located using CodeGraph. -- **DO NOT double-check or re-verify** CodeGraph results with native file reads. Treat the knowledge graph as the absolute, single source of truth for codebase architecture. -- **Handle uninitialized states immediately:** If any CodeGraph tool returns a `"not initialized"` or missing index error, **STOP execution immediately** and instruct the user to run `codegraph init -i` in their terminal. Do not attempt to parse or scan the codebase manually to compensate. -- **Minimize token overhead:** Prefer targeted structural queries over dumping entire file contents into the context window. - ---- - -## 🛠️ Tool Selection Guide - -Always prefer `codegraph` tools for **structural** questions — tracing call hierarchies, mapping dependencies, determining definitions, and verifying signatures. Use standard filesystem tools *only* for literal text queries or applying actual code edits. - -| Intent / Question | Recommended MCP Tool | -| :--- | :--- | -| *"Where is symbol X defined?"* | `codegraph_search_symbol` | -| *"What callers invoke function Y?"* | `codegraph_callers` | -| *"What methods or functions does Y call?"* | `codegraph_callees` | -| *"What components or files will break if I modify Z?"* | `codegraph_impact` | -| *"Show me Y's exact signature and internal block"* | `codegraph_symbol` | -| *"Give me focused, aggregated context for this task"* | `codegraph_context` | -| *"What files exist under a specific path/ directory?"* | `codegraph_graphcode_files` | -| *"Is the local knowledge graph healthy and active?"* | `codegraph_status` | - ---- - -## 💡 Rules of Thumb & Workflows - -### 1. Unified Context Gathering -Do not chain manual searches and individual node inspections yourself. **`codegraph_context` is designed to perform aggregate lookups in a single call.** Execute it first when onboarding onto a new task or analyzing a localized bug. - -### 2. Defensive Token Preservation -Before updating an API route, a UI component, or a system utility, query `codegraph_impact` to pinpoint downstream effects. This ensures you only request and modify files strictly relevant to the current objective, preventing context window saturation. - -### 3. Strict AST Reliance -Because CodeGraph parses the Abstract Syntax Tree (AST), its structural insights are guaranteed. If a symbol look-up yields no results, assume the symbol does not exist in the current active workspace index. \ No newline at end of file diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000000..18f628da1f --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,99 @@ +# AGENTS.md — CodeGraph + +Rust workspace: tree-sitter parse → semantic graph (global symbol ID + call chain) trong `.codegraph/`, +phục vụ AI agent qua MCP; CLI `codegraph`. 15 crate, binary duy nhất. + +## Bắt buộc: CodeGraph MCP là công cụ ĐẦU TIÊN + +Trước khi grep/read để hiểu code, gọi MCP: + +| Câu hỏi | Tool | +|---|---| +| Symbol ở đâu / signature | `codegraph_search_symbol`, `codegraph_symbol` | +| Ai gọi / gọi ai | `codegraph_callers`, `codegraph_callees` | +| Sửa X ảnh hưởng gì | `codegraph_impact` | +| Luồng điều khiển / trace | `codegraph_flow`, `codegraph_trace_full_flow` | +| Tìm theo marker | `codegraph_search_flow` | +| Context gói gọn | `codegraph_context` | +| Review diff | `codegraph_graphcode_diff` | +| Bind / re-index | `codegraph_init` (index mặc định **false**), `codegraph_index` | + +- Index là ảnh chụp. Vừa sửa code mà chưa `codegraph_index` → kết quả cũ; đọc file để xác nhận. +- Grep/read vẫn đúng cho: text literal, `docs/`, `config.toml`, file ngoài index. +- Search lớn: `timeout_ms` + retry đúng args kèm `resume`. +- Đừng tin số liệu trong docs (tool count, version, language count) — đọc source. + +## Build & verify + +```bash +cargo build --workspace +cargo test --workspace +cargo clippy --workspace --all-targets -- -D warnings # gate CI +cargo fmt --all +``` + +- Toolchain: `codegraph-graph`, `codegraph-docs`, `codegraph-bench` dùng **edition 2024** → cần Rust ≥ 1.85. + `rust-version = "1.80"` trong `Cargo.toml` là số sai, đừng tin. +- `web/` là UI Svelte (npm) — chỉ cần khi sửa UI: `scripts/build-web.sh` sinh + `crates/codegraph-web/assets/` (không commit) rồi `rust-embed` nhúng lúc compile. +- Binary `codegraph` mà MCP host đang chạy là bản **đã cài**, không phải `target/debug`. Sửa tool surface + (`crates/codegraph-mcp/src/tools.rs`) mà không rebuild/reinstall thì agent vẫn thấy tool cũ. + +## Test quirks + +- `cargo test -p codegraph` chạy **0 test** — CLI chưa có test. Đừng coi đó là đã verify. +- Backend cần service thật + env: + `TEST_REDIS_DSN=redis://127.0.0.1:6379` (feature `redis`); + `TEST_RDBMS_DSN=...` + `TEST_RDBMS_REPO_ID=1` (feature `postgres`/`mysql`). + Chạy `#[ignore]`: `cargo test -p codegraph-graph --features postgres --test rdbms -- --ignored`. + Lưu ý: 4 test `storage::redis::tests::*` trong `crates/codegraph-graph/src/storage/redis.rs` KHÔNG `#[ignore]`, + hardcode `redis://127.0.0.1:6379/15` → fail `Connection refused` nếu không có redis-server (pre-existing, + không phải port contention). Chạy lại standalone để xác nhận. +- `crates/codegraph-binary/tests/extract.rs` cần `radare2` trên PATH (`--ignored`). +- `crates/codegraph-mcp/src/response_tests.rs` assert số tool đăng ký — thêm/bớt tool phải sửa số đó. +- Test config/flow cần `install_current(...)` và reset `install_current(None)`; classifier là thread-local. + +## Quy trình verify + PR (bắt buộc, đúng thứ tự) + +1. `cargo fmt --all` +2. `cargo clippy --workspace --all-targets -- -D warnings` + - storage feature-gated: `cargo clippy -p codegraph-graph --features postgres,mysql,redis --tests -- -D warnings` +3. `cargo test --workspace` — **toàn bộ** unit + integration (không dùng `--lib` để kết luận) +4. Regression đúng vùng vừa sửa: + - extract/config/marker → `cargo test -p codegraph-extract --test effects_config --test chains --test dynamic_dispatch` + - trace/sandbox → `cargo test -p codegraph-graph --test trace_flow_tests` + `cargo test -p codegraph-sboxes` + - backend → `cargo test -p codegraph-graph --features sqlite,lmdb` + - lưu ý: 4 test `storage::redis` fail khi chạy song song toàn workspace (port contention, pre-existing) → chạy lại standalone để xác nhận. +5. Tạo nhánh → commit (conventional commits) → mở PR. +6. Chờ GitHub Actions xanh (test matrix + clippy + web build). +7. Chờ review benchmark (Criterion/CodSpeed) và review Codecov — coverage không được giảm. +8. Chỉ merge sau khi 6–7 xanh. + +## Gotchas kiến trúc + +- Config chỉ có **một bảng** `[[effect_rules]]`: mỗi rule đặt `effect` và/hoặc `marker` (≥1). + `[[flow_rules]]` và `codegraph_config_flow` đã gộp/xoá; key cũ bị bỏ qua + warn. +- `CALL_ENTER` đọc `chain[pos+1]` làm callee → không chèn marker vào giữa `CALL_ENTER…CALL_EXIT`. +- Marker/ID là contract serialize ổn định — không đổi số cũ (`crates/codegraph-core/src/semgraph.rs`). +- `.codegraph/` không commit. `codegraph clean` xoá index nhưng giữ `config.toml`; `deinit` xoá cả thư mục. +- Thêm language: module trong `crates/codegraph-extract/src/languages/` + đăng ký `languages.rs` + + feature `lang-` + fixture/test. + +## Nguồn sự thật + +| Thứ | File | +|---|---| +| MCP tool + schema | `crates/codegraph-mcp/src/tools.rs` | +| Config + validation | `crates/codegraph-extract/src/config.rs` | +| Marker/ID/type lõi | `crates/codegraph-core/src/semgraph.rs` | +| Parser registry | `crates/codegraph-extract/src/languages.rs` | +| CI thật (feature/service) | `.github/workflows/ci.yml` | + +## Chi tiết + +- [MCP workflow](docs/agent/mcp-workflow.md) +- [Rust conventions](docs/agent/conventions.md) +- [Testing](docs/agent/testing.md) +- [Extraction & config](docs/agent/extraction.md) +- [Git & release](docs/agent/git-release.md) +- [Architecture](docs/architecture.md) · [Development](docs/development.md) · [Configuration](docs/configuration.md) diff --git a/crates/codegraph-core/src/lib.rs b/crates/codegraph-core/src/lib.rs index 1feabca0c1..038b19bd4a 100644 --- a/crates/codegraph-core/src/lib.rs +++ b/crates/codegraph-core/src/lib.rs @@ -13,12 +13,12 @@ pub use route::StorageRoute; pub use semgraph::{ is_marker, marker_id, marker_name, Annotation, BranchLabel, CallRecord, CallSite, CallSiteResult, ClassInfo, DbStats as SemgraphStats, DependenciesReport, Dependency, - EffectCallPattern, EffectRule, EffectType, FileInfo, FlowCall, FlowResult, FlowStep, - FullFlowPath, FunctionScope, Marker, MemberInfo, PathEnding, ResolutionKind, ResolveResult, - ScopeLevel, SearchFlowResult, Symbol, SymbolId, SymbolKind, SymbolMatch, TraceAction, - MARKER_AWAIT, MARKER_BRANCH_END, MARKER_BREAK, MARKER_CALL_ENTER, MARKER_CALL_EXIT, - MARKER_CATCH_START, MARKER_CONTINUE, MARKER_DEFER, MARKER_GO, MARKER_IF_FALSE, MARKER_IF_TRUE, - MARKER_LOOP, MARKER_LOOP_BACK, MARKER_REC_CALL, MARKER_RETURN, MARKER_STMT_END, - MARKER_SWITCH_CASE, MARKER_SWITCH_CLOSE, MARKER_SWITCH_END, MARKER_SWITCH_START, MARKER_THROW, - MARKER_TRY_END, MARKER_TRY_START, SYMBOL_BASE, + EffectCallPattern, EffectRule, EffectType, FileInfo, FlowCall, FlowMarker, FlowResult, + FlowStep, FullFlowPath, FunctionScope, Marker, MemberInfo, PathEnding, ResolutionKind, + ResolveResult, ScopeLevel, SearchFlowResult, Symbol, SymbolId, SymbolKind, SymbolMatch, + TraceAction, MARKER_AWAIT, MARKER_BRANCH_END, MARKER_BREAK, MARKER_CALL_ENTER, + MARKER_CALL_EXIT, MARKER_CATCH_START, MARKER_CONTINUE, MARKER_DEFER, MARKER_GO, + MARKER_IF_FALSE, MARKER_IF_TRUE, MARKER_LOOP, MARKER_LOOP_BACK, MARKER_REC_CALL, MARKER_RETURN, + MARKER_STMT_END, MARKER_SWITCH_CASE, MARKER_SWITCH_CLOSE, MARKER_SWITCH_END, + MARKER_SWITCH_START, MARKER_THROW, MARKER_TRY_END, MARKER_TRY_START, SYMBOL_BASE, }; diff --git a/crates/codegraph-core/src/semgraph.rs b/crates/codegraph-core/src/semgraph.rs index 96279bf474..edf53f2851 100644 --- a/crates/codegraph-core/src/semgraph.rs +++ b/crates/codegraph-core/src/semgraph.rs @@ -409,13 +409,112 @@ pub enum EffectCallPattern { } /// Một effect rule từ `[[effect_rules]]` trong config.toml. +/// Một rule có thể khai báo `effect`, `marker`, hoặc cả hai (≥1 trong hai): +/// - `effect` — tác động bên ngoài (sql_query, http_call, panic, …). +/// - `marker` — vai trò control-flow (try_start, throw, if_true, …). #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] pub struct EffectRule { #[serde(rename = "call")] pub call: EffectCallPattern, - pub effect: EffectType, + /// `None` = rule không khai báo effect (chỉ marker). `Some(None)` vẫn dùng + /// để shadow một bảng default (ví dụ `effect = "none"` chặn default sql_write). + #[serde(default)] + pub effect: Option, + /// Vai trò control-flow (None = rule effect thuần). + #[serde(default)] + pub marker: Option, + /// Nhãn exception / nhánh nhánh — hợp lệ với `throw` | `catch_start` | `if_true`. + #[serde(default)] + pub label: Option, + /// `true` → vẫn ghi CallRecord (throw-er là call thật); `false` → chỉ emit + /// marker (macro không phải call thật). Default: throw=true, còn lại=false. + #[serde(default)] + pub keep_call: Option, +} + +impl EffectRule { + /// Default `keep_call` theo marker: `throw` = call thật (giữ CallRecord), + /// các marker khác = macro (chỉ marker). + pub fn keep_call_or_default(&self) -> bool { + self.keep_call + .unwrap_or(self.marker == Some(FlowMarker::Throw)) + } +} + +/// Marker control-flow mà một flow rule được map vào — whitelist các marker +/// "điều khiển" (`[[effect_rules]]` báo macro/magic call đóng vai trò này). +/// Không cho phép marker call-edge (`CALL_ENTER`/`CALL_EXIT`) → không phá được +/// call graph. Các variant đều đã có sẵn `Marker` (serialization ổn định). +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)] +#[serde(rename_all = "snake_case")] +pub enum FlowMarker { + /// Mở khối try — interpreter push `catch_stack` tại vị trí này. + TryStart, + /// Đóng khối try gần nhất — pop `catch_stack`. + TryEnd, + /// Handler catch trong khối try; không label → match mọi exception. + CatchStart, + /// Throw — interpreter nhảy tới handler khớp, hoặc unwind lên caller. + Throw, + /// Nhánh nếu-đúng của macro if. + IfTrue, + /// Nhánh nếu-sai / else của macro if. + IfFalse, + /// Đóng khối nhánh của macro if/endif. + BranchEnd, + /// Break của macro loop. + Break, + /// Continue của macro loop. + Continue, + /// Mở loop của macro loop. + Loop, + /// Đóng loop (quay lại LOOP tương ứng). + LoopBack, + /// Return của macro return sớm. + Return, +} + +impl FlowMarker { + pub fn as_str(self) -> &'static str { + match self { + Self::TryStart => "try_start", + Self::TryEnd => "try_end", + Self::CatchStart => "catch_start", + Self::Throw => "throw", + Self::IfTrue => "if_true", + Self::IfFalse => "if_false", + Self::BranchEnd => "branch_end", + Self::Break => "break", + Self::Continue => "continue", + Self::Loop => "loop", + Self::LoopBack => "loop_back", + Self::Return => "return", + } + } + + /// Parse snake_case (case-insensitive) — chuỗi không biết → `None`. + pub fn parse(s: &str) -> Option { + Some(match s.trim().to_ascii_lowercase().as_str() { + "try_start" => Self::TryStart, + "try_end" => Self::TryEnd, + "catch_start" => Self::CatchStart, + "throw" => Self::Throw, + "if_true" => Self::IfTrue, + "if_false" => Self::IfFalse, + "branch_end" => Self::BranchEnd, + "break" => Self::Break, + "continue" => Self::Continue, + "loop" => Self::Loop, + "loop_back" => Self::LoopBack, + "return" => Self::Return, + _ => return None, + }) + } } +// Xoá `FlowRule` riêng — logic dùng chung gộp vào `EffectRule` (xem plan +// effect-rules-control-flow-upgrade.md). + // ==================== Entities ==================== /// Id global của symbol (u64, bắt đầu từ `SYMBOL_BASE`). diff --git a/crates/codegraph-extract/src/config.rs b/crates/codegraph-extract/src/config.rs index ddef8b7f08..6c92434857 100644 --- a/crates/codegraph-extract/src/config.rs +++ b/crates/codegraph-extract/src/config.rs @@ -3,6 +3,7 @@ use codegraph_source::DiskSource; use serde::Deserialize; use std::fs; +use crate::languages::flows::FlowClassifier; #[cfg(feature = "binary")] use codegraph_binary::BinaryConfig; use codegraph_core::{EffectCallPattern, EffectRule, EffectType, StorageRoute}; @@ -61,6 +62,9 @@ struct ConfigFile { #[serde(default)] languages: LanguagesSection, /// Project extra effect rules — xét trước bảng default (override). + /// Mỗi rule có thể khai báo `effect`, `marker`, hoặc cả hai (≥1 trong hai). + /// Key `[[flow_rules]]` đã gộp vào `[[effect_rules]]` — key cũ bị bỏ qua + + /// warn trong `load_from`. #[serde(default)] effect_rules: Vec, /// Backend storage (mặc định sqlite). @@ -206,6 +210,9 @@ struct EmbeddingSection { // trong .codegraph/config.toml bằng toml_edit (giữ nguyên mọi section khác). // Orchestrator install_current lại config MỖI lần index → rule mới áp dụng // từ lần index kế tiếp (đồng thời install ngay cho thread hiện tại). +// Một rule có thể khai báo `effect`, `marker`, hoặc cả hai (≥1 trong hai); +// effect classifier và flow classifier được build từ cùng list, lọc riêng rẽ +// theo từng dimension (first-match-wins trên mỗi dimension). /// Một rule trong file — dạng raw để list trung thực cả rule không hợp lệ. #[derive(Debug, Clone)] @@ -214,6 +221,9 @@ pub struct EffectRuleView { pub matcher: String, pub text: String, pub effect: String, + pub marker: String, + pub label: String, + pub keep_call: String, } fn effect_config_path(root: &Utf8Path) -> Utf8PathBuf { @@ -248,35 +258,84 @@ pub fn list_effect_rules(root: &Utf8Path) -> Vec { .and_then(|e| e.as_str()) .unwrap_or_default() .to_string(); + let marker = t + .get("marker") + .and_then(|e| e.as_str()) + .unwrap_or_default() + .to_string(); + let label = t + .get("label") + .and_then(|e| e.as_str()) + .unwrap_or_default() + .to_string(); + let keep_call = t + .get("keep_call") + .and_then(|e| e.as_bool()) + .map(|b| b.to_string()) + .unwrap_or_default(); out.push(EffectRuleView { matcher, text, effect, + marker, + label, + keep_call, }); } } out } -/// Thêm rule (prepend vị trí cuối) + áp dụng ngay cho thread hiện tại. -/// Rule config được xét TRƯỚC bảng default (first-match-wins). -pub fn add_effect_rule( - root: &Utf8Path, - matcher: &str, - text: &str, - effect: &str, -) -> Result<(), String> { - if !matches!(matcher, "contains" | "prefix" | "exact") { +/// Thêm rule (append cuối — index order = thứ tự match) + áp dụng ngay cho +/// thread hiện tại. Rule config được xét TRƯỚC bảng default (first-match-wins +/// trên từng dimension). +pub struct NewEffectRule<'a> { + pub matcher: &'a str, + pub text: &'a str, + pub effect: Option<&'a str>, + pub marker: Option<&'a str>, + pub label: Option<&'a str>, + pub keep_call: Option, +} + +pub fn add_effect_rule(root: &Utf8Path, rule: NewEffectRule<'_>) -> Result<(), String> { + if !matches!(rule.matcher, "contains" | "prefix" | "exact") { return Err(format!( - "matcher phải là contains|prefix|exact, nhận được {matcher:?}" + "matcher phải là contains|prefix|exact, nhận được {:?}", + rule.matcher )); } - if codegraph_core::EffectType::parse(effect).is_none() { - return Err(format!( - "effect không hợp lệ {effect:?} — dùng snake_case: panic, sql_query, http_call, …" - )); + if let Some(eff) = rule.effect { + if codegraph_core::EffectType::parse(eff).is_none() { + return Err(format!( + "effect không hợp lệ {:?} — dùng snake_case: panic, sql_query, http_call, …", + eff + )); + } + } + if let Some(mk) = rule.marker { + if codegraph_core::FlowMarker::parse(mk).is_none() { + return Err(format!( + "marker không hợp lệ {:?} — dùng snake_case: try_start, try_end, catch_start, throw, if_true, if_false, branch_end, break, continue, loop, loop_back, return", + mk + )); + } + } + if rule.effect.is_none() && rule.marker.is_none() { + return Err("phải khai báo ít nhất một trong `effect` hoặc `marker`".into()); + } + if rule.label.is_some() + && !matches!( + rule.marker, + Some("throw") | Some("catch_start") | Some("if_true") + ) + { + return Err("label chỉ hợp lệ với marker = throw | catch_start | if_true".into()); + } + if rule.keep_call.is_some() && rule.marker.is_none() { + return Err("keep_call chỉ hợp lệ khi có marker".into()); } - if text.trim().is_empty() { + if rule.text.trim().is_empty() { return Err("text matcher rỗng".into()); } let path = effect_config_path(root); @@ -292,15 +351,27 @@ pub fn add_effect_rule( .as_array_of_tables_mut() .ok_or("effect_rules phải là array-of-tables")?; let mut call = toml_edit::Table::new(); - call.insert(matcher, toml_edit::value(text)); - let mut rule = toml_edit::Table::new(); - rule.insert("call", toml_edit::Item::Table(call)); - rule.insert("effect", toml_edit::value(effect)); - arr.push(rule); + call.insert(rule.matcher, toml_edit::value(rule.text)); + let mut rule_table = toml_edit::Table::new(); + rule_table.insert("call", toml_edit::Item::Table(call)); + if let Some(eff) = rule.effect { + rule_table.insert("effect", toml_edit::value(eff)); + } + if let Some(mk) = rule.marker { + rule_table.insert("marker", toml_edit::value(mk)); + } + if let Some(l) = rule.label.filter(|l| !l.trim().is_empty()) { + rule_table.insert("label", toml_edit::value(l)); + } + if let Some(kc) = rule.keep_call { + rule_table.insert("keep_call", toml_edit::value(kc)); + } + arr.push(rule_table); std::fs::write(&path, doc.to_string()).map_err(|e| e.to_string())?; // Áp dụng ngay cho thread hiện tại (orchestrator tự reload mỗi index). - let classifier = ExtractConfig::load(root).effect_classifier; - crate::languages::effects::install_current(Some(std::sync::Arc::new(classifier))); + let cfg = ExtractConfig::load(root); + crate::languages::effects::install_current(Some(std::sync::Arc::new(cfg.effect_classifier))); + crate::languages::flows::install_current(Some(std::sync::Arc::new(cfg.flow_classifier))); Ok(()) } @@ -319,26 +390,41 @@ pub fn remove_effect_rule(root: &Utf8Path, index: usize) -> Result } arr.remove(index); std::fs::write(&path, doc.to_string()).map_err(|e| e.to_string())?; - let classifier = ExtractConfig::load(root).effect_classifier; - crate::languages::effects::install_current(Some(std::sync::Arc::new(classifier))); + let cfg = ExtractConfig::load(root); + crate::languages::effects::install_current(Some(std::sync::Arc::new(cfg.effect_classifier))); + crate::languages::flows::install_current(Some(std::sync::Arc::new(cfg.flow_classifier))); Ok(true) } -/// Raw rule — `effect` để string để rule lỗi (unknown) bị skip + warn, không -/// làm hỏng toàn bộ config; parse lại bằng `EffectType::parse`. +/// Raw rule — `effect`/`marker` để string để rule lỗi (unknown) bị skip + +/// warn, không làm hỏng toàn bộ config. ≥1 trong hai phải có giá trị. +/// Parse lại bằng `EffectType::parse` / `FlowMarker::parse`. #[derive(Debug, Deserialize)] struct EffectRuleRaw { #[serde(rename = "call")] call: EffectCallPattern, - effect: String, + #[serde(default)] + effect: Option, + #[serde(default)] + marker: Option, + #[serde(default)] + label: Option, + #[serde(default)] + keep_call: Option, } +// Xoá `FlowRuleRaw` riêng — logic dùng chung gộp vào `EffectRuleRaw` (xem +// plan effect-rules-control-flow-upgrade.md). + /// Project-level extraction settings (`.codegraph/config.toml`). #[derive(Debug, Clone, Default)] pub struct ExtractConfig { pub header_language: HeaderLanguage, /// Classifier effect của project — config rules override bảng default. pub effect_classifier: EffectClassifier, + /// Classifier flow (macro → marker) — KHÔNG có default: config trống → + /// mọi call map `None`. LLM khai báo qua MCP `codegraph_config_effects`. + pub flow_classifier: crate::languages::flows::FlowClassifier, /// Backend storage được chọn trong config (mặc định sqlite). pub storage: StorageConfig, /// Cấu hình embedding backend (semantic search) — đọc từ `[embedding]`. @@ -397,9 +483,15 @@ impl ExtractConfig { let Ok(file) = toml::from_str::(&text) else { return Self::default(); }; + // Key `[[flow_rules]]` cũ đã gộp vào `[[effect_rules]]`. + if text.contains("flow_rules") { + tracing::warn!("`[[flow_rules]]` đã gộp vào `[[effect_rules]]` — rule cũ bị bỏ qua"); + } + let (effect_classifier, flow_classifier) = build_classifiers(file.effect_rules); Self { header_language: parse_header_language(file.languages.headers.as_deref()), - effect_classifier: build_classifier(file.effect_rules), + effect_classifier, + flow_classifier, embedding: codegraph_graph::embeddings::EmbeddingConfig::from_raw( file.embedding.backend.as_deref(), file.embedding.model.as_deref(), @@ -662,23 +754,51 @@ fn split_format_override(entry: &str) -> (&str, Option<&str>) { (entry, None) } -/// Setup rule config → skip rule effect unknown (warn) + giữ phần còn lại. -fn build_classifier(raw: Vec) -> EffectClassifier { - let mut rules = Vec::with_capacity(raw.len()); +/// Setup rule config → skip rule effect/marker unknown (warn) + giữ phần còn +/// lại. `effect`/`marker` là hai dimension độc lập: rule có effect sai vẫn +/// giữ marker đúng và ngược lại. Config trống → classifier rỗng (`classify_flow` +/// → `None`, `classify_effect` chỉ có bảng default). +fn build_classifiers(raw: Vec) -> (EffectClassifier, FlowClassifier) { + let mut effect_rules = Vec::with_capacity(raw.len()); + let mut flow_rules = Vec::with_capacity(raw.len()); for r in raw { - let Some(effect) = EffectType::parse(&r.effect) else { - tracing::warn!( - "[[effect_rules]]: unknown effect `{}`, rule ignored", - r.effect - ); - continue; - }; - rules.push(EffectRule { - call: r.call, - effect, - }); + // ── effect dimension ── + if let Some(s) = r.effect.as_deref().map(str::trim).filter(|s| !s.is_empty()) { + match EffectType::parse(s) { + Some(effect) => effect_rules.push(EffectRule { + call: r.call.clone(), + effect: Some(effect), + marker: None, + label: None, + keep_call: None, + }), + None => tracing::warn!( + "[[effect_rules]]: unknown effect `{s}` — effect dimension ignored" + ), + } + } + // ── marker dimension ── + if let Some(s) = r.marker.as_deref().map(str::trim).filter(|s| !s.is_empty()) { + match codegraph_core::FlowMarker::parse(s) { + Some(marker) => flow_rules.push(EffectRule { + call: r.call, + effect: None, + marker: Some(marker), + label: r.label.clone(), + keep_call: r.keep_call, + }), + None => tracing::warn!( + "[[effect_rules]]: unknown marker `{s}` — marker dimension ignored" + ), + } + } else if r.label.is_some() || r.keep_call.is_some() { + tracing::warn!("[[effect_rules]]: label/keep_call without marker — ignored"); + } } - EffectClassifier::with_config(rules) + ( + EffectClassifier::with_config(effect_rules), + FlowClassifier::with_config(flow_rules), + ) } fn parse_header_language(raw: Option<&str>) -> HeaderLanguage { @@ -699,8 +819,40 @@ pub const DEFAULT_CONFIG_TOML: &str = r#"# CodeGraph project configuration headers = "auto" # Critical effect rules — matched before the built-in defaults (first match wins). -# call matchers: prefix / contains / exact. Effects: sql_query, sql_write, -# cache_read, cache_write, http_call, event_emit, file_read, file_write, log. +# Rule có thể khai báo `effect`, `marker`, hoặc cả hai (≥1 trong hai). +# call matchers: prefix / contains / exact. +# effect: sql_query, sql_write, cache_read, cache_write, http_call, event_emit, +# file_read, file_write, log, panic (shadow default bằng "none"). +# marker: try_start, try_end, catch_start, throw, if_true, if_false, branch_end, +# break, continue, loop, loop_back, return. +# label (optional): exception / nhánh — chỉ với throw | catch_start | if_true. +# keep_call (optional): mặc định true cho throw, false cho các marker khác. +# Ví dụ PostgreSQL-style exception engine: +# [[effect_rules]] +# call = { exact = "PG_TRY" } +# marker = "try_start" +# [[effect_rules]] +# call = { exact = "PG_CATCH" } +# marker = "catch_start" +# [[effect_rules]] +# call = { exact = "PG_END_TRY" } +# marker = "try_end" +# [[effect_rules]] +# call = { contains = "ereport" } +# effect = "panic" +# marker = "throw" +# Ví dụ macro if/else/endif tự chế: +# [[effect_rules]] +# call = { exact = "IF" } +# marker = "if_true" +# label = "..." +# [[effect_rules]] +# call = { exact = "ELSE" } +# marker = "if_false" +# [[effect_rules]] +# call = { exact = "ENDIF" } +# marker = "branch_end" +# Ví dụ rule chỉ effect (như trước): # [[effect_rules]] # call = { prefix = "db." } # effect = "sql_query" @@ -855,8 +1007,30 @@ fn effect_rule_crud_roundtrip() { crate::project::init_project(&root).unwrap(); assert!(list_effect_rules(&root).is_empty()); // add: CHECK macro → panic (exception-like) - add_effect_rule(&root, "contains", "CHECK", "panic").unwrap(); - add_effect_rule(&root, "prefix", "db.", "sql_query").unwrap(); + add_effect_rule( + &root, + NewEffectRule { + matcher: "contains", + text: "CHECK", + effect: Some("panic"), + marker: None, + label: None, + keep_call: None, + }, + ) + .unwrap(); + add_effect_rule( + &root, + NewEffectRule { + matcher: "prefix", + text: "db.", + effect: Some("sql_query"), + marker: None, + label: None, + keep_call: None, + }, + ) + .unwrap(); let rules = list_effect_rules(&root); assert_eq!(rules.len(), 2); assert_eq!(rules[0].matcher, "contains"); @@ -867,7 +1041,18 @@ fn effect_rule_crud_roundtrip() { assert_eq!(classifier.classify("CHECK_OK").0, EffectType::Panic); assert_eq!(classifier.classify("db.Query").0, EffectType::SqlQuery); // invalidate: effect sai bị chặn, file không đổi - assert!(add_effect_rule(&root, "contains", "x", "nonsense").is_err()); + assert!(add_effect_rule( + &root, + NewEffectRule { + matcher: "contains", + text: "x", + effect: Some("nonsense"), + marker: None, + label: None, + keep_call: None, + }, + ) + .is_err()); assert_eq!(list_effect_rules(&root).len(), 2); // remove theo index assert!(remove_effect_rule(&root, 0).unwrap()); @@ -1118,4 +1303,171 @@ effect = "not_a_real_effect" assert_eq!(cfg.header_language, HeaderLanguage::Auto); assert_eq!(cfg.storage.kind, StorageKind::default()); } + + /// CRUD: add → list → remove → list. + #[test] + fn effect_rule_crud() { + let fx = CfgFixture::without_config(); + let root = fx.root(); + assert!(list_effect_rules(root).is_empty()); + + add_effect_rule( + root, + NewEffectRule { + matcher: "exact", + text: "PG_TRY", + effect: None, + marker: Some("try_start"), + label: None, + keep_call: None, + }, + ) + .unwrap(); + add_effect_rule( + root, + NewEffectRule { + matcher: "contains", + text: "ereport", + effect: Some("panic"), + marker: Some("throw"), + label: Some("Error"), + keep_call: Some(true), + }, + ) + .unwrap(); + + let rules = list_effect_rules(root); + assert_eq!(rules.len(), 2); + assert_eq!(rules[0].matcher, "exact"); + assert_eq!(rules[0].text, "PG_TRY"); + assert_eq!(rules[0].marker, "try_start"); + assert_eq!(rules[1].matcher, "contains"); + assert_eq!(rules[1].effect, "panic"); + assert_eq!(rules[1].marker, "throw"); + assert_eq!(rules[1].label, "Error"); + assert_eq!(rules[1].keep_call, "true"); + + assert!(remove_effect_rule(root, 0).unwrap()); + let rules = list_effect_rules(root); + assert_eq!(rules.len(), 1); + assert_eq!(rules[0].text, "ereport"); + + assert!(!remove_effect_rule(root, 99).unwrap()); + assert_eq!(list_effect_rules(root).len(), 1); + } + + /// Validation: phải có ≥1 trong effect/marker; label chỉ với throw|catch_start|if_true; + /// keep_call chỉ với marker; effect/marker/matcher phải hợp lệ; text không rỗng. + #[test] + fn effect_rule_validation() { + let fx = CfgFixture::without_config(); + let root = fx.root(); + + let bad = |rule: NewEffectRule<'_>| assert!(add_effect_rule(root, rule).is_err()); + + // Không có effect và marker → lỗi. + bad(NewEffectRule { + matcher: "exact", + text: "x", + effect: None, + marker: None, + label: None, + keep_call: None, + }); + // Label với marker không hợp lệ (branch_end) → lỗi. + bad(NewEffectRule { + matcher: "exact", + text: "x", + effect: None, + marker: Some("branch_end"), + label: Some("err"), + keep_call: None, + }); + // keep_call không có marker → lỗi. + bad(NewEffectRule { + matcher: "exact", + text: "x", + effect: None, + marker: None, + label: None, + keep_call: Some(true), + }); + // Effect không hợp lệ → lỗi. + bad(NewEffectRule { + matcher: "exact", + text: "x", + effect: Some("not_an_effect"), + marker: None, + label: None, + keep_call: None, + }); + // Marker không hợp lệ → lỗi. + bad(NewEffectRule { + matcher: "exact", + text: "x", + effect: None, + marker: Some("not_a_marker"), + label: None, + keep_call: None, + }); + // Matcher sai → lỗi. + bad(NewEffectRule { + matcher: "regex", + text: "x", + effect: Some("panic"), + marker: None, + label: None, + keep_call: None, + }); + // Text rỗng → lỗi. + bad(NewEffectRule { + matcher: "exact", + text: "", + effect: Some("panic"), + marker: None, + label: None, + keep_call: None, + }); + + // Hợp lệ: chỉ effect. + add_effect_rule( + root, + NewEffectRule { + matcher: "exact", + text: "panic", + effect: Some("panic"), + marker: None, + label: None, + keep_call: None, + }, + ) + .unwrap(); + // Hợp lệ: chỉ marker. + add_effect_rule( + root, + NewEffectRule { + matcher: "exact", + text: "PG_TRY", + effect: None, + marker: Some("try_start"), + label: None, + keep_call: None, + }, + ) + .unwrap(); + assert_eq!(list_effect_rules(root).len(), 2); + } + + /// Key cũ `[[flow_rules]]` bị bỏ qua (không đọc được như `[[effect_rules]]`). + #[test] + fn legacy_flow_rules_ignored() { + let fx = CfgFixture::with_config( + r#" +[[flow_rules]] +call = { exact = "PG_TRY" } +marker = "try_start" +"#, + ); + assert!(list_effect_rules(fx.root()).is_empty()); + } } diff --git a/crates/codegraph-extract/src/languages.rs b/crates/codegraph-extract/src/languages.rs index c10c3b9ed3..4e76fa0bf5 100644 --- a/crates/codegraph-extract/src/languages.rs +++ b/crates/codegraph-extract/src/languages.rs @@ -2,6 +2,7 @@ pub mod common; pub mod effects; +pub mod flows; #[cfg(feature = "lang-c")] pub mod c; diff --git a/crates/codegraph-extract/src/languages/common.rs b/crates/codegraph-extract/src/languages/common.rs index 84da77ec76..3f65c544ee 100644 --- a/crates/codegraph-extract/src/languages/common.rs +++ b/crates/codegraph-extract/src/languages/common.rs @@ -14,6 +14,8 @@ //! Kết quả là `codegraph_graph::ParseResult` — input của pipeline 2 phase. use crate::languages::effects::classify_effect; +use crate::languages::flows::{classify_flow, FlowHit}; +use codegraph_core::FlowMarker; use codegraph_core::{ Annotation, CallRecord, Result, ScopeLevel, Symbol, SymbolKind, MARKER_BRANCH_END, MARKER_BREAK, MARKER_CALL_ENTER, MARKER_CALL_EXIT, MARKER_CATCH_START, MARKER_CONTINUE, @@ -949,6 +951,7 @@ pub fn build_chain( func_id, chain: vec![func_id], calls: Vec::new(), + flow_try: Vec::new(), }; if let Some(body) = node.child_by_field_name(spec.markers.body_field) { walk_chain(&mut ctx, &body, 0, 0, None); @@ -964,8 +967,27 @@ struct ChainCtx<'a> { func_id: u64, chain: Vec, calls: Vec, + /// Trạng thái lifecycle của macro try/catch dùng chung mọi ngôn ngữ — + /// cho phép remap marker theo thứ tự statement (PG style: TRY_START, + /// body, [TRY_END, CATCH_START, handler, BRANCH_END]...). Stack hỗ trợ + /// try lồng nhau trong body / handler. + flow_try: Vec, } +/// Phase của một `TRY_START` đang mở trong function đang build. +#[derive(Clone, Copy, PartialEq, Eq, Debug)] +enum TryPhase { + /// Ở trong body try, chưa gặp handler. + Body, + /// Ở trong handler (sau `CATCH_START`, chưa đóng bằng `BRANCH_END`). + Handler, +} + +impl Default for ChainCtx<'_> { + fn default() -> Self { + unreachable!("ChainCtx is built manually in build_chain") + } +} fn walk_chain( ctx: &mut ChainCtx, node: &Node, @@ -1004,12 +1026,34 @@ fn walk_chain( // 1. Call sites. if let Some(rule) = ctx.spec.calls.iter().find(|r| r.kind == k) { + // Flow rule (`[[effect_rules]]`) — macro/magic call → marker: + // - keep_call=false (PG_TRY/PG_CATCH…): macro không phải call thật — + // chỉ emit marker lifecycle, KHÔNG wrap CALL_ENTER/EXIT (wrapper + // nuốt marker như callee, phá interpreter catch_stack). + // Tham số (vd IF(has_perm(u))) vẫn được walk TRƯỚC marker. + // - keep_call=true (ereport, longjmp…): emit call như thường, rồi + // THROW marker TRẦN sau CALL_EXIT — nếu đặt trong wrapper thì + // CALL_ENTER nuốt marker làm callee, THROW không bao giờ processing. + let flow_hit = macro_flow_hit(ctx, node, rule); + if let Some(hit) = &flow_hit { + if !hit.keep_call { + // Tham số của macro (nếu có) vẫn vào graph trước marker. + for ch in named_children(node) { + walk_chain(ctx, &ch, depth + 1, in_loop, None); + } + emit_flow_macro(ctx, hit); + return; + } + } chain_push(ctx, MARKER_CALL_ENTER); emit_call(ctx, node, rule, in_loop, condition); for ch in named_children(node) { walk_chain(ctx, &ch, depth + 1, in_loop, None); } chain_push(ctx, MARKER_CALL_EXIT); + if let Some(hit) = &flow_hit.filter(|h| h.keep_call) { + emit_flow_throw(ctx, hit); + } return; } @@ -1557,12 +1601,123 @@ fn exception_type_of(node: &Node, src: &[u8]) -> Option { (!t.is_empty()).then(|| t.to_string()) } -/// Gắn **nhãn nhánh** vào đúng vị trí marker trong chain (không thêm node): -/// thêm một `CallRecord` với `position = `, `condition = label`, -/// `call_name = ""`. Marker không bao giờ có call thật ở cùng position nên record -/// này tách hẳn khỏi call — không tạo edge, không lọt vào call-name index, và -/// `flow()` đọc lại thành `branch_labels` cho UI (`IF_TRUE: x > 0`, `LOOP: i < n`, -/// `SWITCH_CASE: Cmd::Init`). +// Gắn **nhãn nhánh** vào đúng vị trí marker trong chain (không thêm node): +// thêm một `CallRecord` với `position = `, `condition = label`, +// `call_name = ""`. Marker không bao giờ có call thật ở cùng position nên record +// này tách hẳn khỏi call — không tạo edge, không lọt vào call-name index, và +// `flow()` đọc lại thành `branch_labels` cho UI (`IF_TRUE: x > 0`, `LOOP: i < n`, +// `SWITCH_CASE: Cmd::Init`). +// ── Macro → marker lifecycle (`[[flow_rules]]`, docs/architecture.md) ── +// Macro try/catch đứng dạng statement tuần tự nên layout marker khác try +// node thật của ngôn ngữ: TRY_START(PG_TRY) → body → *TRY_END + CATCH_START* +// (PG_CATCH, remap!) → handler → *BRANCH_END* (PG_END_TRY, remap!). +// Interpreter hiện có giả định handler nằm ngay sau `TRY_END` +// (`handler_positions`/`skip_handlers`) — remap này thỏa được giả định đó. + +/// Flow hit cho macro tại call node — tính tên call giống `emit_call` rồi +/// classify. Gọi TRƯỚC khi wrap `CALL_ENTER/EXIT` vì wrapper nuốt marker như +/// callee, phá interpreter catch_stack. +fn macro_flow_hit(ctx: &ChainCtx, node: &Node, rule: &CallRule) -> Option { + let callee = if rule.callee_field.is_empty() { + named_children(node).into_iter().next() + } else { + node.child_by_field_name(rule.callee_field) + }; + let name = if let Some(f) = rule.name_fn { + f(node, ctx.src) + } else { + callee.as_ref().and_then(|c| text(c, ctx.src)) + }?; + if name.is_empty() { + return None; + } + classify_flow(&name) +} + +/// Emit marker cho macro **không phải call thật** (PG_TRY/PG_CATCH/…, +/// `keep_call = false`): chỉ đóng gói control-flow lifecycle, không CallRecord. +/// +/// State machine dùng stack `Vec` để hỗ trợ try lồng nhau trong +/// body / handler (PG có rất nhiều trường hợp này). LifecyclePG_CATCH remap: +/// đóng try body (`TRY_END` → interpreter skip handlers trong path bình +/// thường) rồi mới mở handler. `PG_END_TRY` → `BRANCH_END` đóng handler, hoặc +/// `TRY_END` đóng try không handler. +fn emit_flow_macro(ctx: &mut ChainCtx, hit: &FlowHit) { + match hit.marker { + FlowMarker::TryStart => { + chain_push(ctx, MARKER_TRY_START); + ctx.flow_try.push(TryPhase::Body); + } + FlowMarker::CatchStart => match ctx.flow_try.last() { + Some(TryPhase::Body) => { + // PG_CATCH remap: đóng try body, mở handler. + ctx.flow_try.pop(); + chain_push(ctx, MARKER_TRY_END); + chain_push(ctx, MARKER_CATCH_START); + let label = match &hit.label { + Some(l) => format!("catch {l}"), + None => "catch".to_string(), + }; + emit_branch_label(ctx, &label); + ctx.flow_try.push(TryPhase::Handler); + } + _ => { + // `catch_start` ngoài scope try — marker mồ côi, bỏ qua + // (tránh break `handler_positions` / `skip_handlers`). + tracing::debug!("flow marker `catch_start` ngoài try đang mở — bỏ qua"); + } + }, + FlowMarker::TryEnd => match ctx.flow_try.pop() { + Some(TryPhase::Handler) => { + // Đóng handler bằng BRANCH_END. + chain_push(ctx, MARKER_BRANCH_END); + } + Some(TryPhase::Body) => { + // Đóng try không handler. + chain_push(ctx, MARKER_TRY_END); + } + None => { + tracing::debug!("flow marker `try_end` không có try đang mở — bỏ qua"); + } + }, + FlowMarker::Throw => { + // Macro throw thuần (keep_call=false): chỉ marker + label. + chain_push(ctx, MARKER_THROW); + emit_branch_label(ctx, &throw_label(hit)); + } + FlowMarker::IfTrue => { + chain_push(ctx, MARKER_IF_TRUE); + if let Some(l) = &hit.label { + emit_branch_label(ctx, l); + } + } + FlowMarker::IfFalse => chain_push(ctx, MARKER_IF_FALSE), + FlowMarker::BranchEnd => chain_push(ctx, MARKER_BRANCH_END), + FlowMarker::Break => chain_push(ctx, MARKER_BREAK), + FlowMarker::Continue => chain_push(ctx, MARKER_CONTINUE), + FlowMarker::Loop => chain_push(ctx, MARKER_LOOP), + FlowMarker::LoopBack => chain_push(ctx, MARKER_LOOP_BACK), + FlowMarker::Return => chain_push(ctx, MARKER_RETURN), + } +} + +/// Nhãn của marker `throw`: rule có `label` → `throw