From c094f65a41bb17c43592f76f8558537ab526d67f Mon Sep 17 00:00:00 2001 From: tae2089 Date: Sat, 22 Aug 2026 01:49:06 +0900 Subject: [PATCH 01/10] docs(skills): allow bounded CCG pagination --- skills/ccg/SKILL.md | 27 ++++++++++++++++++--------- 1 file changed, 18 insertions(+), 9 deletions(-) diff --git a/skills/ccg/SKILL.md b/skills/ccg/SKILL.md index d9655c3..83ee5e4 100644 --- a/skills/ccg/SKILL.md +++ b/skills/ccg/SKILL.md @@ -2,7 +2,7 @@ name: ccg description: "Fast read-only code discovery with a bounded code-context-graph search and targeted source verification. Use when an ordinary positive lookup or explanation needs an entry point, recorded intent, known-path inventory, or direct relationship evidence. Do not use for absence, completeness, exhaustive inventory, deep flow or impact analysis, or graph writes; use ccg-search-verify for defensible negative claims, and require explicit invocation for ccg-analyze or ccg-build." metadata: - version: 3.0.0 + version: 3.0.1 openclaw: category: "code-intelligence" domain: "core" @@ -46,20 +46,27 @@ impact-analysis procedure. or the user. Do not call `get_minimal_context`, `list_namespaces`, or `list_graph_stats` for an ordinary search when that information is already known. -2. With an exact clue, use grep/read. Otherwise call `search` at most once with +2. With an exact clue, use grep/read. Otherwise start one `search` with `limit: 5`: - known thing: one short identifier or rare keyword; - unknown symbol: one concise plain-language question that can match recorded `@intent` or `@domainRule` reasons. -3. Verify the best credible candidate in one or two source ranges. Stop when +3. If the current page has no credible candidate and the result supplies a + continuation, follow that exact `next` call up to three times. Preserve the + query, limit, namespace, and continuation offsets supplied by CCG; do not + calculate offsets or reformulate the query. Stop paging as soon as one + credible candidate appears, `next` disappears, or the third continuation + has been read. +4. Verify the best credible candidate in one or two source ranges. Stop when those ranges answer the question. -4. Use `get_node`, `describe`, or one bounded `query_graph` call only when the +5. Use `get_node`, `describe`, or one bounded `query_graph` call only when the answer needs exact identity, an unranked known-path inventory, or one direct relationship fact. Every query word must occur in the same indexed document, so do not concatenate -the prompt’s examples into a long query. If the first query has no credible hit, -do not fan out into synonyms inside this fast workflow. +the prompt’s examples into a long query. If the search pages have no credible +hit, do not fan out into synonyms inside this fast workflow; use narrowly +targeted grep/read and do not make a negative claim. ```bash ccg search --limit 5 "" @@ -84,11 +91,13 @@ This skill is read-only. Report a missing or stale graph instead of invoking ## Response Budget Rule -- One CCG `search` at most, with `limit: 5`. +- Start one CCG `search` with `limit: 5`; follow its exact `next` continuation + at most three times and stop early when a credible candidate appears. - Verify no more than one or two source ranges unless the selected source itself points to one necessary continuation. -- Do not page an ordinary ranked answer. Disclose truncation only when it limits - the answer. +- Do not start broad grep exploration while an actionable CCG continuation + remains. Disclose truncation only when the three-continuation cap limits the + answer. - Do not echo raw result lists or mandatory operational reports. Return the answer and its relevant paths or symbols. - Read [`references/supported-languages.md`](references/supported-languages.md) From 8748b714679c8fd9f3e10280b580072ed83c402b Mon Sep 17 00:00:00 2001 From: tae2089 Date: Sat, 22 Aug 2026 01:56:20 +0900 Subject: [PATCH 02/10] docs(skills): verify CCG candidate sets before paging --- skills/ccg/SKILL.md | 56 +++++++++++++++++++++++++++++---------------- 1 file changed, 36 insertions(+), 20 deletions(-) diff --git a/skills/ccg/SKILL.md b/skills/ccg/SKILL.md index 83ee5e4..a3c4a2d 100644 --- a/skills/ccg/SKILL.md +++ b/skills/ccg/SKILL.md @@ -2,7 +2,7 @@ name: ccg description: "Fast read-only code discovery with a bounded code-context-graph search and targeted source verification. Use when an ordinary positive lookup or explanation needs an entry point, recorded intent, known-path inventory, or direct relationship evidence. Do not use for absence, completeness, exhaustive inventory, deep flow or impact analysis, or graph writes; use ccg-search-verify for defensible negative claims, and require explicit invocation for ccg-analyze or ccg-build." metadata: - version: 3.0.1 + version: 3.0.2 openclaw: category: "code-intelligence" domain: "core" @@ -14,8 +14,9 @@ metadata: # ccg — Fast Search -Use CCG to find one credible entry point, then verify it in current source. Keep -ordinary discovery small enough that it beats broad grep exploration. +Use CCG to find one directly relevant entry point, then verify it in current +source. Keep ordinary discovery small enough that it beats broad grep +exploration. ## Task Routing and Entry @@ -51,22 +52,32 @@ impact-analysis procedure. - known thing: one short identifier or rare keyword; - unknown symbol: one concise plain-language question that can match recorded `@intent` or `@domainRule` reasons. -3. If the current page has no credible candidate and the result supplies a - continuation, follow that exact `next` call up to three times. Preserve the +3. On the current result page, inspect all returned paths, symbols, and + summaries. Collect the candidates related to the requested component or + behavior, then run one targeted grep across only those returned paths. Limit + that candidate check to 20 matching lines, and read at most the strongest one + or two source ranges. Stop when the verified source provides enough evidence + to answer. This candidate-set check is not the repository-wide fallback. +4. If the candidate check is insufficient and the response supplies a + continuation, follow that exact `next` call and repeat step 3. Preserve the query, limit, namespace, and continuation offsets supplied by CCG; do not - calculate offsets or reformulate the query. Stop paging as soon as one - credible candidate appears, `next` disappears, or the third continuation - has been read. -4. Verify the best credible candidate in one or two source ranges. Stop when - those ranges answer the question. -5. Use `get_node`, `describe`, or one bounded `query_graph` call only when the + calculate offsets or reformulate the query. Perform at most three such + continuation-and-check cycles, stopping if source evidence answers the + question or `next` disappears. Do not run repository-wide grep during these + cycles. +5. Only after `next` disappears or three continuation calls have been consumed, + use one bounded grep fallback if the question is still unanswered. Restrict + it to production source where possible, exclude tests by default, and return + at most 20 matching lines. Read only the best matching source range. Do not + make a negative claim if this fallback also misses. +6. Use `get_node`, `describe`, or one bounded `query_graph` call only when the answer needs exact identity, an unranked known-path inventory, or one direct relationship fact. Every query word must occur in the same indexed document, so do not concatenate -the prompt’s examples into a long query. If the search pages have no credible -hit, do not fan out into synonyms inside this fast workflow; use narrowly -targeted grep/read and do not make a negative claim. +the prompt’s examples into a long query. If the search pages have no qualifying +result, do not fan out into synonyms inside this fast workflow; use the single +bounded grep/read fallback and do not make a negative claim. ```bash ccg search --limit 5 "" @@ -92,12 +103,17 @@ This skill is read-only. Report a missing or stale graph instead of invoking ## Response Budget Rule - Start one CCG `search` with `limit: 5`; follow its exact `next` continuation - at most three times and stop early when a credible candidate appears. -- Verify no more than one or two source ranges unless the selected source itself - points to one necessary continuation. -- Do not start broad grep exploration while an actionable CCG continuation - remains. Disclose truncation only when the three-continuation cap limits the - answer. + at most three times and stop early when verified source evidence answers the + question. +- For each result page, inspect all returned candidate metadata, run one + targeted grep across the relevant returned paths with at most 20 matching + lines, and read only the strongest one or two source ranges. +- Do not run repository-wide grep while an actionable CCG continuation remains. + After continuation ends, allow only one grep fallback with tests excluded and + at most 20 matching lines. +- Do not reread a source range solely to obtain line numbers. Preserve line + numbers on the first read when the response needs source citations. +- Disclose truncation only when the three-continuation cap limits the answer. - Do not echo raw result lists or mandatory operational reports. Return the answer and its relevant paths or symbols. - Read [`references/supported-languages.md`](references/supported-languages.md) From ee27d49d452379764fac8a7ca8e17fa0d86eeb9f Mon Sep 17 00:00:00 2001 From: tae2089 Date: Sat, 22 Aug 2026 02:00:46 +0900 Subject: [PATCH 03/10] docs(skills): enforce bounded CCG source checks --- skills/ccg/SKILL.md | 51 +++++++++++++++++++++++---------------------- 1 file changed, 26 insertions(+), 25 deletions(-) diff --git a/skills/ccg/SKILL.md b/skills/ccg/SKILL.md index a3c4a2d..c7c8cee 100644 --- a/skills/ccg/SKILL.md +++ b/skills/ccg/SKILL.md @@ -2,7 +2,7 @@ name: ccg description: "Fast read-only code discovery with a bounded code-context-graph search and targeted source verification. Use when an ordinary positive lookup or explanation needs an entry point, recorded intent, known-path inventory, or direct relationship evidence. Do not use for absence, completeness, exhaustive inventory, deep flow or impact analysis, or graph writes; use ccg-search-verify for defensible negative claims, and require explicit invocation for ccg-analyze or ccg-build." metadata: - version: 3.0.2 + version: 3.0.3 openclaw: category: "code-intelligence" domain: "core" @@ -54,30 +54,32 @@ impact-analysis procedure. `@intent` or `@domainRule` reasons. 3. On the current result page, inspect all returned paths, symbols, and summaries. Collect the candidates related to the requested component or - behavior, then run one targeted grep across only those returned paths. Limit - that candidate check to 20 matching lines, and read at most the strongest one - or two source ranges. Stop when the verified source provides enough evidence - to answer. This candidate-set check is not the repository-wide fallback. -4. If the candidate check is insufficient and the response supplies a - continuation, follow that exact `next` call and repeat step 3. Preserve the + behavior. Every path passed to the candidate grep must literally appear in + that page's CCG response; repository roots and broader directories are not + candidate paths unless CCG returned them. Run one grep across that exact path + set, keep at most 20 matching lines, and read at most the strongest two source + ranges of no more than 80 lines each. Stop when the verified source provides + enough evidence to answer. +4. If the candidate evidence is insufficient and the response supplies a + continuation, the next action must be that exact `next` call. Preserve the query, limit, namespace, and continuation offsets supplied by CCG; do not - calculate offsets or reformulate the query. Perform at most three such - continuation-and-check cycles, stopping if source evidence answers the - question or `next` disappears. Do not run repository-wide grep during these - cycles. -5. Only after `next` disappears or three continuation calls have been consumed, - use one bounded grep fallback if the question is still unanswered. Restrict - it to production source where possible, exclude tests by default, and return - at most 20 matching lines. Read only the best matching source range. Do not - make a negative claim if this fallback also misses. + calculate offsets or reformulate the query. Repeat step 3 after each page for + at most three continuation calls, stopping if source evidence answers the + question or `next` disappears. +5. A repository-wide grep is allowed only when `next` is absent or three + continuation calls have been consumed. Run exactly one bounded fallback from + the repository root, exclude tests by default, and keep at most 20 matching + lines. When include and exclude filters are both used, apply exclusions last + so a later include cannot re-enable test files. Read at most the strongest + two source ranges of no more than 80 lines each. Do not make a negative claim + if this fallback also misses. 6. Use `get_node`, `describe`, or one bounded `query_graph` call only when the answer needs exact identity, an unranked known-path inventory, or one direct relationship fact. Every query word must occur in the same indexed document, so do not concatenate -the prompt’s examples into a long query. If the search pages have no qualifying -result, do not fan out into synonyms inside this fast workflow; use the single -bounded grep/read fallback and do not make a negative claim. +the prompt’s examples into a long query. Do not fan out into synonym searches +inside this fast workflow. ```bash ccg search --limit 5 "" @@ -105,12 +107,11 @@ This skill is read-only. Report a missing or stale graph instead of invoking - Start one CCG `search` with `limit: 5`; follow its exact `next` continuation at most three times and stop early when verified source evidence answers the question. -- For each result page, inspect all returned candidate metadata, run one - targeted grep across the relevant returned paths with at most 20 matching - lines, and read only the strongest one or two source ranges. -- Do not run repository-wide grep while an actionable CCG continuation remains. - After continuation ends, allow only one grep fallback with tests excluded and - at most 20 matching lines. +- For each result page, run at most one candidate grep, using only literal paths + returned on that page and keeping at most 20 matching lines. +- Read at most two source ranges per page and at most 80 lines per range. +- Run exactly one repository-wide grep only after `next` is absent or three + continuations have been consumed; exclude tests and keep at most 20 matches. - Do not reread a source range solely to obtain line numbers. Preserve line numbers on the first read when the response needs source citations. - Disclose truncation only when the three-continuation cap limits the answer. From 1717a1a032b458d78f7477f8c986d7337550bf1a Mon Sep 17 00:00:00 2001 From: tae2089 Date: Sat, 22 Aug 2026 02:09:53 +0900 Subject: [PATCH 04/10] docs(skills): prioritize production CCG evidence --- skills/ccg/SKILL.md | 65 +++++++++++++++++++++++++++------------------ 1 file changed, 39 insertions(+), 26 deletions(-) diff --git a/skills/ccg/SKILL.md b/skills/ccg/SKILL.md index c7c8cee..e6512be 100644 --- a/skills/ccg/SKILL.md +++ b/skills/ccg/SKILL.md @@ -2,7 +2,7 @@ name: ccg description: "Fast read-only code discovery with a bounded code-context-graph search and targeted source verification. Use when an ordinary positive lookup or explanation needs an entry point, recorded intent, known-path inventory, or direct relationship evidence. Do not use for absence, completeness, exhaustive inventory, deep flow or impact analysis, or graph writes; use ccg-search-verify for defensible negative claims, and require explicit invocation for ccg-analyze or ccg-build." metadata: - version: 3.0.3 + version: 3.0.5 openclaw: category: "code-intelligence" domain: "core" @@ -52,28 +52,37 @@ impact-analysis procedure. - known thing: one short identifier or rare keyword; - unknown symbol: one concise plain-language question that can match recorded `@intent` or `@domainRule` reasons. -3. On the current result page, inspect all returned paths, symbols, and - summaries. Collect the candidates related to the requested component or - behavior. Every path passed to the candidate grep must literally appear in - that page's CCG response; repository roots and broader directories are not - candidate paths unless CCG returned them. Run one grep across that exact path - set, keep at most 20 matching lines, and read at most the strongest two source - ranges of no more than 80 lines each. Stop when the verified source provides - enough evidence to answer. -4. If the candidate evidence is insufficient and the response supplies a +3. Classify every returned candidate as `production`, `test`, or `unknown`. + Confirm `test` only from, in priority order: CCG test-node metadata; explicit + repository source-set or build rules; or a repository/language test path or + filename convention. If none applies, keep the candidate `unknown` rather + than guessing. For a question not about tests, defer confirmed test + candidates while any unverified production or unknown candidate remains. If + the page contains only tests and has `next`, continue before reading tests. +4. On the current result page, collect the active production and unknown + candidates related to the requested component or behavior. A candidate check + may search only paths explicitly returned on that page; do not replace them + with parent directories or guessed paths. Inspect the strongest one or two + files, starting with narrow source ranges and extending within those files + only when the declaration or control flow continues. Stop when the verified + source provides enough evidence to answer. For a test-focused question, + include confirmed test candidates from the start. +5. If the candidate evidence is insufficient and the response supplies a continuation, the next action must be that exact `next` call. Preserve the query, limit, namespace, and continuation offsets supplied by CCG; do not - calculate offsets or reformulate the query. Repeat step 3 after each page for - at most three continuation calls, stopping if source evidence answers the - question or `next` disappears. -5. A repository-wide grep is allowed only when `next` is absent or three - continuation calls have been consumed. Run exactly one bounded fallback from - the repository root, exclude tests by default, and keep at most 20 matching - lines. When include and exclude filters are both used, apply exclusions last - so a later include cannot re-enable test files. Read at most the strongest - two source ranges of no more than 80 lines each. Do not make a negative claim - if this fallback also misses. -6. Use `get_node`, `describe`, or one bounded `query_graph` call only when the + calculate offsets or reformulate the query. Repeat steps 3 and 4 after each + page for at most three continuation calls, stopping if source evidence + answers the question or `next` disappears. +6. Any search that includes a path not explicitly returned on the current CCG + page is a fallback search, regardless of the path's name or apparent size. + Enter the fallback phase only when `next` is absent or three continuation + calls have been consumed. In that single phase, run one bounded grouped + search and, only if its results are ambiguous, one narrower refinement. + Apply the same three-way classification to direct-search results and inspect + production and unknown files first. If they still provide insufficient + evidence, confirmed tests may be read as supporting evidence. Do not make a + negative claim if fallback also misses. +7. Use `get_node`, `describe`, or one bounded `query_graph` call only when the answer needs exact identity, an unranked known-path inventory, or one direct relationship fact. @@ -107,11 +116,15 @@ This skill is read-only. Report a missing or stale graph instead of invoking - Start one CCG `search` with `limit: 5`; follow its exact `next` continuation at most three times and stop early when verified source evidence answers the question. -- For each result page, run at most one candidate grep, using only literal paths - returned on that page and keeping at most 20 matching lines. -- Read at most two source ranges per page and at most 80 lines per range. -- Run exactly one repository-wide grep only after `next` is absent or three - continuations have been consumed; exclude tests and keep at most 20 matches. +- For each result page, search only paths explicitly returned on that page and + inspect at most the strongest one or two files. Start with narrow ranges and + extend within those files only when required by the code structure. +- For non-test questions, prioritize production and unknown candidates. Defer + confirmed tests until CCG continuations and direct production-source evidence + are insufficient; never classify an ambiguous path as test merely to skip it. +- Treat every scope expansion beyond returned paths as fallback. Enter fallback + once, only after `next` is absent or three continuations have been consumed; + allow one grouped search plus at most one narrower refinement. - Do not reread a source range solely to obtain line numbers. Preserve line numbers on the first read when the response needs source citations. - Disclose truncation only when the three-continuation cap limits the answer. From a871811e9e6ec3e5285fe612e0ced9559fe95ec3 Mon Sep 17 00:00:00 2001 From: tae2089 Date: Sat, 22 Aug 2026 02:16:41 +0900 Subject: [PATCH 05/10] docs(skills): bound CCG verification by declarations --- skills/ccg/SKILL.md | 57 ++++++++++++++++++++++++++++----------------- 1 file changed, 36 insertions(+), 21 deletions(-) diff --git a/skills/ccg/SKILL.md b/skills/ccg/SKILL.md index e6512be..71938c1 100644 --- a/skills/ccg/SKILL.md +++ b/skills/ccg/SKILL.md @@ -2,7 +2,7 @@ name: ccg description: "Fast read-only code discovery with a bounded code-context-graph search and targeted source verification. Use when an ordinary positive lookup or explanation needs an entry point, recorded intent, known-path inventory, or direct relationship evidence. Do not use for absence, completeness, exhaustive inventory, deep flow or impact analysis, or graph writes; use ccg-search-verify for defensible negative claims, and require explicit invocation for ccg-analyze or ccg-build." metadata: - version: 3.0.5 + version: 3.0.6 openclaw: category: "code-intelligence" domain: "core" @@ -52,6 +52,10 @@ impact-analysis procedure. - known thing: one short identifier or rare keyword; - unknown symbol: one concise plain-language question that can match recorded `@intent` or `@domainRule` reasons. + Write the natural-language part in the dominant language of the indexed + source comments and annotations. If the user uses another language, translate + the intent but preserve identifiers, paths, literals, and error text. Do not + combine both languages in one query. 3. Classify every returned candidate as `production`, `test`, or `unknown`. Confirm `test` only from, in priority order: CCG test-node metadata; explicit repository source-set or build rules; or a repository/language test path or @@ -60,13 +64,17 @@ impact-analysis procedure. candidates while any unverified production or unknown candidate remains. If the page contains only tests and has `next`, continue before reading tests. 4. On the current result page, collect the active production and unknown - candidates related to the requested component or behavior. A candidate check - may search only paths explicitly returned on that page; do not replace them - with parent directories or guessed paths. Inspect the strongest one or two - files, starting with narrow source ranges and extending within those files - only when the declaration or control flow continues. Stop when the verified - source provides enough evidence to answer. For a test-focused question, - include confirmed test candidates from the start. + candidates related to the requested component or behavior. Check only paths + explicitly returned on that page; do not replace them with parent directories + or guessed paths, and do not recheck a path already examined on an earlier + page. Before reading source, identify the specific claim each candidate could + support from its returned symbol or summary. Skip candidates that share only + broad query words. Use the returned line or a targeted grep over those exact + paths to locate the relevant declaration. Read its attached annotation or doc + comment and the complete declaration, not a large contiguous file range. + Follow another declaration only when the selected declaration directly + references it. Stop when the verified source answers the question. For a + test-focused question, include confirmed test candidates from the start. 5. If the candidate evidence is insufficient and the response supplies a continuation, the next action must be that exact `next` call. Preserve the query, limit, namespace, and continuation offsets supplied by CCG; do not @@ -75,13 +83,17 @@ impact-analysis procedure. answers the question or `next` disappears. 6. Any search that includes a path not explicitly returned on the current CCG page is a fallback search, regardless of the path's name or apparent size. - Enter the fallback phase only when `next` is absent or three continuation - calls have been consumed. In that single phase, run one bounded grouped - search and, only if its results are ambiguous, one narrower refinement. - Apply the same three-way classification to direct-search results and inspect - production and unknown files first. If they still provide insufficient - evidence, confirmed tests may be read as supporting evidence. Do not make a - negative claim if fallback also misses. + Begin fallback only when `next` is absent or three continuation calls have + been consumed. The first fallback search may expand scope. Every later search + must narrow scope using a concrete path, symbol, literal, or error text newly + produced by the preceding result; never repeat a broad search or fan out into + synonyms. For a non-test question, apply confirmed test-path exclusions in + the search command before matches are returned. Apply the same three-way + classification to direct-search results and inspect production and unknown + files first. If they still provide insufficient evidence, confirmed tests + may be read as supporting evidence. Stop when source evidence answers the + question or a search yields no new clue. Do not make a negative claim if + fallback also misses. 7. Use `get_node`, `describe`, or one bounded `query_graph` call only when the answer needs exact identity, an unranked known-path inventory, or one direct relationship fact. @@ -117,14 +129,17 @@ This skill is read-only. Report a missing or stale graph instead of invoking at most three times and stop early when verified source evidence answers the question. - For each result page, search only paths explicitly returned on that page and - inspect at most the strongest one or two files. Start with narrow ranges and - extend within those files only when required by the code structure. + inspect only unseen candidates that can support a specific claim. Read + attached documentation plus complete declarations rather than arbitrary file + ranges; follow only directly referenced declarations. - For non-test questions, prioritize production and unknown candidates. Defer confirmed tests until CCG continuations and direct production-source evidence - are insufficient; never classify an ambiguous path as test merely to skip it. -- Treat every scope expansion beyond returned paths as fallback. Enter fallback - once, only after `next` is absent or three continuations have been consumed; - allow one grouped search plus at most one narrower refinement. + are insufficient, and exclude confirmed test paths before direct-search + output is produced. Never classify an ambiguous path as test merely to skip it. +- Treat every scope expansion beyond returned paths as fallback. Begin fallback + only after `next` is absent or three continuations have been consumed. After + the initial expansion, every search must narrow using a newly found concrete + clue; stop when no new clue appears. - Do not reread a source range solely to obtain line numbers. Preserve line numbers on the first read when the response needs source citations. - Disclose truncation only when the three-continuation cap limits the answer. From 1f7e2fe8018e383d287679eda48c65748efa60c7 Mon Sep 17 00:00:00 2001 From: tae2089 Date: Sat, 22 Aug 2026 02:23:18 +0900 Subject: [PATCH 06/10] docs(skills): tighten CCG declaration reads --- skills/ccg/SKILL.md | 51 ++++++++++++++++++++++++++------------------- 1 file changed, 30 insertions(+), 21 deletions(-) diff --git a/skills/ccg/SKILL.md b/skills/ccg/SKILL.md index 71938c1..4f077e4 100644 --- a/skills/ccg/SKILL.md +++ b/skills/ccg/SKILL.md @@ -2,7 +2,7 @@ name: ccg description: "Fast read-only code discovery with a bounded code-context-graph search and targeted source verification. Use when an ordinary positive lookup or explanation needs an entry point, recorded intent, known-path inventory, or direct relationship evidence. Do not use for absence, completeness, exhaustive inventory, deep flow or impact analysis, or graph writes; use ccg-search-verify for defensible negative claims, and require explicit invocation for ccg-analyze or ccg-build." metadata: - version: 3.0.6 + version: 3.0.7 openclaw: category: "code-intelligence" domain: "core" @@ -59,8 +59,11 @@ impact-analysis procedure. 3. Classify every returned candidate as `production`, `test`, or `unknown`. Confirm `test` only from, in priority order: CCG test-node metadata; explicit repository source-set or build rules; or a repository/language test path or - filename convention. If none applies, keep the candidate `unknown` rather - than guessing. For a question not about tests, defer confirmed test + filename convention. Test-only support artifacts such as fixtures, golden or + snapshot data, testdata, and generated mocks also count as `test` only when + repository or language conventions confirm that role; a generic directory + name alone is not proof. If no rule applies, keep the candidate `unknown` + rather than guessing. For a question not about tests, defer confirmed test candidates while any unverified production or unknown candidate remains. If the page contains only tests and has `next`, continue before reading tests. 4. On the current result page, collect the active production and unknown @@ -69,12 +72,16 @@ impact-analysis procedure. or guessed paths, and do not recheck a path already examined on an earlier page. Before reading source, identify the specific claim each candidate could support from its returned symbol or summary. Skip candidates that share only - broad query words. Use the returned line or a targeted grep over those exact - paths to locate the relevant declaration. Read its attached annotation or doc - comment and the complete declaration, not a large contiguous file range. - Follow another declaration only when the selected declaration directly - references it. Stop when the verified source answers the question. For a - test-focused question, include confirmed test candidates from the start. + broad query words. When a result exposes `node_id`, use `get_node` to obtain + its declaration bounds. Otherwise use the returned line, language-aware + symbol navigation, or a targeted grep over those exact paths to locate the + declaration boundary. Read its attached annotation or doc comment and that + complete declaration only. If multiple declarations are needed, read them as + separate ranges; never read one continuous range that includes unrelated + declarations between them. Follow another declaration only when the selected + declaration directly references it. Stop when the verified source answers + the question. For a test-focused question, include confirmed test candidates + from the start. 5. If the candidate evidence is insufficient and the response supplies a continuation, the next action must be that exact `next` call. Preserve the query, limit, namespace, and continuation offsets supplied by CCG; do not @@ -87,16 +94,16 @@ impact-analysis procedure. been consumed. The first fallback search may expand scope. Every later search must narrow scope using a concrete path, symbol, literal, or error text newly produced by the preceding result; never repeat a broad search or fan out into - synonyms. For a non-test question, apply confirmed test-path exclusions in - the search command before matches are returned. Apply the same three-way - classification to direct-search results and inspect production and unknown - files first. If they still provide insufficient evidence, confirmed tests - may be read as supporting evidence. Stop when source evidence answers the - question or a search yields no new clue. Do not make a negative claim if + synonyms. For a non-test question, apply confirmed test and test-support path + exclusions in the search command before matches are returned. Apply the same + three-way classification to direct-search results and inspect production and + unknown files first. If they still provide insufficient evidence, confirmed + tests may be read as supporting evidence. Stop when source evidence answers + the question or a search yields no new clue. Do not make a negative claim if fallback also misses. 7. Use `get_node`, `describe`, or one bounded `query_graph` call only when the - answer needs exact identity, an unranked known-path inventory, or one direct - relationship fact. + answer needs exact identity, declaration bounds, an unranked known-path + inventory, or one direct relationship fact. Every query word must occur in the same indexed document, so do not concatenate the prompt’s examples into a long query. Do not fan out into synonym searches @@ -130,12 +137,14 @@ This skill is read-only. Report a missing or stale graph instead of invoking question. - For each result page, search only paths explicitly returned on that page and inspect only unseen candidates that can support a specific claim. Read - attached documentation plus complete declarations rather than arbitrary file - ranges; follow only directly referenced declarations. + attached documentation plus exact declaration ranges rather than arbitrary + file ranges. Read separate declarations separately, and follow only directly + referenced declarations. - For non-test questions, prioritize production and unknown candidates. Defer confirmed tests until CCG continuations and direct production-source evidence - are insufficient, and exclude confirmed test paths before direct-search - output is produced. Never classify an ambiguous path as test merely to skip it. + are insufficient, and exclude confirmed test and test-support paths before + direct-search output is produced. Never classify an ambiguous path as test + merely to skip it. - Treat every scope expansion beyond returned paths as fallback. Begin fallback only after `next` is absent or three continuations have been consumed. After the initial expansion, every search must narrow using a newly found concrete From 988ea84a1e5da50e55c5746e486e8ef2aa75e68f Mon Sep 17 00:00:00 2001 From: tae2089 Date: Sat, 22 Aug 2026 02:34:11 +0900 Subject: [PATCH 07/10] docs(skills): simplify CCG search loop --- skills/ccg/SKILL.md | 232 ++++++++-------------- skills/ccg/references/search-execution.md | 73 +++++++ 2 files changed, 152 insertions(+), 153 deletions(-) create mode 100644 skills/ccg/references/search-execution.md diff --git a/skills/ccg/SKILL.md b/skills/ccg/SKILL.md index 4f077e4..5c7ead4 100644 --- a/skills/ccg/SKILL.md +++ b/skills/ccg/SKILL.md @@ -1,8 +1,8 @@ --- name: ccg -description: "Fast read-only code discovery with a bounded code-context-graph search and targeted source verification. Use when an ordinary positive lookup or explanation needs an entry point, recorded intent, known-path inventory, or direct relationship evidence. Do not use for absence, completeness, exhaustive inventory, deep flow or impact analysis, or graph writes; use ccg-search-verify for defensible negative claims, and require explicit invocation for ccg-analyze or ccg-build." +description: "Fast read-only code discovery with mandatory structured CCG search for unknown entry points, followed by bounded source verification. Use for ordinary positive lookups, recorded intent, known-path inventory, or one direct relationship fact. Do not use for absence, completeness, exhaustive inventory, deep flow or impact analysis, or graph writes." metadata: - version: 3.0.7 + version: 4.0.0 openclaw: category: "code-intelligence" domain: "core" @@ -14,160 +14,86 @@ metadata: # ccg — Fast Search -Use CCG to find one directly relevant entry point, then verify it in current -source. Keep ordinary discovery small enough that it beats broad grep -exploration. +Find one useful CCG candidate, verify it in current source, and stop as soon as +the positive question is answered. -## Task Routing and Entry +## Route -| User intent | Start with | +| User intent | Route | | --- | --- | -| Known filename, identifier, literal, or error text | Grep + Read | -| Unknown entry point, recorded intent, or keyword | One bounded `search` | -| Contents of a known file or folder | `describe` | -| One direct caller/callee fact | One bounded `query_graph` lookup | -| Absence, completeness, or exhaustive inventory | `ccg-search-verify` skill if available | -| Deep flow, change impact, or blast radius | Stop and report that the user must explicitly invoke `ccg-analyze` | -| Generated documentation | `ccg-docs` skill if available | -| Write or repair annotations | `ccg-annotate` skill if available | -| Multiple repositories or services | `ccg-namespace` skill if available | -| Build, update, migrate, postprocess, or scoped graph write | Stop and report that the user must explicitly invoke `ccg-build` | - -An ordinary miss is not permission to make a negative claim. When the answer -would become “not found,” “does not exist,” “complete,” or “all,” switch to -`ccg-search-verify` instead of widening this workflow. - -Do not invoke `ccg-analyze` automatically. A request about algorithms, -pipelines, impact, callers, or flow does not itself authorize its traversal and -impact-analysis procedure. - -## Fast Workflow - -1. Reuse a namespace already supplied by repository instructions, configuration, - or the user. Do not call `get_minimal_context`, `list_namespaces`, or - `list_graph_stats` for an ordinary search when that information is already - known. -2. With an exact clue, use grep/read. Otherwise start one `search` with - `limit: 5`: - - known thing: one short identifier or rare keyword; - - unknown symbol: one concise plain-language question that can match recorded - `@intent` or `@domainRule` reasons. - Write the natural-language part in the dominant language of the indexed - source comments and annotations. If the user uses another language, translate - the intent but preserve identifiers, paths, literals, and error text. Do not - combine both languages in one query. -3. Classify every returned candidate as `production`, `test`, or `unknown`. - Confirm `test` only from, in priority order: CCG test-node metadata; explicit - repository source-set or build rules; or a repository/language test path or - filename convention. Test-only support artifacts such as fixtures, golden or - snapshot data, testdata, and generated mocks also count as `test` only when - repository or language conventions confirm that role; a generic directory - name alone is not proof. If no rule applies, keep the candidate `unknown` - rather than guessing. For a question not about tests, defer confirmed test - candidates while any unverified production or unknown candidate remains. If - the page contains only tests and has `next`, continue before reading tests. -4. On the current result page, collect the active production and unknown - candidates related to the requested component or behavior. Check only paths - explicitly returned on that page; do not replace them with parent directories - or guessed paths, and do not recheck a path already examined on an earlier - page. Before reading source, identify the specific claim each candidate could - support from its returned symbol or summary. Skip candidates that share only - broad query words. When a result exposes `node_id`, use `get_node` to obtain - its declaration bounds. Otherwise use the returned line, language-aware - symbol navigation, or a targeted grep over those exact paths to locate the - declaration boundary. Read its attached annotation or doc comment and that - complete declaration only. If multiple declarations are needed, read them as - separate ranges; never read one continuous range that includes unrelated - declarations between them. Follow another declaration only when the selected - declaration directly references it. Stop when the verified source answers - the question. For a test-focused question, include confirmed test candidates - from the start. -5. If the candidate evidence is insufficient and the response supplies a - continuation, the next action must be that exact `next` call. Preserve the - query, limit, namespace, and continuation offsets supplied by CCG; do not - calculate offsets or reformulate the query. Repeat steps 3 and 4 after each - page for at most three continuation calls, stopping if source evidence - answers the question or `next` disappears. -6. Any search that includes a path not explicitly returned on the current CCG - page is a fallback search, regardless of the path's name or apparent size. - Begin fallback only when `next` is absent or three continuation calls have - been consumed. The first fallback search may expand scope. Every later search - must narrow scope using a concrete path, symbol, literal, or error text newly - produced by the preceding result; never repeat a broad search or fan out into - synonyms. For a non-test question, apply confirmed test and test-support path - exclusions in the search command before matches are returned. Apply the same - three-way classification to direct-search results and inspect production and - unknown files first. If they still provide insufficient evidence, confirmed - tests may be read as supporting evidence. Stop when source evidence answers - the question or a search yields no new clue. Do not make a negative claim if - fallback also misses. -7. Use `get_node`, `describe`, or one bounded `query_graph` call only when the - answer needs exact identity, declaration bounds, an unranked known-path - inventory, or one direct relationship fact. - -Every query word must occur in the same indexed document, so do not concatenate -the prompt’s examples into a long query. Do not fan out into synonym searches -inside this fast workflow. - -```bash -ccg search --limit 5 "" -ccg status # population only; not proof of freshness -``` - -Use `ccg --help` rather than relying on remembered flags. - -## Freshness Boundary - -Current source is authoritative for text, branches, runtime semantics, and -location. CCG supplies indexed intent and relationship candidates. - -Positive evidence verified in current source can answer an ordinary question -without a separate freshness preflight. A graph miss, stale result, or missing -namespace cannot support a negative claim; route that task to -`ccg-search-verify`. Call `list_graph_stats` only when the user asks about graph -population or graph state itself. - -This skill is read-only. Report a missing or stale graph instead of invoking -`ccg-build`. The user must explicitly name `ccg-build` before any graph write. - -## Response Budget Rule - -- Start one CCG `search` with `limit: 5`; follow its exact `next` continuation - at most three times and stop early when verified source evidence answers the - question. -- For each result page, search only paths explicitly returned on that page and - inspect only unseen candidates that can support a specific claim. Read - attached documentation plus exact declaration ranges rather than arbitrary - file ranges. Read separate declarations separately, and follow only directly - referenced declarations. -- For non-test questions, prioritize production and unknown candidates. Defer - confirmed tests until CCG continuations and direct production-source evidence - are insufficient, and exclude confirmed test and test-support paths before - direct-search output is produced. Never classify an ambiguous path as test - merely to skip it. -- Treat every scope expansion beyond returned paths as fallback. Begin fallback - only after `next` is absent or three continuations have been consumed. After - the initial expansion, every search must narrow using a newly found concrete - clue; stop when no new clue appears. -- Do not reread a source range solely to obtain line numbers. Preserve line - numbers on the first read when the response needs source citations. -- Disclose truncation only when the three-continuation cap limits the answer. -- Do not echo raw result lists or mandatory operational reports. Return the - answer and its relevant paths or symbols. -- Read [`references/supported-languages.md`](references/supported-languages.md) - only for a supported-language or extension question. - -## Boundary - -- `search` returns ranked candidates, not proof of absence or completeness. -- Reserve impact-radius and flow-tracing workflows for an explicit - `ccg-analyze` invocation. -- Use `ccg-search-verify` for defensible negative or exhaustive claims. -- Use `ccg-build` only when explicitly invoked. +| Known filename, identifier, literal, or error text | Grep + Read may start directly | +| Unknown entry point, behavior, reason, or keyword | Run the Core Loop below | +| Contents of a known file or folder | One `describe` call | +| One direct caller/callee fact | One bounded `query_graph` call | +| Absence, completeness, or exhaustive inventory | `ccg-search-verify` | +| Deep flow, change impact, or blast radius | Require explicit `ccg-analyze` invocation | +| Build, update, migrate, postprocess, or graph write | Require explicit `ccg-build` invocation | + +An ordinary miss never authorizes a negative claim. If the answer would become +“not found,” “does not exist,” “complete,” or “all,” switch to +`ccg-search-verify`. + +## Core Loop + +For an unknown entry point, behavior, reason, or keyword, follow these steps in +order: + +1. **Search.** Before grep or source browsing, run one structured CCG `search` + with `limit: 5`. Prefer MCP. If MCP is unavailable, use + `ccg search --json --limit 5 ""`; never use the plain CLI display. +2. **Verify this page.** Inspect only relevant `production` and `unknown` paths + returned on the current page. Each source read must verify one concrete + claim. Read the exact declaration and its attached documentation, not broad + file ranges. +3. **Decide.** If verified evidence answers the question, stop. If it does not + and the response contains `next`, the next action must be that continuation + verbatim. Do not grep a wider scope while an allowed continuation remains. +4. **Continue.** Repeat page verification and decision for at most three `next` + calls. Preserve the query, limit, namespace, and offsets supplied by CCG; + never calculate or rewrite them. +5. **Fallback.** Only when `next` is absent or three `next` calls have been used, + search current source directly. The first fallback search may expand scope; + every later search must narrow from a new concrete path, symbol, literal, or + error text found by the preceding step. +6. **Stop.** Answer when the smallest verified evidence set is sufficient. If a + search yields no new clue, stop and report that the entry point could not be + located without claiming that the code does not exist. + +Do not replace a returned path with a guessed directory, reread an examined +range, fan out into synonym searches, or echo raw candidate lists. + +## Conditional Details + +Read +[`references/search-execution.md`](references/search-execution.md) only when a +branch needs it: + +- MCP is unavailable and CLI fallback is required; +- the user's language differs from repository annotations or comments; +- a candidate may be test-only or declaration bounds are missing; +- the direct-source fallback begins. + +Read +[`references/supported-languages.md`](references/supported-languages.md) only +for a supported-language or extension question. + +## Boundaries + +- Current source is authoritative for text, location, and runtime semantics; + CCG supplies ranked intent and relationship candidates. +- Positive evidence verified in current source needs no freshness preflight. + A graph miss or stale result is never negative evidence. +- Do not call `get_minimal_context`, `list_namespaces`, or `list_graph_stats` + when repository instructions or configuration already supply the namespace. +- Use `get_node` only for exact identity or declaration bounds, and follow + another declaration only when the selected declaration directly references + it. +- This skill is read-only. Never invoke graph build, update, migration, + postprocessing, impact-radius, or flow-tracing tools from this workflow. ## Completion -Answer from the smallest verified evidence set. Name the selected path or -qualified symbol and disclose only uncertainty that materially limits the -answer. Do not append namespace, freshness, limit, or tool-call accounting -unless the user requests it or it changes the conclusion. +Return the answer with the relevant path or qualified symbol. Mention +truncation only when the three-continuation cap limits the answer, and disclose +only uncertainty that materially changes the conclusion. Do not append tool, +namespace, freshness, or call-count reports unless requested. diff --git a/skills/ccg/references/search-execution.md b/skills/ccg/references/search-execution.md new file mode 100644 index 0000000..cb0a720 --- /dev/null +++ b/skills/ccg/references/search-execution.md @@ -0,0 +1,73 @@ +# Search Execution Details + +Use only the section required by the active branch of the `ccg` Core Loop. + +## Structured Search Interfaces + +Prefer MCP `search` with `limit: 5`. When MCP is unavailable, use JSON CLI +output so node IDs, declaration bounds, truncation, and continuation arguments +remain machine-readable: + +```bash +ccg search --json --limit 5 "" +``` + +For MCP, execute the returned `next` call verbatim. For CLI JSON, execute the +command represented by `next.args` without calculating offsets or changing the +query, limit, or namespace. Plain CLI output is not a substitute because it can +hide structured continuation and declaration data. + +## Query Language + +Use one short identifier or rare keyword for a known thing. When the symbol is +unknown, use one concise question that can match recorded `@intent` or +`@domainRule` text. + +Write natural-language terms in the dominant language of repository comments +and annotations. Translate the user's intent when needed, but preserve exact +identifiers, paths, literals, and error messages. Do not combine translations +or prompt examples into one long query: every query word must occur in the same +indexed document. + +## Candidate Classification + +Classify candidates as `production`, `test`, or `unknown`. + +Confirm `test` only from one of these signals, in priority order: + +1. CCG test-node metadata; +2. explicit repository source-set or build rules; +3. established repository or language test path and filename conventions. + +Fixtures, golden files, snapshots, testdata, and generated mocks are test +support only when those same signals confirm the role. A generic directory name +is not enough. Ambiguous candidates remain `unknown`. + +For a non-test question, inspect production and unknown candidates first. If a +page contains only confirmed tests and has `next`, continue before reading the +tests. Tests may be used later as supporting evidence when production evidence +is insufficient. + +## Declaration Evidence + +When a result supplies a node ID, use `get_node` for exact declaration bounds. +Otherwise use the returned line, language-aware symbol navigation, or a +targeted locator search restricted to the returned paths. A locator grep should +not include broad context ranges. + +Read the complete declaration and attached annotation or documentation. Read +multiple declarations as separate ranges rather than one large range containing +unrelated code. Preserve line numbers on the first read when citations may be +needed. + +## Direct-Source Fallback + +A source search that includes paths not returned by the current CCG page is a +fallback, even when the scope looks small. + +For non-test questions, exclude confirmed test and test-support paths before +fallback matches are returned. Inspect production and unknown files first. The +initial fallback may search a task-relevant scope; every subsequent search must +narrow using a new concrete clue from the preceding result. Stop if the result +provides no new clue. Never turn a fallback miss into an absence or completeness +claim; route such claims to `ccg-search-verify`. From 6dda4dc4245299dafb18e92f59f20e6fb389e828 Mon Sep 17 00:00:00 2001 From: tae2089 Date: Sat, 22 Aug 2026 02:51:58 +0900 Subject: [PATCH 08/10] docs(skills): focus CCG query generation --- skills/ccg/SKILL.md | 14 ++++++++++---- skills/ccg/references/search-execution.md | 14 ++++++++------ 2 files changed, 18 insertions(+), 10 deletions(-) diff --git a/skills/ccg/SKILL.md b/skills/ccg/SKILL.md index 5c7ead4..47bc7e5 100644 --- a/skills/ccg/SKILL.md +++ b/skills/ccg/SKILL.md @@ -2,7 +2,7 @@ name: ccg description: "Fast read-only code discovery with mandatory structured CCG search for unknown entry points, followed by bounded source verification. Use for ordinary positive lookups, recorded intent, known-path inventory, or one direct relationship fact. Do not use for absence, completeness, exhaustive inventory, deep flow or impact analysis, or graph writes." metadata: - version: 4.0.0 + version: 4.0.1 openclaw: category: "code-intelligence" domain: "core" @@ -38,9 +38,15 @@ An ordinary miss never authorizes a negative claim. If the answer would become For an unknown entry point, behavior, reason, or keyword, follow these steps in order: -1. **Search.** Before grep or source browsing, run one structured CCG `search` - with `limit: 5`. Prefer MCP. If MCP is unavailable, use - `ccg search --json --limit 5 ""`; never use the plain CLI display. +1. **Search.** Before grep or source browsing, rewrite the request as a CCG + query; never submit the user's full sentence. Preserve an exact identifier, + path, literal, or error text as one clue. Otherwise select two to four + discriminative repository-language terms that name the component and + behavior and are likely to coexist in one declaration or annotation. Drop + question words, connective prose, and illustrative examples. Run one + structured CCG `search` with that query and `limit: 5`. Prefer MCP. If MCP + is unavailable, use `ccg search --json --limit 5 ""`; never use the + plain CLI display. 2. **Verify this page.** Inspect only relevant `production` and `unknown` paths returned on the current page. Each source read must verify one concrete claim. Read the exact declaration and its attached documentation, not broad diff --git a/skills/ccg/references/search-execution.md b/skills/ccg/references/search-execution.md index cb0a720..e02bb4c 100644 --- a/skills/ccg/references/search-execution.md +++ b/skills/ccg/references/search-execution.md @@ -19,15 +19,17 @@ hide structured continuation and declaration data. ## Query Language -Use one short identifier or rare keyword for a known thing. When the symbol is -unknown, use one concise question that can match recorded `@intent` or -`@domainRule` text. +Use one exact identifier, path, literal, or error fragment for a known thing. +When the symbol is unknown, extract two to four discriminative terms that name +the component and behavior. Prefer repository vocabulary, nouns, and rare +operational terms over question words, generic verbs, connective prose, or the +user's examples. Choose terms likely to occur together in one declaration or +recorded `@intent` or `@domainRule`; do not submit the user's full sentence. Write natural-language terms in the dominant language of repository comments and annotations. Translate the user's intent when needed, but preserve exact -identifiers, paths, literals, and error messages. Do not combine translations -or prompt examples into one long query: every query word must occur in the same -indexed document. +identifiers, paths, literals, and error messages. Do not mix translations in one +query: every query term must occur in the same indexed document. ## Candidate Classification From 78f5191dad57d4c2bf30ce1372b6f4c64552d51f Mon Sep 17 00:00:00 2001 From: tae2089 Date: Sat, 22 Aug 2026 07:53:43 +0900 Subject: [PATCH 09/10] docs(skills): clarify explicit graph build calls --- skills/ccg-build/SKILL.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/skills/ccg-build/SKILL.md b/skills/ccg-build/SKILL.md index c774279..e649f36 100644 --- a/skills/ccg-build/SKILL.md +++ b/skills/ccg-build/SKILL.md @@ -54,8 +54,8 @@ ccg migrate # Existing database schema upgrade when required Over MCP: -- `build_or_update_graph(full_rebuild=false)` requests an incremental update. -- `build_or_update_graph(full_rebuild=true)` requests a full rebuild. +- Use `build_or_update_graph` with `full_rebuild=false` for an incremental update. +- Use `build_or_update_graph` with `full_rebuild=true` for a full rebuild. - `parse_project` writes parsed graph state without search postprocessing. - `run_postprocess` refreshes selected derived artifacts after graph changes. From 0188c85da0bca4204ff33ae428ff012d96a9f3d2 Mon Sep 17 00:00:00 2001 From: tae2089 Date: Sat, 22 Aug 2026 07:54:05 +0900 Subject: [PATCH 10/10] feat(search): optimize natural-language discovery --- AGENTS.md | 7 +- guide/cli-reference.md | 1 + guide/mcp-tools.md | 2 +- internal/adapters/inbound/cli/search.go | 26 ++- internal/adapters/inbound/cli/search_test.go | 47 +++++ .../inbound/cli/skills_contract_test.go | 80 +++++++- .../adapters/inbound/mcp/handler_query.go | 40 +++- .../adapters/inbound/mcp/handlers_test.go | 33 ++++ internal/adapters/inbound/mcp/tools_query.go | 9 +- .../adapters/outbound/searchsql/backend.go | 4 + .../adapters/outbound/searchsql/postgres.go | 42 ++++- .../adapters/outbound/searchsql/retrieval.go | 43 +++++ .../outbound/searchsql/retrieval_test.go | 130 +++++++++++++ .../adapters/outbound/searchsql/sanitize.go | 51 +++--- .../outbound/searchsql/sanitize_test.go | 20 +- .../adapters/outbound/searchsql/sqlite.go | 50 ++++- internal/app/search/queryterm/queryterm.go | 36 +++- .../app/search/queryterm/queryterm_test.go | 27 +++ internal/app/search/service.go | 35 ++++ internal/app/search/service_test.go | 33 ++++ internal/app/search/wire/wire.go | 93 ++++++++++ internal/app/search/wire/wire_test.go | 73 ++++++++ skills/ccg/SKILL.md | 172 ++++++++++-------- skills/ccg/references/search-execution.md | 97 ++++++++-- 24 files changed, 1003 insertions(+), 148 deletions(-) create mode 100644 internal/adapters/outbound/searchsql/retrieval_test.go create mode 100644 internal/app/search/queryterm/queryterm_test.go diff --git a/AGENTS.md b/AGENTS.md index 7f70da6..d9c3c13 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -52,9 +52,12 @@ Use `.ccg.yaml` to manage project defaults such as exclude patterns and DB setti ## Code Search Rules When looking for code locations, related implementations, call relationships, impact radius, or architecture context, -use ccg MCP tools and Agent Skills first. +use CCG Agent Skills and environment-aware structured search first. The `/ccg` skill selects MCP when repository +instructions provide server-visible routing, and local JSON CLI when the repository has usable local `ccg` configuration. +For this repository, use the local JSON CLI backed by `.ccg.yaml`; the MCP server documentation below describes the +product and is not a routing instruction for the default remote MCP connection. -- `/ccg` is the fast default for ordinary positive discovery: use at most one `search` call with `limit: 5`, then verify the best candidate in one or two source ranges. Skip namespace, minimal-context, and graph-stat preflights when repository instructions already provide what the query needs. +- `/ccg` is the fast default for ordinary positive discovery: start with one `search` call using `limit: 5`, pass the namespace already supplied by repository instructions or `.ccg.yaml` without a preflight, verify the best candidates in targeted source ranges, and follow at most three verbatim `next` calls only while the evidence remains insufficient. Skip namespace-list, minimal-context, and graph-stat preflights when repository instructions already provide what the query needs. - Use `/ccg-search-verify` when the user asks whether code does not exist, requests completeness or exhaustive inventory, or when a miss would become a defensible negative claim. It owns freshness, hybrid source checking, and truncation paging. - CCG `search` answers identifier queries and "why was this built" questions from one index. Use the `/ccg-docs` skill and `get_doc_content` to read a generated doc. - For exact symbol locations and one direct call relationship, use ccg MCP `query_graph`, `get_node`, or the `/ccg` skill. Use `get_minimal_context` only when the MCP tool contract needed for the task is unavailable; it is not an ordinary search preflight. diff --git a/guide/cli-reference.md b/guide/cli-reference.md index abf6bba..cbdfec9 100644 --- a/guide/cli-reference.md +++ b/guide/cli-reference.md @@ -48,6 +48,7 @@ ccg update ./backend --namespace backend | `ccg search --offset ` | Skip the first `n` files, so reading on never splits a file; the last line names the offset to use next | | `ccg search --include-weak ` | Also show candidates whose name, path, and `@intent` say nothing about the query | | `ccg search --json ` | Print the answer as JSON, in the same shape the MCP `search` tool returns — stable for scripts and diffs | +| `ccg search --json --compact ` | Print a smaller agent-oriented JSON view that keeps file paths, declaration bounds, evidence, truncation, and exact next actions while omitting redundant IDs and repeated per-hit names/paths | | `ccg docs [--out dir]` | Generate Markdown documentation and the `wiki-index.json` compatibility snapshot (prunes stale generator-managed docs by default) | | `ccg docs --rag-index-dir ` | Override the legacy-named Wiki index output directory (default `.ccg` or `rag.index_dir`) | | `ccg docs --prune=false` | Regenerate docs without deleting older generator-managed files | diff --git a/guide/mcp-tools.md b/guide/mcp-tools.md index ffeeb60..3fbc07f 100644 --- a/guide/mcp-tools.md +++ b/guide/mcp-tools.md @@ -15,7 +15,7 @@ code-context-graph exposes 18 MCP tools through both local `ccg serve` and the s | Tool | Purpose | | ---- | ------- | | `get_node` | Read one node by qualified name | -| `search` | Full-text search across code nodes, grouped by file: results arrive in `files`, one entry per file as `{file_path, hit_count, hits[]}`, with `file_count` giving how many of them are on this page, and a file that appears appears whole. It takes both shapes of query: a symbol you can name (an identifier, a type, two or three such words) and a plain-language question ("how does the graph get built"). Named symbols match against the indexed nodes, where every term has to appear in the same node; a question is additionally scored against the reasons authors recorded (`@intent`, `@domainRule`), and files those reasons justify are appended after the name matches. `limit` counts files and `offset` pages by files, so a page never splits one. Every hit carries its evidence (`matched` signals plus the node's `@intent`; a reason-matched hit also carries `reason` and `matched_terms`); unjustifiable candidates are cut and counted in `weak_filtered`. Optionally scoped by `path`; `include_weak: true` returns the cut ones; `namespaces: []` federates across namespaces with per-item labels, and there `limit` and `offset` are per namespace, so every namespace with a hit is on the page whatever the limit is. `truncated` says whether more files answered than this page reached, `pool_truncated` says the page ended at the edge of the candidates that were fetched rather than at the end of the answer, and `next` names the calls that retrieve them. Only both signals false means the search is complete. `limits` restates the bounds this page was cut to — `files` and `offset` as they were requested, plus the `hit_budget` that decides whether one more file still fits. `annotation_coverage` reports `with_reason` out of `declarations` — how many searched declarations carry an `@intent` or a `@domainRule`, counted per declaration rather than per tag — so `with_reason: 0` says a question was put to an index nobody has recorded a reason in, and the empty answer is about the missing annotations rather than about the code. When nothing on the page could justify itself, `next` also carries an entry that names a `skill` (`ccg-annotate`) instead of a tool. `note` is set only when the answer came back with no files at all, and says which kind of empty it is: nothing retrieved, nothing that could justify itself, a page past the end, or a repository where nobody has recorded a reason yet | +| `search` | Full-text search across code nodes, grouped by file: results arrive in `files`, one entry per file as `{file_path, hit_count, hits[]}`, with `file_count` giving how many of them are on this page, and a file that appears appears whole. It takes both shapes of query: a symbol you can name (an identifier, a type, two or three such words) and a plain-language question ("how does the graph get built"). Named symbols require every term in one indexed node. Questions try that precise match first; if it is empty, they retrieve nodes matching any meaningful term and rank nodes higher when they match more distinctive terms. Questions are also scored against reasons authors recorded (`@intent`, `@domainRule`), and files those reasons justify are appended after the general matches. `limit` counts files and `offset` pages by files, so a page never splits one. Every hit carries its evidence (`matched` signals plus the node's `@intent`; a reason-matched hit also carries `reason` and `matched_terms`); unjustifiable candidates are cut and counted in `weak_filtered`. `compact: true` returns the agent-oriented view that keeps paths, declaration bounds, evidence, completion signals, and exact next actions while omitting storage IDs and repeated per-hit names/paths; continuations keep compact mode. Optionally scoped by `path`; `include_weak: true` returns the cut ones; `namespaces: []` federates across namespaces with per-item labels, and there `limit` and `offset` are per namespace, so every namespace with a hit is on the page whatever the limit is. `truncated` says whether more files answered than this page reached, `pool_truncated` says the page ended at the edge of the candidates that were fetched rather than at the end of the answer, and `next` names the calls that retrieve them. Only both signals false means the search is complete. `limits` restates the bounds this page was cut to — `files` and `offset` as they were requested, plus the `hit_budget` that decides whether one more file still fits. `annotation_coverage` reports `with_reason` out of `declarations` — how many searched declarations carry an `@intent` or a `@domainRule`, counted per declaration rather than per tag — so `with_reason: 0` says a question was put to an index nobody has recorded a reason in, and the empty answer is about the missing annotations rather than about the code. When nothing on the page could justify itself, `next` also carries an entry that names a `skill` (`ccg-annotate`) instead of a tool. `note` is set only when the answer came back with no files at all, and says which kind of empty it is: nothing retrieved, nothing that could justify itself, a page past the end, or a repository where nobody has recorded a reason yet | | `describe` | List what the graph holds under one path, with no ranking. The answer echoes the path back as `target` and says in `scope` which kind of thing it turned out to name — `directory`, `file`, or `unknown` — and that decides which list is filled. A folder fills `children` with the folders and files directly inside, one level down, each with its file and declaration counts; a file fills `declarations` with every declaration written in it, in written order, each carrying its line range, `node_id`, and recorded `@intent`. This is what `search` hands off to: search ranks and can be wrong, this one only reports what exists. No query, no limit, no relevance. A target the graph does not hold answers with `scope` set to `unknown` and fills `suggestions` with the places that name is actually declared. It replaced the `children_of` and `file_summary` patterns of `query_graph` | | `get_annotation` | Read annotations and documentation tags for one node | | `query_graph` | Run callers, callees, imports, importers, tests, or inheritors queries; `namespaces: []` groups results per namespace. For what is written inside a file or folder, use `describe` | diff --git a/internal/adapters/inbound/cli/search.go b/internal/adapters/inbound/cli/search.go index 94f37ff..9194e23 100644 --- a/internal/adapters/inbound/cli/search.go +++ b/internal/adapters/inbound/cli/search.go @@ -26,6 +26,7 @@ func newSearchCmd(deps *Deps) *cobra.Command { var pathPrefix string var includeWeak bool var asJSON bool + var compact bool cmd := &cobra.Command{ Use: "search ", @@ -33,6 +34,9 @@ func newSearchCmd(deps *Deps) *cobra.Command { Args: cobra.ExactArgs(1), RunE: func(cmd *cobra.Command, args []string) error { query := args[0] + if compact && !asJSON { + return fmt.Errorf("--compact requires --json") + } if limit <= 0 { return fmt.Errorf("limit must be > 0, got %d", limit) } @@ -54,7 +58,11 @@ func newSearchCmd(deps *Deps) *cobra.Command { } if asJSON { - return printJSONResponse(stdout(cmd), searchwire.NewResponse(list, query, limit, offset, false)) + response := searchwire.NewResponse(list, query, limit, offset, false) + if compact { + return printCompactJSONResponse(stdout(cmd), response.Compact()) + } + return printJSONResponse(stdout(cmd), response) } printEvidenceList(stdout(cmd), list, offset) return nil @@ -66,6 +74,12 @@ func newSearchCmd(deps *Deps) *cobra.Command { cmd.Flags().StringVar(&pathPrefix, "path", "", "Filter results to file paths starting with this prefix (e.g. internal/auth)") cmd.Flags().BoolVar(&includeWeak, "include-weak", false, "Also show candidates whose name, path, and @intent say nothing about the query") cmd.Flags().BoolVar(&asJSON, "json", false, "Print the answer as JSON, in the same shape the MCP search tool returns") + cmd.Flags().BoolVar( + &compact, + "compact", + false, + "With --json, omit redundant fields while preserving evidence, source bounds, and next actions", + ) return cmd } @@ -73,12 +87,20 @@ func newSearchCmd(deps *Deps) *cobra.Command { // printJSONResponse writes the wire payload as one indented JSON document. // @intent keep --json output byte-stable and diffable while staying the MCP contract. // @sideEffect writes the whole search answer to out. -func printJSONResponse(out io.Writer, response searchwire.Response) error { +func printJSONResponse(out io.Writer, response any) error { encoder := json.NewEncoder(out) encoder.SetIndent("", " ") return trace.Wrap(encoder.Encode(response), "encode search response") } +// printCompactJSONResponse writes one unindented JSON document so formatting +// whitespace does not consume an agent's context window. +// @intent keep compact search output compact on the wire as well as in its fields. +// @sideEffect writes the whole compact search answer to out. +func printCompactJSONResponse(out io.Writer, response searchwire.CompactResponse) error { + return trace.Wrap(json.NewEncoder(out).Encode(response), "encode compact search response") +} + // printEvidenceList writes one result per unindented line and everything else // indented, so the plain result lines stay as machine-readable as they were. // diff --git a/internal/adapters/inbound/cli/search_test.go b/internal/adapters/inbound/cli/search_test.go index 7b66273..76f3ff0 100644 --- a/internal/adapters/inbound/cli/search_test.go +++ b/internal/adapters/inbound/cli/search_test.go @@ -168,6 +168,53 @@ func TestSearchCommand_JSONSpeaksTheMCPContract(t *testing.T) { } } +func TestSearchCommand_CompactJSONKeepsOnlyDecisionEvidence(t *testing.T) { + deps, stdout, stderr, db := setupSearchTest(t) + seedSearchData(t, db) + + if err := executeCmd(deps, stdout, stderr, "search", "--json", "--compact", "Hello"); err != nil { + t.Fatalf("search: %v", err) + } + + var payload map[string]any + if err := json.Unmarshal(stdout.Bytes(), &payload); err != nil { + t.Fatalf("decode: %v output=%s", err, stdout.String()) + } + if lines := strings.Count(stdout.String(), "\n"); lines != 1 { + t.Errorf("compact JSON used %d lines, want one encoded document", lines) + } + file := payload["files"].([]any)[0].(map[string]any) + hit := file["hits"].([]any)[0].(map[string]any) + for _, key := range []string{"qualified_name", "kind", "start_line", "end_line", "matched"} { + if _, ok := hit[key]; !ok { + t.Errorf("compact hit is missing %q: %s", key, stdout.String()) + } + } + for _, duplicate := range []string{"id", "name", "file_path"} { + if _, ok := hit[duplicate]; ok { + t.Errorf("compact hit kept duplicate field %q: %s", duplicate, stdout.String()) + } + } + compactSize := stdout.Len() + stdout.Reset() + if err := executeCmd(deps, stdout, stderr, "search", "--json", "Hello"); err != nil { + t.Fatalf("full search: %v", err) + } + if compactSize*4 >= stdout.Len()*3 { + t.Errorf("compact CLI output is %d bytes versus %d full bytes, want at least 25%% smaller", compactSize, stdout.Len()) + } +} + +func TestSearchCommand_CompactRequiresJSON(t *testing.T) { + deps, stdout, stderr, db := setupSearchTest(t) + seedSearchData(t, db) + + err := executeCmd(deps, stdout, stderr, "search", "--compact", "Hello") + if err == nil || !strings.Contains(err.Error(), "--compact requires --json") { + t.Fatalf("error = %v, want --compact requires --json", err) + } +} + // A truncated --json answer carries the same next actions MCP emits, phrased as // a repeatable search call. func TestSearchCommand_JSONNamesTheNextPage(t *testing.T) { diff --git a/internal/adapters/inbound/cli/skills_contract_test.go b/internal/adapters/inbound/cli/skills_contract_test.go index 166b0c3..aae9d50 100644 --- a/internal/adapters/inbound/cli/skills_contract_test.go +++ b/internal/adapters/inbound/cli/skills_contract_test.go @@ -223,7 +223,11 @@ func TestProjectInstructionsRouteFastAndVerifiedSearchSeparately(t *testing.T) { text := string(raw) for _, phrase := range []string{ "`/ccg` is the fast default", - "at most one `search` call with `limit: 5`", + "environment-aware structured search", + "For this repository, use the local JSON CLI", + "start with one `search` call using `limit: 5`", + "pass the namespace already supplied", + "at most three verbatim `next` calls", "Use `/ccg-search-verify`", "freshness, hybrid source checking, and truncation paging", } { @@ -231,6 +235,64 @@ func TestProjectInstructionsRouteFastAndVerifiedSearchSeparately(t *testing.T) { t.Errorf("project instructions are missing search-mode boundary %q", phrase) } } + if strings.Contains(text, "use ccg MCP tools and Agent Skills first") { + t.Error("project instructions force MCP even when the fast skill selects a repository-local JSON CLI") + } +} + +func TestFastSearchSkillDescribesSoftRetrievalAndAdaptiveVerification(t *testing.T) { + raw, err := os.ReadFile(filepath.Join("..", "..", "..", "..", "skills", "ccg", "SKILL.md")) + if err != nil { + t.Fatal(err) + } + text := strings.ToLower(strings.Join(strings.Fields(string(raw)), " ")) + for _, phrase := range []string{ + "or matching", + "rejects single-term coincidences", + "bm25/idf", + "search first when the path is unknown", + "use it verbatim", + "do not compress a behavior or reason question into keywords", + "repository-local `.ccg.yaml`", + "use json cli", + "--compact", + "compact: true", + "explicit mcp or server-visible", + "mcp tool availability or mcp documentation alone", + "path-only reference lookup", + "before semantic paging", + "do not re-locate a hit with grep", + "each chosen declaration's exact range separately", + "exact-identifier miss", + "repeat the same local graph query", + "name the specific evidence gap", + "material evidence gap can", + "author-recorded design reason", + "one directly relevant author-recorded design reason", + "closes the rationale gap", + "secondary consequences are optional", + "do not trace constructor or dependency-injection wiring", + "state that gap in one sentence", + "do not create a new evidence gap from identifiers", + "do not load a reference merely because declaration bounds are missing", + "do not read tests to corroborate production behavior already established", + "next action must be the final answer", + } { + if !strings.Contains(text, phrase) { + t.Errorf("fast search skill is missing retrieval contract %q", phrase) + } + } + for _, overfit := range []string{ + "default fast budget is two production", + "at most one targeted in-file grep", + "Known filename, identifier, literal, or error text | Grep + Read may start directly", + "Never restart with repository-wide", + "Prefer MCP", + } { + if strings.Contains(text, strings.ToLower(overfit)) { + t.Errorf("fast search skill hard-codes an example-specific budget %q", overfit) + } + } } func TestProjectSkillsDoNotAdvertiseRemovedCommands(t *testing.T) { @@ -330,8 +392,10 @@ func TestProjectSkillsCoverOperationalHazards(t *testing.T) { required := map[string][]string{ "ccg": { "ordinary positive", - "`search` at most once", - "one or two source ranges", + "one initial structured ccg `search`", + "include it in the initial search arguments", + "at most three `next` calls", + "exact declaration", "do not call `get_minimal_context`", "`ccg-search-verify`", }, @@ -413,9 +477,10 @@ func TestProjectSkillsCentralizeSharedOperationalGuidance(t *testing.T) { skillsRoot := filepath.Join("..", "..", "..", "..", "skills") required := map[string][]string{ "ccg": { - "## Task Routing and Entry", - "## Freshness Boundary", - "## Response Budget Rule", + "## Route", + "## Core Loop", + "## Boundaries", + "## Completion", }, "ccg-search-verify": { "## Mandatory Verification", @@ -474,7 +539,8 @@ func TestProjectSkillsKeepCoreDiscoveryBoundedAndDeepAnalysisExplicit(t *testing required := map[string][]string{ "ccg": { "do not invoke `ccg-analyze` automatically", - "one bounded `search`", + "one initial structured ccg `search`", + "at most three `next` calls", "explicit `ccg-analyze` invocation", }, "ccg-analyze": { diff --git a/internal/adapters/inbound/mcp/handler_query.go b/internal/adapters/inbound/mcp/handler_query.go index 2acc333..1128318 100644 --- a/internal/adapters/inbound/mcp/handler_query.go +++ b/internal/adapters/inbound/mcp/handler_query.go @@ -186,6 +186,7 @@ func (h *handlers) search(ctx context.Context, request mcp.CallToolRequest) (*mc offset := request.GetInt("offset", 0) pathPrefix := request.GetString("path", "") includeWeak := request.GetBool("include_weak", false) + compact := request.GetBool("compact", false) if err := validateQueryGraphLimit(limit); err != nil { return finalizeToolResult("", err) } @@ -200,10 +201,14 @@ func (h *handlers) search(ctx context.Context, request mcp.CallToolRequest) (*mc } if namespaces := requestNamespaces(request); len(namespaces) > 0 { - return h.searchFederated(ctx, query, limit, offset, pathPrefix, includeWeak, namespaces) + return h.searchFederated(ctx, query, limit, offset, pathPrefix, includeWeak, compact, namespaces) } - return finalizeToolResult(h.cachedExecute(ctx, "search:", map[string]any{"query": query, "limit": limit, "offset": offset, "path": pathPrefix, "include_weak": includeWeak, "namespace": requestNamespace(request)}, func() (string, error) { + cacheKey := map[string]any{ + "query": query, "limit": limit, "offset": offset, "path": pathPrefix, + "include_weak": includeWeak, "compact": compact, "namespace": requestNamespace(request), + } + return finalizeToolResult(h.cachedExecute(ctx, "search:", cacheKey, func() (string, error) { list, err := searchapp.New(h.deps.Graph.Search).Search(ctx, searchapp.Params{ Query: query, Limit: limit, Offset: offset, PathPrefix: pathPrefix, IncludeWeak: includeWeak, }) @@ -214,7 +219,7 @@ func (h *handlers) search(ctx context.Context, request mcp.CallToolRequest) (*mc log.Info("search completed", "query", query, "file_count", len(list.Files), "weak_filtered", list.WeakFiltered) - result, err := marshalJSON(searchwire.NewResponse(list, query, limit, offset, false)) + result, err := marshalSearchResponse(searchwire.NewResponse(list, query, limit, offset, false), compact) if err != nil { return "", trace.Wrap(err, "marshal result") } @@ -225,9 +230,22 @@ func (h *handlers) search(ctx context.Context, request mcp.CallToolRequest) (*mc // searchFederated fans full-text search out over an explicit namespace set and merges reranked hits. // @intent answer one search across several repositories with per-item namespace labels. // @domainRule each namespace is queried in isolation; every namespace's hits keep their own backend rank when fused. -func (h *handlers) searchFederated(ctx context.Context, query string, limit, offset int, pathPrefix string, includeWeak bool, namespaces []string) (*mcp.CallToolResult, error) { +func (h *handlers) searchFederated( + ctx context.Context, + query string, + limit int, + offset int, + pathPrefix string, + includeWeak bool, + compact bool, + namespaces []string, +) (*mcp.CallToolResult, error) { log := h.logger() - return finalizeToolResult(h.cachedExecute(ctx, "search:", map[string]any{"query": query, "limit": limit, "offset": offset, "path": pathPrefix, "include_weak": includeWeak, "namespaces": namespaces}, func() (string, error) { + cacheKey := map[string]any{ + "query": query, "limit": limit, "offset": offset, "path": pathPrefix, + "include_weak": includeWeak, "compact": compact, "namespaces": namespaces, + } + return finalizeToolResult(h.cachedExecute(ctx, "search:", cacheKey, func() (string, error) { list, err := searchapp.New(h.deps.Graph.Search).SearchFederated(ctx, namespaces, searchapp.Params{ Query: query, Limit: limit, Offset: offset, PathPrefix: pathPrefix, IncludeWeak: includeWeak, }) @@ -237,7 +255,7 @@ func (h *handlers) searchFederated(ctx context.Context, query string, limit, off } log.Info("federated search completed", "query", query, "namespaces", namespaces, "file_count", len(list.Files), "weak_filtered", list.WeakFiltered) - result, err := marshalJSON(searchwire.NewResponse(list, query, limit, offset, true)) + result, err := marshalSearchResponse(searchwire.NewResponse(list, query, limit, offset, true), compact) if err != nil { return "", trace.Wrap(err, "marshal result") } @@ -245,6 +263,16 @@ func (h *handlers) searchFederated(ctx context.Context, query string, limit, off })) } +// marshalSearchResponse selects the full compatibility payload or the compact +// agent view without duplicating the search execution path. +// @intent keep CLI-equivalent compact search semantics on single and federated MCP calls. +func marshalSearchResponse(response searchwire.Response, compact bool) (string, error) { + if compact { + return marshalJSON(response.Compact()) + } + return marshalJSON(response) +} + // getAnnotation returns stored annotation metadata for a graph node. // @intent fetch stored annotation tags and summary data so semantic search results can show business context. // @param request qualified_name is the fully qualified node name whose annotations should be loaded. diff --git a/internal/adapters/inbound/mcp/handlers_test.go b/internal/adapters/inbound/mcp/handlers_test.go index 983d666..68eb676 100644 --- a/internal/adapters/inbound/mcp/handlers_test.go +++ b/internal/adapters/inbound/mcp/handlers_test.go @@ -277,6 +277,39 @@ func TestHandler_Search(t *testing.T) { } } +func TestHandler_Search_CompactKeepsOnlyDecisionEvidence(t *testing.T) { + deps := setupTestDeps(t) + ctx := context.Background() + + testGraphStoreFor(deps).UpsertNodes(ctx, []graph.Node{ + {QualifiedName: "pkg.AuthenticateUser", Kind: graph.NodeKindFunction, Name: "AuthenticateUser", FilePath: "auth.go", StartLine: 1, EndLine: 10, Language: "go"}, + }) + node, _ := testGraphStoreFor(deps).GetNode(ctx, "pkg.AuthenticateUser") + testDBFor(deps).Create(&graph.SearchDocument{ + NodeID: node.ID, Content: "AuthenticateUser authenticates user credentials", Language: "go", + }) + testSearchBackendFor(deps).Rebuild(ctx, testDBFor(deps)) + + result := callTool(t, deps, "search", map[string]any{"query": "authenticate", "limit": 10, "compact": true}) + if result.IsError { + t.Fatalf("search returned error: %s", getTextContent(result)) + } + var payload map[string]any + if err := json.Unmarshal([]byte(getTextContent(result)), &payload); err != nil { + t.Fatalf("decode: %v result=%s", err, getTextContent(result)) + } + file := payload["files"].([]any)[0].(map[string]any) + hit := file["hits"].([]any)[0].(map[string]any) + if _, ok := hit["qualified_name"]; !ok { + t.Fatalf("compact hit lost qualified_name: %v", hit) + } + for _, duplicate := range []string{"id", "name", "file_path"} { + if _, ok := hit[duplicate]; ok { + t.Errorf("compact hit kept duplicate field %q: %v", duplicate, hit) + } + } +} + // A caller that gets a result has to be able to open it and see why it is here, // without a second round trip. func TestHandler_Search_CarriesLineNumbersAndEvidence(t *testing.T) { diff --git a/internal/adapters/inbound/mcp/tools_query.go b/internal/adapters/inbound/mcp/tools_query.go index 2e3508a..4fb8e13 100644 --- a/internal/adapters/inbound/mcp/tools_query.go +++ b/internal/adapters/inbound/mcp/tools_query.go @@ -35,12 +35,19 @@ func queryTools(h *handlers) []server.ServerTool { }, { Tool: mcp.NewTool("search", withFederatedNamespaceParams( - mcp.WithDescription("Full-text search across code nodes, grouped by file. It takes both shapes of query: a symbol you can name (an identifier, a type, a package word, or two or three such words together) and a plain-language question ('how does the graph get built', 'why do we verify the webhook signature'). Named symbols are matched against the indexed nodes, where every term has to appear in the same node; a question is additionally scored against the reasons authors recorded (@intent and @domainRule), and files those reasons justify are appended after the name matches. The answer is a list of files, and every file it shows it shows whole: all of that file's hits are in its 'hits' array. Every hit carries the evidence for it — the signals the query matched (name, path, intent), the node's own @intent tag, and, for a reason-matched hit, the recorded reason and the words of the question written in it ('reason', 'matched_terms'). Candidates nothing can justify are left out and counted in weak_filtered. Two separate signals say whether this is the whole answer: 'truncated' is true when more files answered the query than this page reached, and 'pool_truncated' is true when the page ended at the edge of the candidates that were fetched rather than at the end of the answer — one file whose hits fill the candidate pool leaves nothing for 'truncated' to count while later files are still waiting. Only both false means the search is complete; while either is true, make the call in 'next', which lists the exact calls that read on. Use 'path' to scope results to a module for token-efficient queries."), + mcp.WithDescription("Full-text search across code nodes, grouped by file. It takes both shapes of query: a symbol you can name (an identifier, a type, a package word, or two or three such words together) and a plain-language question ('how does the graph get built', 'why do we verify the webhook signature'). Named symbols require every term in one indexed node. Questions try that precise match first; if it is empty, they retrieve nodes matching any meaningful term and rank nodes higher when they match more distinctive terms. Questions are also scored against reasons authors recorded (@intent and @domainRule), and files those reasons justify are appended after the general matches. The answer is a list of files, and every file it shows it shows whole: all of that file's hits are in its 'hits' array. Every hit carries the evidence for it — the signals the query matched (name, path, intent), the node's own @intent tag, and, for a reason-matched hit, the recorded reason and the words of the question written in it ('reason', 'matched_terms'). Candidates nothing can justify are left out and counted in weak_filtered. Two separate signals say whether this is the whole answer: 'truncated' is true when more files answered the query than this page reached, and 'pool_truncated' is true when the page ended at the edge of the candidates that were fetched rather than at the end of the answer — one file whose hits fill the candidate pool leaves nothing for 'truncated' to count while later files are still waiting. Only both false means the search is complete; while either is true, make the call in 'next', which lists the exact calls that read on. Use 'path' to scope results to a module for token-efficient queries."), mcp.WithString("query", mcp.Description("Search query string"), mcp.Required()), mcp.WithNumber("limit", mcp.Description("Maximum number of files to return; every hit inside a returned file is included"), mcp.DefaultNumber(10)), mcp.WithNumber("offset", mcp.Description("Skip this many files before the page starts, so paging never splits a file"), mcp.DefaultNumber(0)), mcp.WithString("path", mcp.Description("Filter results to file paths starting with this prefix (e.g. internal/auth)")), mcp.WithBoolean("include_weak", mcp.Description("Also return candidates whose name, path, and @intent say nothing about the query; they are appended after the justified results")), + mcp.WithBoolean( + "compact", + mcp.Description( + "Return a smaller agent-oriented payload that preserves file paths, declaration bounds, "+ + "rank evidence, completion signals, and next actions while omitting redundant ids and per-hit names/paths", + ), + ), )...), Handler: h.search, }, diff --git a/internal/adapters/outbound/searchsql/backend.go b/internal/adapters/outbound/searchsql/backend.go index cbc4aa6..a9e1f65 100644 --- a/internal/adapters/outbound/searchsql/backend.go +++ b/internal/adapters/outbound/searchsql/backend.go @@ -103,3 +103,7 @@ func loadNodesInOrder(ctx context.Context, db *gorm.DB, nodeIDs []uint) ([]graph // where a question can match more than this many reasons would be ranked on a // biased sample. const maxIntentCandidates = 10000 + +// maxNaturalCandidates is a runaway guard for soft retrieval, not a result +// limit. Ranking needs the full any-term candidate set to measure term rarity. +const maxNaturalCandidates = 10000 diff --git a/internal/adapters/outbound/searchsql/postgres.go b/internal/adapters/outbound/searchsql/postgres.go index 8a1ff86..00c9830 100644 --- a/internal/adapters/outbound/searchsql/postgres.go +++ b/internal/adapters/outbound/searchsql/postgres.go @@ -10,6 +10,7 @@ import ( "github.com/tae2089/trace" "github.com/tae2089/code-context-graph/internal/app/search/intentrank" + "github.com/tae2089/code-context-graph/internal/app/search/queryterm" requestctx "github.com/tae2089/code-context-graph/internal/ctx" "github.com/tae2089/code-context-graph/internal/db/migration" "github.com/tae2089/code-context-graph/internal/domain/graph" @@ -136,12 +137,13 @@ func (p *PostgresBackend) matchRows(ctx context.Context, db *gorm.DB, tsQuery, n // Query searches for related nodes using PostgreSQL tsquery. // -// Every term is required, mirroring the SQLite backend. See SQLiteBackend.Query -// for why widening to any-term was measured and rejected. +// Every meaningful term is required first, mirroring the SQLite backend. A +// sentence-shaped query widens to any-term retrieval only when that strict pass +// is empty, then uses the same Go scorer as SQLite. // // @intent Converts the user's search term into a prefix tsquery to find related nodes. // @requires limit must be greater than 0 to get meaningful results. -// @return Returns a list of nodes sorted by ts_rank. +// @return Returns strict matches by ts_rank, or soft sentence matches by shared relevance rank when strict retrieval is empty. func (p *PostgresBackend) Query(ctx context.Context, db *gorm.DB, query string, limit int) ([]graph.Node, error) { if limit <= 0 { return nil, fmt.Errorf("limit must be > 0, got %d", limit) @@ -162,6 +164,13 @@ func (p *PostgresBackend) Query(ctx context.Context, db *gorm.DB, query string, // driven by an agent quoting identifiers out of code it has already read, so // a query that matches nothing exactly is naming something that does not // exist. Returning nothing says that; returning a near neighbour would not. + if len(rows) == 0 && queryterm.IsNaturalLanguage(query) { + docs, matchErr := p.matchContent(ctx, db, query, maxNaturalCandidates) + if matchErr != nil { + return nil, matchErr + } + return rankContentCandidates(ctx, db, query, docs, limit) + } if len(rows) == 0 { return nil, nil } @@ -177,6 +186,33 @@ func (p *PostgresBackend) Query(ctx context.Context, db *gorm.DB, query string, return promoteExactNameMatch(nodes, query), nil } +// matchContent retrieves every indexed code document containing any meaningful +// term of a natural-language query and leaves relevance ordering to shared Go +// scoring. +// @intent retrieve PostgreSQL soft-match candidates without backend-specific ranking. +func (p *PostgresBackend) matchContent(ctx context.Context, db *gorm.DB, query string, maxCandidates int) ([]intentrank.Doc, error) { + if maxCandidates <= 0 { + return nil, fmt.Errorf("maxCandidates must be > 0, got %d", maxCandidates) + } + tsQuery := SanitizePostgresNaturalTSQuery(query) + if tsQuery == "" { + return nil, nil + } + + var docs []intentrank.Doc + if err := db.WithContext(ctx).Raw(` + SELECT sd.node_id, sd.content, + n.file_path, n.qualified_name, n.kind, n.namespace, n.start_line + FROM search_documents sd + JOIN nodes n ON n.id = sd.node_id + WHERE sd.tsv @@ to_tsquery('simple', ?) + AND sd.namespace = ? + LIMIT ?`, tsQuery, requestctx.FromContext(ctx), maxCandidates).Scan(&docs).Error; err != nil { + return nil, trace.Wrap(err, "natural-language ts_query") + } + return docs, nil +} + // MatchIntent finds every recorded reason holding any term of the question. // // It used to order by ts_rank, which is where the deployment gap lived: ts_rank diff --git a/internal/adapters/outbound/searchsql/retrieval.go b/internal/adapters/outbound/searchsql/retrieval.go index 33aa904..d39b556 100644 --- a/internal/adapters/outbound/searchsql/retrieval.go +++ b/internal/adapters/outbound/searchsql/retrieval.go @@ -39,6 +39,49 @@ func (r *Reader) Query(ctx context.Context, query string, limit int) ([]graph.No return r.backend.Query(ctx, r.db, query, limit) } +// rankContentCandidates gives natural-language retrieval one backend-neutral +// order. The SQL engines only admit documents containing any meaningful query +// term; the same BM25/IDF implementation then rewards both distinctive terms +// and coverage of more of the question. +// @intent rank soft-matched general search documents identically on SQLite and PostgreSQL. +func rankContentCandidates(ctx context.Context, db *gorm.DB, query string, docs []intentrank.Doc, limit int) ([]graph.Node, error) { + if len(docs) == 0 { + return nil, nil + } + var corpusSize int64 + if err := db.WithContext(ctx).Model(&graph.SearchDocument{}). + Where("namespace = ?", requestctx.FromContext(ctx)). + Count(&corpusSize).Error; err != nil { + return nil, trace.Wrap(err, "count indexed search documents") + } + ranked := intentrank.Rank(query, docs, int(corpusSize), len(docs)) + nodeIDs := make([]uint, 0, min(limit, len(ranked.Matches))) + for _, match := range ranked.Matches { + // OR admits candidates; it does not make a single coincidental word an + // answer to a sentence. Requiring two distinct content terms preserves + // the useful widening while dropping common one-word neighbours. + if distinctTermCount(match.Terms) < 2 { + continue + } + nodeIDs = append(nodeIDs, match.NodeID) + if len(nodeIDs) >= limit { + break + } + } + return loadNodesInOrder(ctx, db, nodeIDs) +} + +// distinctTermCount prevents a repeated query word from satisfying the +// multi-signal guard by itself. +// @intent count independent soft-match signals rather than repeated wording. +func distinctTermCount(terms []string) int { + seen := make(map[string]struct{}, len(terms)) + for _, term := range terms { + seen[term] = struct{}{} + } + return len(seen) +} + // QueryIntent answers a question from the recorded-reason index only. // // The database finds the candidates and Go ranks them. That split is what makes diff --git a/internal/adapters/outbound/searchsql/retrieval_test.go b/internal/adapters/outbound/searchsql/retrieval_test.go new file mode 100644 index 0000000..b9bd307 --- /dev/null +++ b/internal/adapters/outbound/searchsql/retrieval_test.go @@ -0,0 +1,130 @@ +//go:build fts5 + +package searchsql + +import ( + "context" + "testing" + + searchapp "github.com/tae2089/code-context-graph/internal/app/search" + "github.com/tae2089/code-context-graph/internal/domain/graph" +) + +func TestReaderQuery_NaturalLanguageUsesSoftContentRanking(t *testing.T) { + db := setupTestDB(t) + backend := NewSQLiteBackend() + if err := backend.Migrate(db); err != nil { + t.Fatal(err) + } + + nodes := []graph.Node{ + {Name: "SyncQueue", QualifiedName: "reposync.SyncQueue", Kind: graph.NodeKindType, FilePath: "internal/app/reposync/queue.go", Language: "go"}, + {Name: "WebhookHandler", QualifiedName: "webhook.WebhookHandler", Kind: graph.NodeKindType, FilePath: "internal/adapters/inbound/webhook/handler.go", Language: "go"}, + {Name: "GraphStats", QualifiedName: "status.GraphStats", Kind: graph.NodeKindType, FilePath: "internal/status/graph.go", Language: "go"}, + } + for i := range nodes { + if err := db.Create(&nodes[i]).Error; err != nil { + t.Fatal(err) + } + } + + docs := []graph.SearchDocument{ + {NodeID: nodes[0].ID, Content: "SyncQueue coalesces webhook push graph updates into a background job", Language: "go"}, + {NodeID: nodes[1].ID, Content: "WebhookHandler receives a GitHub push webhook", Language: "go"}, + {NodeID: nodes[2].ID, Content: "GraphStats reports graph state", Language: "go"}, + } + for i := range docs { + if err := db.Create(&docs[i]).Error; err != nil { + t.Fatal(err) + } + } + if err := backend.Rebuild(context.Background(), db); err != nil { + t.Fatal(err) + } + + reader := NewReader(db, backend) + query := "why are graph updates processed as a separate background job instead of immediately when a GitHub push webhook is received" + got, err := reader.Query(context.Background(), query, 10) + if err != nil { + t.Fatal(err) + } + if len(got) == 0 { + t.Fatal("natural-language query returned no candidates") + } + if got[0].QualifiedName != "reposync.SyncQueue" { + t.Fatalf("first candidate = %q, want reposync.SyncQueue", got[0].QualifiedName) + } +} + +func TestSearchService_NaturalLanguageReturnsJustifiedSoftCandidate(t *testing.T) { + db := setupTestDB(t) + sqlDB, err := db.DB() + if err != nil { + t.Fatal(err) + } + sqlDB.SetMaxOpenConns(1) + backend := NewSQLiteBackend() + if err := backend.Migrate(db); err != nil { + t.Fatal(err) + } + + node := graph.Node{ + Name: "SyncQueue", QualifiedName: "reposync.SyncQueue", Kind: graph.NodeKindType, + FilePath: "internal/app/reposync/queue.go", Language: "go", + } + if err := db.Create(&node).Error; err != nil { + t.Fatal(err) + } + annotation := graph.Annotation{NodeID: node.ID, Tags: []graph.DocTag{{ + Kind: graph.TagIntent, Value: "coalesce webhook push graph updates into a background job", + }}} + if err := db.Create(&annotation).Error; err != nil { + t.Fatal(err) + } + if err := db.Create(&graph.SearchDocument{ + NodeID: node.ID, Content: "SyncQueue coalesces webhook push graph updates into a background job", Language: "go", + }).Error; err != nil { + t.Fatal(err) + } + if err := backend.Rebuild(context.Background(), db); err != nil { + t.Fatal(err) + } + + query := "why are graph updates processed as a separate background job instead of immediately when a GitHub push webhook is received" + list, err := searchapp.New(NewReader(db, backend)).Search(context.Background(), searchapp.Params{Query: query, Limit: 5}) + if err != nil { + t.Fatal(err) + } + if len(list.Files) != 1 || len(list.Files[0].Hits) != 1 { + t.Fatalf("files = %#v, want one justified SyncQueue hit", list.Files) + } + if got := list.Files[0].Hits[0]; got.Node.QualifiedName != "reposync.SyncQueue" || len(got.Matched) == 0 { + t.Fatalf("hit = %#v, want justified reposync.SyncQueue", got) + } +} + +func TestReaderQuery_ShortMultiTermQueryRemainsStrict(t *testing.T) { + db := setupTestDB(t) + seedNodes(t, db) + backend := NewSQLiteBackend() + if err := backend.Migrate(db); err != nil { + t.Fatal(err) + } + if err := backend.Rebuild(context.Background(), db); err != nil { + t.Fatal(err) + } + + got, err := NewReader(db, backend).Query(context.Background(), "session credentials", 10) + if err != nil { + t.Fatal(err) + } + if len(got) != 0 { + t.Fatalf("strict query returned %d candidates, want none", len(got)) + } +} + +func TestDistinctTermCount_DoesNotCountRepeatedWordsTwice(t *testing.T) { + if got := distinctTermCount([]string{"graph", "graph", "updates"}); got != 2 { + t.Fatalf("distinctTermCount = %d, want 2", got) + } +} diff --git a/internal/adapters/outbound/searchsql/sanitize.go b/internal/adapters/outbound/searchsql/sanitize.go index a7fd9ce..6b1f22c 100644 --- a/internal/adapters/outbound/searchsql/sanitize.go +++ b/internal/adapters/outbound/searchsql/sanitize.go @@ -16,30 +16,28 @@ import ( // @intent build SQLite FTS queries that preserve prefix matching without exposing parser-breaking characters. // @domainRule empty or fully stripped input returns an empty query string. func SanitizeFTS5(query string) string { - return buildPrefixQuery(query, alwaysPrefix(`"`+"%s"+`"*`), " AND ", " OR ", " ") + return buildPrefixQuery(query, alwaysPrefix(`"`+"%s"+`"*`), " AND ", " OR ", " AND ") } -// SanitizeIntentFTS5 converts a question into an any-term FTS5 prefix query for -// the intent index. +// SanitizeNaturalFTS5 converts prose into an any-term FTS5 query. // // SanitizeFTS5 requires every term because the searcher there typed identifiers, // and an identifier a caller half-remembers is still worth demanding in full. -// An intent question is the opposite: it is a sentence aimed at another sentence, -// and no recorded reason will contain all of "why do we verify the signature on a -// push". Requiring every term would answer almost nothing. -// -// Any-term is only safe here because this index holds nothing but recorded -// reasons. The same widening was measured on the shared index and rejected: there -// a common word could match an identifier, a path segment, or a language alias, -// so widening pulled in fifty unrelated nodes. With the names removed, a term can -// only match prose somebody wrote on purpose, and bm25 discounts the words that -// appear in many of those. -// @intent let a sentence-shaped question match a sentence-shaped reason. -// @domainRule any-term matching is confined to the intent index, which holds no identifier text. -func SanitizeIntentFTS5(query string) string { +// Prose is the opposite: no useful code document will contain every word of +// "why do we verify the signature on a push". Stopword removal, any-term +// retrieval, and shared BM25/IDF scoring together let distinctive terms win +// without requiring every term to occur in one document. +// @intent retrieve candidates for a sentence-shaped query without requiring every content word in one document. +func SanitizeNaturalFTS5(query string) string { return buildPrefixQuery(query, intentTerm(`"`+"%s"+`"*`, `"`+"%s"+`"`), " AND ", " OR ", " OR ") } +// SanitizeIntentFTS5 applies the natural-language query shape to recorded reasons. +// @intent keep intent retrieval on the same soft-matching syntax as general prose retrieval. +func SanitizeIntentFTS5(query string) string { + return SanitizeNaturalFTS5(query) +} + // SanitizePostgresTSQuery converts raw user input into a safe prefix tsquery, // mirroring SanitizeFTS5 including camelCase sub-token expansion. // @intent translate free-form user input into a PostgreSQL tsquery that mirrors the SQLite prefix search behavior. @@ -48,31 +46,32 @@ func SanitizePostgresTSQuery(query string) string { return buildPrefixQuery(query, alwaysPrefix("%s:*"), " & ", " | ", " & ") } -// SanitizePostgresIntentTSQuery is the PostgreSQL twin of SanitizeIntentFTS5: -// any term may match, because no recorded reason contains every word of a -// question. See SanitizeIntentFTS5 for why that widening is confined to the -// intent index. -// @intent let a sentence-shaped question match a sentence-shaped reason on PostgreSQL. -// @domainRule any-term matching is confined to the intent index, which holds no identifier text. -func SanitizePostgresIntentTSQuery(query string) string { +// SanitizePostgresNaturalTSQuery is the PostgreSQL twin of SanitizeNaturalFTS5. +// @intent retrieve PostgreSQL candidates for prose with any-term matching. +func SanitizePostgresNaturalTSQuery(query string) string { return buildPrefixQuery(query, intentTerm("%s:*", "%s"), " & ", " | ", " | ") } +// SanitizePostgresIntentTSQuery applies the natural-language query shape to recorded reasons. +// @intent keep PostgreSQL intent retrieval on the same soft-matching syntax as general prose retrieval. +func SanitizePostgresIntentTSQuery(query string) string { + return SanitizePostgresNaturalTSQuery(query) +} + // alwaysPrefix renders every term as a prefix, whatever it looks like. // @intent keep prefix expansion the default for the shared search index. func alwaysPrefix(termFmt string) func(string) string { return func(tok string) string { return strings.Replace(termFmt, "%s", tok, 1) } } -// intentTerm renders one term of an intent question, choosing prefix or exact +// intentTerm renders one term of a natural-language question, choosing prefix or exact // matching by intentrank.MatchesByPrefix. // // The rule lives with the scorer rather than here because both have to apply the // same one: the index decides what a term matches, the scorer decides what that // match is worth, and a term matched one way and scored the other would order // the answer by evidence from a query that never ran. -// @intent keep a short question word from reaching an identifier spelled inside a recorded reason. -// @domainRule only the intent index narrows a term to an exact match. +// @intent keep a short question word from reaching a longer identifier only because it shares a prefix. func intentTerm(prefixFmt, exactFmt string) func(string) string { return func(tok string) string { if intentrank.MatchesByPrefix(tok) { diff --git a/internal/adapters/outbound/searchsql/sanitize_test.go b/internal/adapters/outbound/searchsql/sanitize_test.go index 4f96459..aa9aae4 100644 --- a/internal/adapters/outbound/searchsql/sanitize_test.go +++ b/internal/adapters/outbound/searchsql/sanitize_test.go @@ -13,7 +13,7 @@ func TestSanitizeFTS5_SplitsCamelCaseTokens(t *testing.T) { }{ {query: "", want: ""}, {query: "user", want: `"user"*`}, // 단일 단어 불변 - {query: "get user", want: `"get"* "user"*`}, // 소문자 멀티토큰 불변 + {query: "get user", want: `"get"* AND "user"*`}, // 소문자 멀티토큰 불변 {query: "getUser", want: `("getuser"* OR ("get"* AND "user"*))`}, // camelCase 분할 {query: "UserService", want: `("userservice"* OR ("user"* AND "service"*))`}, } @@ -51,19 +51,19 @@ func TestSanitize_DropsFunctionWords(t *testing.T) { { name: "a question keeps only its content words", query: "what stops the server", - fts5: `"stops"* "server"*`, + fts5: `"stops"* AND "server"*`, pg: "stops:* & server:*", }, { name: "a query made only of function words keeps them", query: "how does the", - fts5: `"how"* "does"* "the"*`, + fts5: `"how"* AND "does"* AND "the"*`, pg: "how:* & does:* & the:*", }, { name: "code words that read like function words survive", query: "get set list new", - fts5: `"get"* "set"* "list"* "new"*`, + fts5: `"get"* AND "set"* AND "list"* AND "new"*`, pg: "get:* & set:* & list:* & new:*", }, } @@ -129,11 +129,21 @@ func TestSanitizeIntent_KeepsShortLatinTermsExact(t *testing.T) { } } +func TestSanitizeNatural_UsesAnyMeaningfulTerm(t *testing.T) { + query := "why are graph updates processed as a background job" + if got, want := SanitizeNaturalFTS5(query), `"graph"* OR "updates"* OR "processed"* OR "background"* OR "job"`; got != want { + t.Errorf("SanitizeNaturalFTS5 = %q, want %q", got, want) + } + if got, want := SanitizePostgresNaturalTSQuery(query), "graph:* | updates:* | processed:* | background:* | job"; got != want { + t.Errorf("SanitizePostgresNaturalTSQuery = %q, want %q", got, want) + } +} + // The shared index keeps prefix expansion for every term. A caller there typed a // symbol out of code it already read, so `get` reaching getAnnotation is the // answer rather than the mistake. func TestSanitizeSearch_KeepsShortTermsAsPrefixes(t *testing.T) { - if got, want := SanitizeFTS5("lock get keeps"), `"lock"* "get"* "keeps"*`; got != want { + if got, want := SanitizeFTS5("lock get keeps"), `"lock"* AND "get"* AND "keeps"*`; got != want { t.Errorf("SanitizeFTS5 = %q, want %q", got, want) } if got, want := SanitizePostgresTSQuery("lock get keeps"), "lock:* & get:* & keeps:*"; got != want { diff --git a/internal/adapters/outbound/searchsql/sqlite.go b/internal/adapters/outbound/searchsql/sqlite.go index b7f0435..9e6ee7a 100644 --- a/internal/adapters/outbound/searchsql/sqlite.go +++ b/internal/adapters/outbound/searchsql/sqlite.go @@ -13,6 +13,7 @@ import ( "github.com/tae2089/trace" "github.com/tae2089/code-context-graph/internal/app/search/intentrank" + "github.com/tae2089/code-context-graph/internal/app/search/queryterm" requestctx "github.com/tae2089/code-context-graph/internal/ctx" "github.com/tae2089/code-context-graph/internal/domain/graph" ) @@ -318,19 +319,15 @@ func (s *SQLiteBackend) matchRows(ctx context.Context, db *gorm.DB, ftsQuery, ns // Query searches for related nodes using FTS5 MATCH queries. // -// Every term is required, and SanitizeFTS5 decides what counts as a term. That -// pairing is the whole retrieval policy: requiring all terms is right when the -// searcher typed identifiers, and it only became wrong for sentences because -// ordinary English words were being required too. -// -// Widening to any-term when all-terms matches nothing was measured and -// rejected. It answered no query the narrow expression missed — the two extra -// nodes it retrieved never reached the top ten — and it filled the deliberate -// nonsense query in the golden set with fifty unrelated hits. +// Every meaningful term is required first. That preserves the precision of +// identifier queries and sentence queries whose content words really do meet +// in one document. Only a sentence-shaped query with no strict hit widens to +// any-term retrieval and shared BM25/IDF scoring; compact identifier queries +// never widen. // // @intent Converts the user's search term into a SQLite FTS prefix query to find nodes. // @requires limit must be greater than 0 to get meaningful results. -// @return Returns a list of nodes sorted by FTS rank. +// @return Returns strict matches by FTS rank, or soft sentence matches by shared relevance rank when strict retrieval is empty. func (s *SQLiteBackend) Query(ctx context.Context, db *gorm.DB, query string, limit int) ([]graph.Node, error) { if limit <= 0 { return nil, fmt.Errorf("limit must be > 0, got %d", limit) @@ -345,6 +342,13 @@ func (s *SQLiteBackend) Query(ctx context.Context, db *gorm.DB, query string, li if err != nil { return nil, err } + if len(rows) == 0 && queryterm.IsNaturalLanguage(query) { + docs, matchErr := s.matchContent(ctx, db, query, maxNaturalCandidates) + if matchErr != nil { + return nil, matchErr + } + return rankContentCandidates(ctx, db, query, docs, limit) + } if len(rows) == 0 { return nil, nil } @@ -360,6 +364,32 @@ func (s *SQLiteBackend) Query(ctx context.Context, db *gorm.DB, query string, li return promoteExactNameMatch(nodes, query), nil } +// matchContent retrieves every indexed code document containing any meaningful +// term of a natural-language query. It deliberately leaves ordering to shared +// Go scoring so SQLite and PostgreSQL cannot disagree about relevance. +// @intent retrieve soft-matched general search candidates for backend-neutral ranking. +func (s *SQLiteBackend) matchContent(ctx context.Context, db *gorm.DB, query string, maxCandidates int) ([]intentrank.Doc, error) { + if maxCandidates <= 0 { + return nil, fmt.Errorf("maxCandidates must be > 0, got %d", maxCandidates) + } + ftsQuery := SanitizeNaturalFTS5(query) + if ftsQuery == "" { + return nil, nil + } + + var docs []intentrank.Doc + if err := db.WithContext(ctx).Raw( + `SELECT CAST(search_fts.node_id AS INTEGER) AS node_id, search_fts.content, + n.file_path, n.qualified_name, n.kind, n.namespace, n.start_line + FROM search_fts + JOIN nodes n ON n.id = CAST(search_fts.node_id AS INTEGER) + WHERE search_fts MATCH ? AND search_fts.namespace = ? + LIMIT ?`, ftsQuery, requestctx.FromContext(ctx), maxCandidates).Scan(&docs).Error; err != nil { + return nil, trace.Wrap(err, "natural-language fts query") + } + return docs, nil +} + // MatchIntent finds every recorded reason holding any term of the question. // // FTS5 could order these by bm25 and used to, but the ordering moved to diff --git a/internal/app/search/queryterm/queryterm.go b/internal/app/search/queryterm/queryterm.go index 3e2f01b..a6811d5 100644 --- a/internal/app/search/queryterm/queryterm.go +++ b/internal/app/search/queryterm/queryterm.go @@ -5,7 +5,11 @@ // the searcher asked for. package queryterm -import "strings" +import ( + "strings" + + "github.com/tae2089/code-context-graph/internal/app/search/identtoken" +) // functionWords are English words that carry no meaning in a corpus of code // identifiers and one-line annotations. They hurt both searches, in opposite @@ -34,6 +38,15 @@ var functionWords = map[string]bool{ "would": true, "should": true, "will": true, "shall": true, "there": true, } +// questionWords identify queries whose grammar says they are prose even when +// only a few content words survive stopword removal. They are kept separate +// from functionWords because the two sets answer different questions: one +// chooses a retrieval shape, while the other chooses terms worth matching. +var questionWords = map[string]bool{ + "how": true, "what": true, "where": true, "when": true, + "why": true, "who": true, "which": true, +} + // IsFunctionWord reports whether a term is one of the meaningless words. // @intent let a caller judge one term without copying the list. func IsFunctionWord(term string) bool { @@ -57,3 +70,24 @@ func DropFunctionWords(tokens []string) []string { } return kept } + +// IsNaturalLanguage reports whether a query should retrieve documents by any +// meaningful term and let shared scoring combine the evidence. +// +// Two or three code terms are normally a partially remembered identifier and +// retain strict all-term matching. Prose either announces itself with a +// question word or contains more than three meaningful terms. The +// count happens after function-word removal so "get user by id" remains the +// compact identifier-shaped query "get user id". +// @intent choose soft retrieval only for sentence-shaped queries while preserving precise identifier lookup. +// @domainRule compact queries of at most three meaningful terms remain strict unless their grammar explicitly asks a question. +func IsNaturalLanguage(query string) bool { + tokens := identtoken.FieldsLower(query) + meaningful := DropFunctionWords(tokens) + for _, token := range tokens { + if questionWords[token] && len(meaningful) >= 2 { + return true + } + } + return len(meaningful) > 3 +} diff --git a/internal/app/search/queryterm/queryterm_test.go b/internal/app/search/queryterm/queryterm_test.go new file mode 100644 index 0000000..e8bd341 --- /dev/null +++ b/internal/app/search/queryterm/queryterm_test.go @@ -0,0 +1,27 @@ +package queryterm + +import "testing" + +func TestIsNaturalLanguage(t *testing.T) { + tests := []struct { + name string + query string + want bool + }{ + {name: "question marker", query: "why graph updates", want: true}, + {name: "explanation with question marker", query: "explain why graph updates", want: true}, + {name: "long descriptive phrase", query: "graph updates use a background job", want: true}, + {name: "three code terms", query: "session token credentials", want: false}, + {name: "function word does not widen", query: "get user by id", want: false}, + {name: "single identifier", query: "buildOrUpdateGraph", want: false}, + {name: "ambiguous command name", query: "explain", want: false}, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + if got := IsNaturalLanguage(tt.query); got != tt.want { + t.Fatalf("IsNaturalLanguage(%q) = %v, want %v", tt.query, got, tt.want) + } + }) + } +} diff --git a/internal/app/search/service.go b/internal/app/search/service.go index 5bd1b6a..8eccc94 100644 --- a/internal/app/search/service.go +++ b/internal/app/search/service.go @@ -8,11 +8,13 @@ package search import ( "context" + "sort" "github.com/tae2089/trace" "github.com/tae2089/code-context-graph/internal/app/search/evidence" intentapp "github.com/tae2089/code-context-graph/internal/app/search/intent" + "github.com/tae2089/code-context-graph/internal/app/search/queryterm" searchrank "github.com/tae2089/code-context-graph/internal/app/search/rank" requestctx "github.com/tae2089/code-context-graph/internal/ctx" "github.com/tae2089/code-context-graph/internal/domain/graph" @@ -232,6 +234,9 @@ func (s *Service) fetch(ctx context.Context, p Params) (pool, error) { // @ensures the order of the first n rows does not depend on any row after them, at every n that is a multiple of block. // @intent keep a page already delivered from being reshuffled by the wider pool the next page fetches. func orderPool(query string, nodes []graph.Node, block int) []graph.Node { + if queryterm.IsNaturalLanguage(query) { + return nodes + } if block <= 0 || len(nodes) <= block { return searchrank.Rerank(query, nodes, 0) } @@ -255,6 +260,9 @@ func orderPool(query string, nodes []graph.Node, block int) []graph.Node { // @requires each group is that repository's rank-ordered pool, fetched in whole multiples of block. // @intent give federated paging the same fixed prefix a single repository's paging has. func orderGroupedPool(query string, groups [][]graph.Node, block int) []graph.Node { + if queryterm.IsNaturalLanguage(query) { + return mergeRankedGroups(groups) + } widest, total := 0, 0 for _, g := range groups { widest = max(widest, len(g)) @@ -277,6 +285,33 @@ func orderGroupedPool(query string, groups [][]graph.Node, block int) []graph.No return out } +// mergeRankedGroups combines backend-neutral natural-language rankings without +// replacing them with identifier-oriented structural scores. Items at the same +// rank across repositories use the canonical node identity as their stable tie +// break, so changing namespace input order cannot change the answer. +// @intent preserve soft-match relevance while merging independently ranked namespace results. +func mergeRankedGroups(groups [][]graph.Node) []graph.Node { + widest, total := 0, 0 + for _, group := range groups { + widest = max(widest, len(group)) + total += len(group) + } + out := make([]graph.Node, 0, total) + for position := range widest { + level := make([]graph.Node, 0, len(groups)) + for _, group := range groups { + if position < len(group) { + level = append(level, group[position]) + } + } + sort.SliceStable(level, func(i, j int) bool { + return graph.CompareIdentity(level[i].Identity(), level[j].Identity()) < 0 + }) + out = append(out, level...) + } + return out +} + // keepPathPrefix drops the candidates that live outside the caller's path // filter, once the answer's order is already decided. // diff --git a/internal/app/search/service_test.go b/internal/app/search/service_test.go index 6fa367c..169d8b0 100644 --- a/internal/app/search/service_test.go +++ b/internal/app/search/service_test.go @@ -66,6 +66,39 @@ func node(id uint, name, path string) graph.Node { return graph.Node{ID: id, Name: name, QualifiedName: name, FilePath: path, Kind: "function"} } +func TestOrderPool_NaturalLanguagePreservesSoftMatchRank(t *testing.T) { + query := "why are graph updates processed as a separate background job" + softRanked := []graph.Node{ + node(1, "SyncQueue", "internal/app/reposync/queue.go"), + node(2, "GraphUpdates", "internal/graph/updates.go"), + } + + got := orderPool(query, softRanked, 10) + if got[0].ID != 1 { + t.Fatalf("first node = %d, want soft-ranked node 1", got[0].ID) + } +} + +func TestMergeRankedGroups_NaturalLanguagePreservesRanksAndStableTies(t *testing.T) { + query := "explain graph update background processing" + alpha := []graph.Node{ + node(1, "A1", "z/a1.go"), + node(2, "A2", "a/a2.go"), + } + beta := []graph.Node{ + node(3, "B1", "a/b1.go"), + node(4, "B2", "z/b2.go"), + } + + got := orderGroupedPool(query, [][]graph.Node{alpha, beta}, 10) + want := []uint{3, 1, 2, 4} + for i, id := range want { + if got[i].ID != id { + t.Fatalf("result[%d] = %d, want %d", i, got[i].ID, id) + } + } +} + func TestSearch_OverfetchesThenCutsToLimit(t *testing.T) { pool := []graph.Node{ node(1, "alpha", "a/alpha.go"), diff --git a/internal/app/search/wire/wire.go b/internal/app/search/wire/wire.go index 5f10cd3..171fa86 100644 --- a/internal/app/search/wire/wire.go +++ b/internal/app/search/wire/wire.go @@ -92,6 +92,99 @@ type Response struct { Note string `json:"note,omitempty"` } +// CompactResultItem keeps the evidence an agent needs to choose and open a hit, +// without repeating data already present in its file group. +// @intent reduce search-response context while preserving rank evidence and exact source bounds. +type CompactResultItem struct { + QualifiedName string `json:"qualified_name"` + Kind graph.NodeKind `json:"kind"` + StartLine int `json:"start_line"` + EndLine int `json:"end_line"` + Intent string `json:"intent,omitempty"` + Matched []evidence.Match `json:"matched,omitempty"` + Reason string `json:"reason,omitempty"` + MatchedTerms []string `json:"matched_terms,omitempty"` +} + +// CompactFileGroup groups compact hits under the one copy of their file path. +// @intent avoid repeating file identity on every hit while keeping federated namespace labels. +type CompactFileGroup struct { + FilePath string `json:"file_path"` + Namespace string `json:"namespace,omitempty"` + Hits []CompactResultItem `json:"hits"` +} + +// CompactResponse is the token-efficient search view used by agents that do +// not need storage ids or redundant per-hit names and paths. +// @intent preserve search decisions, completion signals, and continuations in a smaller wire payload. +type CompactResponse struct { + Files []CompactFileGroup `json:"files"` + WeakFiltered int `json:"weak_filtered"` + Truncated bool `json:"truncated"` + PoolTruncated bool `json:"pool_truncated"` + Limits Limits `json:"limits"` + AnnotationCoverage evidence.Coverage `json:"annotation_coverage"` + Next []NextAction `json:"next,omitempty"` + Note string `json:"note,omitempty"` +} + +// Compact returns a smaller view without changing the full response contract. +// Search continuations retain compact mode so later pages do not grow again. +// @ensures every search action in Next carries compact=true. +// @intent let CLI and MCP share one loss-aware compact representation. +func (r Response) Compact() CompactResponse { + files := make([]CompactFileGroup, len(r.Files)) + for i, file := range r.Files { + hits := make([]CompactResultItem, len(file.Hits)) + for j, hit := range file.Hits { + intent := hit.Intent + if intent == hit.Reason { + intent = "" + } + hits[j] = CompactResultItem{ + QualifiedName: hit.QualifiedName, + Kind: hit.Kind, + StartLine: hit.StartLine, + EndLine: hit.EndLine, + Intent: intent, + Matched: hit.Matched, + Reason: hit.Reason, + MatchedTerms: hit.MatchedTerms, + } + } + files[i] = CompactFileGroup{ + FilePath: file.FilePath, + Namespace: file.Namespace, + Hits: hits, + } + } + + next := make([]NextAction, len(r.Next)) + for i, action := range r.Next { + next[i] = action + if action.Tool != "search" { + continue + } + args := make(map[string]any, len(action.Args)+1) + for key, value := range action.Args { + args[key] = value + } + args["compact"] = true + next[i].Args = args + } + + return CompactResponse{ + Files: files, + WeakFiltered: r.WeakFiltered, + Truncated: r.Truncated, + PoolTruncated: r.PoolTruncated, + Limits: r.Limits, + AnnotationCoverage: r.AnnotationCoverage, + Next: next, + Note: r.Note, + } +} + // Limits states the bounds that shaped this page, all counted in files // except the budget, which only decides whether one more file joins. // @intent let a caller tell a short answer from the first page of a long one. diff --git a/internal/app/search/wire/wire_test.go b/internal/app/search/wire/wire_test.go index 6e72d92..641c4c2 100644 --- a/internal/app/search/wire/wire_test.go +++ b/internal/app/search/wire/wire_test.go @@ -1,6 +1,7 @@ package wire import ( + "encoding/json" "os" "path/filepath" "runtime" @@ -12,6 +13,78 @@ import ( "github.com/tae2089/code-context-graph/internal/domain/graph" ) +func TestResponse_CompactKeepsDecisionEvidenceWithoutDuplicateFields(t *testing.T) { + list := evidence.List{ + Files: []evidence.File{{ + Namespace: "repo-a", + FilePath: "internal/app/search/service.go", + Hits: []evidence.Result{{ + Node: graph.Node{ + ID: 41, + QualifiedName: "search.Service.Search", + Kind: graph.NodeKindFunction, + Name: "Search", + FilePath: "internal/app/search/service.go", + StartLine: 73, + EndLine: 121, + }, + Intent: "rank precise matches before widening a natural-language question", + Matched: []evidence.Match{evidence.MatchName, evidence.MatchIntent}, + Reason: "widen a natural-language question only after the precise match is empty", + MatchedTerms: []string{"natural", "question"}, + }}, + }}, + OverflowFiles: 2, + NextOffset: 1, + Coverage: evidence.Coverage{WithReason: 12, Declarations: 40}, + } + seed := list.Files[0].Hits[0] + list.Files[0].Hits = append(list.Files[0].Hits, seed, seed, seed, seed) + + full := NewResponse(list, "natural question", 1, 0, true) + compact := full.Compact() + raw, err := json.Marshal(compact) + if err != nil { + t.Fatal(err) + } + + var payload map[string]any + if err := json.Unmarshal(raw, &payload); err != nil { + t.Fatal(err) + } + files := payload["files"].([]any) + file := files[0].(map[string]any) + if file["namespace"] != "repo-a" { + t.Errorf("compact file namespace = %v, want repo-a", file["namespace"]) + } + hit := file["hits"].([]any)[0].(map[string]any) + for _, key := range []string{"qualified_name", "kind", "start_line", "end_line", "intent", "matched", "reason", "matched_terms"} { + if _, ok := hit[key]; !ok { + t.Errorf("compact hit is missing %q: %s", key, raw) + } + } + for _, duplicate := range []string{"id", "name", "file_path", "namespace"} { + if _, ok := hit[duplicate]; ok { + t.Errorf("compact hit kept duplicate field %q: %s", duplicate, raw) + } + } + if payload["truncated"] != true || payload["annotation_coverage"] == nil { + t.Errorf("compact response lost completion metadata: %s", raw) + } + next := payload["next"].([]any)[0].(map[string]any) + args := next["args"].(map[string]any) + if args["compact"] != true { + t.Errorf("next args = %v, want compact=true so paging stays compact", args) + } + fullRaw, err := json.Marshal(full) + if err != nil { + t.Fatal(err) + } + if len(raw)*10 >= len(fullRaw)*9 { + t.Errorf("compact response is %d bytes versus %d full bytes, want at least 10%% smaller", len(raw), len(fullRaw)) + } +} + func file(path string, hits int) evidence.File { f := evidence.File{FilePath: path} for i := range hits { diff --git a/skills/ccg/SKILL.md b/skills/ccg/SKILL.md index 47bc7e5..1b779b4 100644 --- a/skills/ccg/SKILL.md +++ b/skills/ccg/SKILL.md @@ -1,8 +1,8 @@ --- name: ccg -description: "Fast read-only code discovery with mandatory structured CCG search for unknown entry points, followed by bounded source verification. Use for ordinary positive lookups, recorded intent, known-path inventory, or one direct relationship fact. Do not use for absence, completeness, exhaustive inventory, deep flow or impact analysis, or graph writes." +description: "Fast read-only code discovery with a bounded code-context-graph search and targeted source verification. Use when an ordinary positive lookup or explanation needs an entry point, recorded intent, known-path inventory, or direct relationship evidence. Do not use for absence, completeness, exhaustive inventory, deep flow or impact analysis, or graph writes; use ccg-search-verify for defensible negative claims, and require explicit invocation for ccg-analyze or ccg-build." metadata: - version: 4.0.1 + version: 4.9.0 openclaw: category: "code-intelligence" domain: "core" @@ -14,92 +14,120 @@ metadata: # ccg — Fast Search -Find one useful CCG candidate, verify it in current source, and stop as soon as -the positive question is answered. +Find the smallest source-verified evidence set that answers an ordinary positive +code question. ## Route -| User intent | Route | +| Request | Route | | --- | --- | -| Known filename, identifier, literal, or error text | Grep + Read may start directly | -| Unknown entry point, behavior, reason, or keyword | Run the Core Loop below | -| Contents of a known file or folder | One `describe` call | +| Source path already known | Targeted Grep + Read | +| Source path unknown, including an exact identifier | Core Loop | +| Known file or folder inventory | One `describe` call | | One direct caller/callee fact | One bounded `query_graph` call | | Absence, completeness, or exhaustive inventory | `ccg-search-verify` | -| Deep flow, change impact, or blast radius | Require explicit `ccg-analyze` invocation | -| Build, update, migrate, postprocess, or graph write | Require explicit `ccg-build` invocation | +| Deep flow or impact analysis | Require explicit `ccg-analyze` invocation | +| Graph write or refresh | Require explicit `ccg-build` invocation | -An ordinary miss never authorizes a negative claim. If the answer would become -“not found,” “does not exist,” “complete,” or “all,” switch to -`ccg-search-verify`. +An ordinary miss is not evidence that code does not exist. Route any negative or +complete claim to `ccg-search-verify`. ## Core Loop -For an unknown entry point, behavior, reason, or keyword, follow these steps in -order: - -1. **Search.** Before grep or source browsing, rewrite the request as a CCG - query; never submit the user's full sentence. Preserve an exact identifier, - path, literal, or error text as one clue. Otherwise select two to four - discriminative repository-language terms that name the component and - behavior and are likely to coexist in one declaration or annotation. Drop - question words, connective prose, and illustrative examples. Run one - structured CCG `search` with that query and `limit: 5`. Prefer MCP. If MCP - is unavailable, use `ccg search --json --limit 5 ""`; never use the - plain CLI display. -2. **Verify this page.** Inspect only relevant `production` and `unknown` paths - returned on the current page. Each source read must verify one concrete - claim. Read the exact declaration and its attached documentation, not broad - file ranges. -3. **Decide.** If verified evidence answers the question, stop. If it does not - and the response contains `next`, the next action must be that continuation - verbatim. Do not grep a wider scope while an allowed continuation remains. -4. **Continue.** Repeat page verification and decision for at most three `next` - calls. Preserve the query, limit, namespace, and offsets supplied by CCG; - never calculate or rewrite them. -5. **Fallback.** Only when `next` is absent or three `next` calls have been used, - search current source directly. The first fallback search may expand scope; - every later search must narrow from a new concrete path, symbol, literal, or - error text found by the preceding step. -6. **Stop.** Answer when the smallest verified evidence set is sufficient. If a - search yields no new clue, stop and report that the entry point could not be - located without claiming that the code does not exist. - -Do not replace a returned path with a guessed directory, reread an examined -range, fan out into synonym searches, or echo raw candidate lists. +1. **Search first when the path is unknown.** Run one initial structured CCG + `search` with `limit: 5`. Use an exact identifier, literal, or error text as a + compact query. If the request already contains one focused code question, use + it verbatim after removing only command wrappers or output instructions. Do + not compress a behavior or reason question into keywords. Otherwise ask one + focused natural-language question in the repository's vocabulary. CCG tries + the precise match first, then uses OR + matching with BM25/IDF, rewards more distinct terms, and rejects single-term + coincidences. Choose the structured surface from available routing evidence: + use MCP when repository instructions provide explicit MCP or server-visible + routing; otherwise, when local `ccg` and a repository-local `.ccg.yaml` exist, + use JSON CLI with `ccg search --json --compact --limit 5 ""`; use MCP + when no usable local configuration exists. Pass `compact: true` to MCP search. + Compact mode keeps paths, declaration bounds, evidence, and continuations + while omitting redundant storage fields. MCP tool availability or MCP + documentation alone is not routing evidence; explicit routing names the + target namespace together with MCP or a server-visible repository path. JSON + CLI reads its namespace and database from `.ccg.yaml`. For MCP, when + configuration supplies a namespace, include it in the initial search + arguments and every continuation. When a namespace must be extracted, read + only the `namespace:` field, never the full configuration. +2. **Choose evidence from the whole page.** Compare the returned file paths, + matched signals, reasons, and declaration hits. Start with the production hit + that most directly addresses the question, not automatically the first row. + Read each chosen declaration's exact range separately; do not span unrelated + hits by reading from a file's earliest result to its latest. Do not re-locate + a hit with grep when `start_line` and `end_line` are present. Use a targeted + in-file locator only for missing bounds or a helper named by the verified + declaration. For a production-behavior question, do not read tests to + corroborate production behavior already established by current source. Tests + become evidence only when the user asks about them or the production source + leaves a material ambiguity. +3. **Stop by claim sufficiency.** Before another tool call, name the specific + evidence gap in the user's question and state that gap in one sentence. If + no material evidence gap can be named, answer. A verified current-source call + site plus the invoked declaration or contract establishes that mechanism; do + not trace constructor or dependency-injection wiring unless the user asks + which runtime implementation or configuration is selected. Use these + completion rules: + - **Where:** the current-source declaration is enough. + - **How/what:** the branch or contract directly implementing the requested + behavior is enough; include public input or output only when asked. + - **Why:** one directly relevant author-recorded design reason plus current + source confirming its mechanism closes the rationale gap unless the user + requests several reasons; secondary consequences are optional. Do not + trace downstream work merely to prove optional consequences. + - **One relationship:** the edge and the endpoint declaration needed to + interpret it are enough. + Mandatory stop: when the matching condition is satisfied, the next action + must be the final answer. Another tool call is allowed only when current + source contradicts the recorded evidence or the user explicitly requested + additional distinct reasons or details. +4. **Follow a new exact clue before semantic paging.** When verified source + reveals an identifier or literal that names a remaining usage, caller, or + exposure gap, run a path-only reference lookup such as `rg -l` before semantic + paging. Print file names only, exclude test-like paths unless relevant, then + read the necessary production declarations. Do not create a new evidence gap + from identifiers encountered after the original question is already answered. + Never print repository-wide match context. +5. **Page only for an unresolved semantic gap.** A truncation flag makes `next` + available; it does not require paging. If the current page and source-derived + exact clues cannot close the named gap, follow the returned `next` call + verbatim. Preserve query, limit, namespace, and offsets. Use at most three + `next` calls. +6. **Handle a miss once.** An exact-identifier miss goes directly to a path-only + source lookup. For a natural-language miss, retry local JSON CLI once only + when it demonstrably uses a different graph or runtime than MCP. Do not repeat + the same local graph query through another interface. If direct source search + yields no new clue, stop without making a negative claim. ## Conditional Details -Read -[`references/search-execution.md`](references/search-execution.md) only when a -branch needs it: - -- MCP is unavailable and CLI fallback is required; -- the user's language differs from repository annotations or comments; -- a candidate may be test-only or declaration bounds are missing; -- the direct-source fallback begins. - -Read -[`references/supported-languages.md`](references/supported-languages.md) only -for a supported-language or extension question. +Do not load a reference merely because declaration bounds are missing; Step 2's +targeted in-file locator is the complete recovery rule. Read +[`references/search-execution.md`](references/search-execution.md) only for +language translation, uncertain test classification, or ambiguous continuation +metadata. Read +[`references/supported-languages.md`](references/supported-languages.md) only for +a language-support question. ## Boundaries -- Current source is authoritative for text, location, and runtime semantics; - CCG supplies ranked intent and relationship candidates. -- Positive evidence verified in current source needs no freshness preflight. - A graph miss or stale result is never negative evidence. -- Do not call `get_minimal_context`, `list_namespaces`, or `list_graph_stats` - when repository instructions or configuration already supply the namespace. -- Use `get_node` only for exact identity or declaration bounds, and follow - another declaration only when the selected declaration directly references - it. -- This skill is read-only. Never invoke graph build, update, migration, - postprocessing, impact-radius, or flow-tracing tools from this workflow. +- Current source is authoritative for location and runtime semantics; CCG ranks + entry points, intent, and relationships. +- Do not call `get_minimal_context`, namespace-list, or graph-stat tools when the + namespace is already configured. +- After starting Core Loop search, do not add `query_graph`, `get_node`, flow, or + impact calls merely to enrich an already supported answer. +- This workflow is read-only. Never build, update, migrate, or postprocess. + Do not invoke `ccg-analyze` automatically; deeper analysis requires explicit + `ccg-analyze` invocation. ## Completion -Return the answer with the relevant path or qualified symbol. Mention -truncation only when the three-continuation cap limits the answer, and disclose -only uncertainty that materially changes the conclusion. Do not append tool, -namespace, freshness, or call-count reports unless requested. +Return the answer with relevant paths or qualified symbols. Mention truncation +only when the continuation cap limits the answer. Do not append namespace, +freshness, or call-count reports unless requested. diff --git a/skills/ccg/references/search-execution.md b/skills/ccg/references/search-execution.md index e02bb4c..1c456cb 100644 --- a/skills/ccg/references/search-execution.md +++ b/skills/ccg/references/search-execution.md @@ -4,32 +4,105 @@ Use only the section required by the active branch of the `ccg` Core Loop. ## Structured Search Interfaces -Prefer MCP `search` with `limit: 5`. When MCP is unavailable, use JSON CLI -output so node IDs, declaration bounds, truncation, and continuation arguments -remain machine-readable: +Use MCP when repository instructions provide explicit MCP or server-visible +routing. Otherwise prefer JSON CLI when local `ccg` and a repository-local +`.ccg.yaml` are available, so the current binary and repository configuration +answer directly. Fast discovery needs declaration bounds, rank evidence, +truncation, and continuation arguments; storage IDs are not required. Use the +compact response on either surface. The CLI form is: ```bash -ccg search --json --limit 5 "" +ccg search --json --compact --limit 5 "" ``` -For MCP, execute the returned `next` call verbatim. For CLI JSON, execute the -command represented by `next.args` without calculating offsets or changing the +For MCP, pass `compact: true` and execute the returned `next` call verbatim. For +CLI JSON, execute the command represented by `next.args` without calculating offsets or changing the query, limit, or namespace. Plain CLI output is not a substitute because it can hide structured continuation and declaration data. +For MCP, include the namespace supplied by repository instructions or +`.ccg.yaml` in the initial call. This is not a freshness or namespace-list +preflight: it is the routing key a shared server needs to select the repository. +Keep that namespace in every returned continuation. If the namespace must be +read from `.ccg.yaml`, extract only the `namespace:` field (for example with a +targeted `rg`) rather than opening the whole file; database configuration may +contain credentials and is irrelevant to read-only search routing. + ## Query Language +When the source path is unknown, use structured CCG search even when an exact +identifier, literal, or error fragment is available. Those clues make excellent +compact CCG queries; they do not justify an unbounded source scan. Direct grep +may start only when its source path is already known. + Use one exact identifier, path, literal, or error fragment for a known thing. -When the symbol is unknown, extract two to four discriminative terms that name -the component and behavior. Prefer repository vocabulary, nouns, and rare -operational terms over question words, generic verbs, connective prose, or the -user's examples. Choose terms likely to occur together in one declaration or -recorded `@intent` or `@domainRule`; do not submit the user's full sentence. +When the symbol is unknown, ask a focused natural-language question that names +the component, behavior, and distinguishing operational context. The search +engine removes English function words and first tries a precise all-term match. +If that is empty, it uses OR retrieval to collect candidates sharing any +meaningful term, rejects single-term coincidences, and applies shared BM25/IDF +ranking. A candidate ranks higher when it matches more distinct query terms and +when those terms are rarer in the indexed corpus. This is lexical matching, not +semantic synonym expansion. Preserve the question's useful evidence instead of +manually compressing it into an AND-friendly keyword list, and use vocabulary +that actually appears in the repository. Write natural-language terms in the dominant language of repository comments and annotations. Translate the user's intent when needed, but preserve exact identifiers, paths, literals, and error messages. Do not mix translations in one -query: every query term must occur in the same indexed document. +query; one focused query in the repository's vocabulary is cheaper and easier +to verify than parallel synonyms. + +If an exact-identifier query returns no files and exposes no continuation, use a +path-only source lookup for that identifier. Do not repeat a lexical index miss +through another interface to the same graph. For a natural-language miss, a +single structured JSON CLI retry is useful only when it is evidenceably a +different runtime or graph — for example, remote MCP versus a repository-local +`.ccg.yaml`. Treat that retry as a replacement result stream and do not page both +MCP and CLI results. When MCP already runs the local `ccg` binary against the +same configuration, go directly to current-source fallback. + +## Sufficiency Before Paging + +One returned file can contain several declaration hits. Verify the relevant hit +ranges on the current page before following `next`; do not treat only the first +declaration in the first file as the whole page. A positive explanation is +complete when ranked intent or domain-rule evidence identifies the design reason +and current production source confirms the behavior being described. Additional +wiring, retries, tests, or examples may be useful context, but they are not a +reason to page or grep after the requested explanation is already supported. +The amount of evidence depends on the claim. Begin with the highest-ranked +relevant production evidence, then inspect another declaration, file, or helper +only when a material part of the requested answer remains unsupported. Before +each extra lookup, state the evidence gap and choose the narrowest operation that +can close it. If no such gap exists, answer instead of collecting optional +detail. + +Treat a declaration hit's `start_line` and `end_line` as its ready-to-read source +range. Read selected ranges directly; do not spend another grep call locating a +symbol whose bounds CCG already returned. Use an in-file locator only for a file +hit, missing bounds, or a newly referenced helper outside the returned range. + +After CCG returns a usable page, ordinary source verification stays within paths +on that page and paths directly referenced by a declaration already verified +there. There is one hybrid handoff: when verified source reveals an exact +identifier or literal that names a remaining usage, caller, or exposure question, +locate its references with a path-only lookup such as `rg -l`. Return file names, +not repository-wide match context; exclude test-like paths unless they are the +subject, then read the relevant production declarations. Prefer this exact +reference lookup before semantic paging because it follows a source-proven clue +instead of asking the broad question again. A truncation flag only says more +candidates are available; follow `next` when a specific part of the user's +question remains unsupported and neither the current page nor an exact +source-derived clue can close it. + +When a returned production file is clearly relevant but its declaration bounds +do not include one referenced helper, run a targeted grep for that exact symbol +or phrase inside the returned file and read its declaration. This remains page +verification. Follow `next` only when the current page failed to identify a +usable production path or symbol, or when current-source verification disproved +the candidates; paging is not a substitute for looking precisely inside a known +candidate. ## Candidate Classification