Skip to content

fix(agent): match tool names across a trailing plural s - #1574

Open
bwachman wants to merge 1 commit into
cl-11366-tool-search-misses-tools-whose-matching-capability-is-afrom
cl-11435-tool-search-misses-tools-when-the-query-and-tool-name-differ
Open

bwachman wants to merge 1 commit into
cl-11366-tool-search-misses-tools-whose-matching-capability-is-afrom
cl-11435-tool-search-misses-tools-when-the-query-and-tool-name-differ

Conversation

@bwachman

Copy link
Copy Markdown
Contributor

Summary

  • Tool, skill, and agent search match a query word against name tokens across a trailing plural "s", so "issue list" finds mcp__linear__list_issues and "issues" finds get_issue. An exact-spelling name match outranks a folded one, so "list issues" and "linear issues" rank list_issues first in any catalog order. Description and parameter tokens keep exact matching.
  • A query word given in both spellings ("issue ... issues") counts once, at whichever spelling scores higher.
  • Known limits, all accepted:
    • On 18 live Linear definitions, the CL-11366 session query ("Linear retrieve pipeline release issue list and issue details") returns list_issues tied for 5th at the result limit, so catalog order decides whether it lands 5th or 6th. Without folding it scores below the cutoff and never returns. The tie sits at the result limit, tie order is outside this issue's scope, and the result is a strict improvement over a miss.
    • The spelling merge applies on every tier, so an unrelated pair like "new" and "news" in one query counts once against a description.
    • Folding inside names also merges real word pairs such as "news" and "new", or "canvas" and "canva".
  • Stacked on fix(agent): match tool_search queries against tool parameters #1573 (CL-11366). This repo keeps merged branches, so after fix(agent): match tool_search queries against tool parameters #1573 merges, retarget this PR's base to main by hand.

Verification

  • bun run check (lint, typecheck, dead exports, build, test) exits 0 on this branch: 9528 pass, 0 fail.
  • Existing tool, skill, and agent search tests, including search-scorer-parity.test.ts, pass with no change to their expected order; the diff adds tests and changes no existing assertion.
  • Against the base branch's scorer, the new singular/plural tests and the Linear-fixture "issue list" test fail (4 fail, 95 pass); the rest guard behavior that holds on both.

Fixes CL-11435

@linear-code

linear-code Bot commented Oct 26, 2026

Copy link
Copy Markdown

CL-11435

@bwachman

Copy link
Copy Markdown
Contributor Author

Self-Review (code-review skill)

The branch carries the following intentional exceptions, each authorized by Greybeard:

  • src/agent/lexical-rank.ts:93-96: query words group by folded form before scoring every tier, so unrelated pairs like "new"/"news" count once against a description. Authorized by Greybeard on grounds that the effect only lowers a score (never a false hit or an empty result) and a second scoring path in the shared scorer costs more than the bug.
  • src/agent/lexical-rank.ts:16-21: name folding merges real word pairs inside tool names (news/new, canvas/canva), which can turn a tool_search miss into a hit. Authorized by Greybeard as an accepted cost of folding tool names.

@TheGreatAxios TheGreatAxios left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review · Request changes

Folds one trailing plural "s" on name tokens so a singular query can match a plural tool name and the reverse, with an exact spelling still scoring higher.

Findings

  • src/agent/lexical-rank.ts:24 — FOLDED_NAME_TOKEN_SCORE is 2.5 so a folded name hit stays under the exact token score of 3. That clears TOOL_SEARCH_SCORE_RATIO (0.75) for a two-word query whose top is 6, which is what the comment next to the constant describes. It does not clear the same cut for a one-word plural query. name.includes(rawQuery) adds 1 on top of the exact token score, so list_issues scores 4 for "issues" and the floor is 3. get_issue and save_issue only fold, so they score 2.5 and createToolIndex(...).search("issues", 20) returns just ["mcp__linear__list_issues"]. The other direction still works: search("issue", 20) keeps list_issues at 3.5. No weight strictly under 3 can clear a floor of 3, and 3 would tie search("issue") (both names also take the substring point), which drops "exact spelling outranks folded." One way out is to stop the one-token raw-substring bonus from setting that floor — a one-word query is already paid for by the token score — so the top stays 3, 2.5 still clears 0.75 * 3, and the exact token still ranks first. The new tests never call search("issues").

Checked with bun -e against a Linear-shaped catalog (list_issues, get_issue, save_issue) in the PR worktree: scores 4 / 2.5 / 2.5 for "issues", and only list_issues returned. bun test src/agent/lexical-rank.test.ts src/agent/tool-search.test.ts was 99 pass, 0 fail; those tests do not cover this query.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants