diff --git a/ts/docs/plans/conversation-search/CONVERSATION-SEARCH.md b/ts/docs/plans/conversation-search/CONVERSATION-SEARCH.md new file mode 100644 index 0000000000..da69028c14 --- /dev/null +++ b/ts/docs/plans/conversation-search/CONVERSATION-SEARCH.md @@ -0,0 +1,209 @@ +# Conversation Search — Design & Plan + +Status: **in progress** — slice 1 complete (fuzzy name find, all clients); slice +2 landed (unified tagged index scaffolding + wiring + unit tests); slice 3 +landed (population: live tee + Copilot batch-append). Next: slice 4 (content +search surface). Last updated: 2026-07-30. + +## Goal + +Let a user find conversations two ways: + +1. **Fuzzy lookup by name** — "switch to the conversation named _blah_", where + _blah_ is imprecise (typo'd, partial, or semantically close: "gym music" → + "workout playlist setup"). +2. **Content search** — "find the conversation where we were talking about / + doing X", answered from conversation memory (knowPro), returning _which_ + conversation(s) discussed X. + +Scope for now: **connected / agent-server mode only.** Standalone Shell is +deferred (decision #4). + +## Current architecture (verified) + +- UI-level conversations are owned by the agent-server + [`ConversationManager`](../../../packages/agentServer/server/src/conversationManager.ts). + Registry = `instanceDir/conversations/conversations.json` + (`{ conversationId (UUIDv4), name, createdAt, source?, readOnly?, copilot? }`). + Each conversation has its own `persistDir = conversations//` + and its own lazily-created `SharedDispatcher`, evicted after 5 min idle. +- `listConversations(name?)` today is a **case-insensitive substring** filter + ([conversationManager.ts](../../../packages/agentServer/server/src/conversationManager.ts)). +- The dispatcher builds **one `conversationMemory` per session dir** in + [`initializeMemory()`](../../../packages/dispatcher/dispatcher/src/context/memory.ts) + (`baseFileName: "conversationMemory"`). So in connected mode there is already + one knowPro index per conversation under its `persistDir` — but it is **siloed + and lazy** (only present while that conversation's dispatcher is live), so + nothing can search across conversations. +- Live turns are queued to memory via `addRequestToMemory` / + `addResultToMemory` ([memory.ts](../../../packages/dispatcher/dispatcher/src/context/memory.ts)). + This is the tee point for a unified index. + +### Key constraint: knowPro is append-only + +`ICollection` / `IMessageCollection` +([interfaces.ts](../../../packages/knowPro/src/interfaces.ts)) are **append-only** +(`append()`, no per-message remove). Message ordinals are positional and semantic +refs point at them, so **you cannot remove one conversation's messages from a +shared index without rebuilding it**. This shapes the deletion story below. + +Tag scoping IS supported: `createTagSearchTermGroup(tags)` → +`PropertyNames.Tag` ([searchLib.ts](../../../packages/knowPro/src/searchLib.ts)), +`ConversationMessage` accepts `tags?: kp.MessageTag[]`, and +`MemorySettings.useScopedSearch` (experimental) uses structured tags. + +## Design decisions + +| # | Decision | Rationale | +| --- | ----------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- | +| 1 | Fuzzy name = **hybrid lexical + embedding** | Lexical nails exact/typo; embeddings catch semantic. Embedding pattern already exists in repo. | +| 2 | Content search = **one unified knowPro index across all conversations**, messages tagged by `conversationId` | Cross-conversation search without spinning up N dispatchers. Accept double-indexing (per-conversation **and** unified); storage is cheap. | +| 3 | **Split** the surface: `@conversation find ` (fuzzy) + `@conversation search ` (content), separate actions | Distinct intents; can unify later. | +| 4 | **Connected mode only** for now | Standalone unification is a bigger, separate effort. | +| 5 | Per-conversation index = **source of truth**; unified index = **derived/rebuildable** | Append-only means hard delete must drop the per-conversation dir; unified index is compacted by rebuild. | +| 6 | Unified-index compaction (reclaim tombstones) triggered **on idle-conversation timeout** + a startup safety check | The idle-eviction moment is contention-free; startup covers the "never idle again" case. | + +## Target architecture + +```mermaid +flowchart TD + subgraph CM[ConversationManager - agent-server] + REG[conversations.json registry] --> NIDX[ConversationNameIndex
embeddings by conversationId] + NIDX --> FIND["@conversation find (fuzzy name)"] + UNI[(Unified knowPro index
tagged by conversationId)] --> SRCH["@conversation search (content)"] + end + subgraph PC[per-conversation dispatcher] + PMEM[(per-conv knowPro index
source of truth)] --> MEMQ["@memory Q&A"] + end + TURN[live turn] --> PMEM + TURN -. tee w/ conversationId .-> UNI + IMPORT["@copilot import (many convos)"] --> PMEM + IMPORT -. batch append tagged .-> UNI +``` + +### Search #1 — fuzzy name + +Reuse the established name-index pattern from +[tabTitleIndex.mts](../../../packages/agents/browser/src/agent/tabTitleIndex.mts) +and [programNameIndex.ts](../../../packages/agents/desktop/src/programNameIndex.ts): + +- `isEmbeddingAvailable()` + `openai.createEmbeddingModel(openai.apiSettingsFromEnv(openai.ModelType.Embedding))`. +- Cache `Record`; embed on create, re-embed + on rename, drop on delete, rebuild from `conversations.json` at startup. +- `generateEmbedding(model, query)` + `indexesOfNearest(embeddings, q, k, SimilarityType.Dot)`. +- **Hybrid**: lexical (substring + edit-distance) results first, then embedding + nearest neighbors merged in. No embedding provider ⇒ lexical-only (graceful). + +### Search #2 — content (unified index) + +- A single `ConversationMemory` instance owned by `ConversationManager` at an + instance-level path (e.g. `conversations/_unified/`), loaded once and kept warm. +- Every message appended with a `conversationId` structured tag (+ store enough + to resolve display name at query time from the registry; never bake stale + names into the index). +- Cross-conversation search runs unscoped, groups/ranks hits by `conversationId`. +- **Delete** ⇒ append-only, so **tombstone** the `conversationId` (filter out at + query time) + **compact** (rebuild from surviving per-conversation indexes) on + idle timeout / startup threshold. +- **Rename** ⇒ no index change. +- **Population**: `ConversationManager` injects a unified-memory sink + + `conversationId` into each per-conversation dispatcher so + `addRequestToMemory` / `addResultToMemory` tee to the unified index; standalone + leaves the sink undefined (no-op). `@copilot import` batch-appends imported + turns (tagged), incremental via the existing `lastSyncedTurnIndex` watermark. + +## Surface (split) + +- `@conversation find ` → `findConversation` action → new RPC + `findConversations(query, maxMatches?)` returning scored `ConversationInfo[]` + (new method; leaves `listConversations` substring semantics untouched). +- `@conversation search ` → `searchConversation` action → queries the + unified index, returns matching conversations + snippets. +- CLI: `agent-cli conversations find ` and `... search `. +- `.agr` grammar patterns for both ("which conversation was about X", etc.). + +## Slices + +- **Slice 1 — fuzzy name (`@conversation find`).** `ConversationNameIndex` in + `ConversationManager`; `findConversations` RPC (protocol + client + server); + `@conversation find` command + `findConversation` action + `.agr`; CLI + `conversations find`. Self-contained, no memory-model changes. + - **Done:** `conversationNameIndex.ts` (hybrid lexical + embedding, unit + tested); `ConversationManager.findConversations` + create/import/rename/ + delete/seed lifecycle wiring; `findConversations` RPC across protocol, + client, server handler (+ browser-extension adapter and client test stub); + CLI `conversations find`. All packages build; 8/8 unit tests pass. + - **Remaining (slice 1b):** in-chat surface — `@conversation find` command + + `findConversation` action + `.agr` grammar, and client rendering of the + result (the `manage-conversation` payload path, per client). Open question: + whether "switch to the conversation about X" should find-then-switch. + - **Slice 1b: DONE.** Extended the shared client `manage.ts` with a `find` + subcommand + `matches` result kind + `manageFind`, and made `manageSwitch` + fall back to a fuzzy find-then-switch (score >= 0.6) when there is no exact + name. Rendered `matches` in the shared `render.ts` (Shell + VS Code shell) + and the CLI + browser-extension renderers. Dispatcher: `findConversation` + action + `@conversation find` command + `.agr` grammar (plus an "about X" + switch phrasing). Tests: manageFind, fuzzy switch, matches render, and + findConversation grammar (all passing). Note: "switch ... about X" = + find-then-switch, `@conversation find` = list ranked matches. +- **Slice 2 — unified index scaffolding.** Create/open the + `ConversationManager`-owned unified `ConversationMemory`; tagged `append`; + tombstone-on-delete; compaction hook on idle timeout + startup. + - **DONE:** `conversationSearchIndex.ts` — one `ConversationMemory` at + `conversations/_unified/`, each message tagged `conv:` + (plain-string knowPro `MessageTag`); `search()` runs `searchWithLanguage`, + reads the tag back off each matched message, and groups/ranks by + conversation via the pure, unit-tested `rankConversationMatches`. Index is + **inert** (no-op / empty) when no model provider is configured, so it wires + unconditionally. Population uses `extractKnowledge: true` by default + (content search resolves through extracted knowledge, so production must + extract; configurable via `createConversationSearchIndex(dir, { +extractKnowledge })`). Wired into `ConversationManager`: + created at startup, **tombstoned on delete**, closed on shutdown, and + exposed via `indexConversationMessage()` + `searchConversationContent()` + (the seams slices 3/4 call). Unit tests: `conversationSearchIndex.spec.ts` + (grouping, tombstone filtering, untagged skip, snippet/conversation caps) — + pure `rankConversationMatches`, no LLM. + - **Deferred:** compaction/rebuild (needs population sources — lands with + slice 3); reliable live end-to-end test. A live test was prototyped and + removed: `searchWithLanguage` makes an LLM query-translation call per + search, so any live search test is LLM-latency-bound and flaky. + Empirically, extraction-off search returns no hits (content search needs + extracted knowledge) and extraction-on ingest+search exceeded 5 min in this + env. Coverage stays on the offline unit tests; revisit extraction-on + latency when slice 3 wires real population. +- **Slice 3 — population.** Tee live turns (inject sink + conversationId into the + per-conversation dispatcher); batch-append on `@copilot import` (incremental). + - **DONE:** Added a host-injected `ConversationContentSink` to the dispatcher + (`commandHandlerContext.ts` type + `DispatcherOptions` + `CommandHandlerContext`, + threaded in `initializeCommandHandlerContext`). `memory.ts` calls it on every + user turn (`addUserMessageToHistory`) and assistant turn (`addResultToMemory`), + **ungated by the knowledge-extraction flags** — which matters because connected + mode sets `requestKnowledgeExtraction: false` / `actionResultKnowledgeExtraction: +false`, so the per-conversation memory tee is off there but the unified index + still populates. `ConversationManager.ensureDispatcher` injects the sink, + closing over the conversation id, so each dispatcher's turns land in the unified + index tagged `conv:`. Read-only Copilot mirrors never replay through the + live tee, so `mirrorImporter` batch-calls `indexConversationMessage` for each + imported turn's user/assistant text, gated on `created` (no double-index on + re-import). Test: `copilotImport.spec.ts` asserts imported turns are indexed in + order with null messages skipped. + - **Deferred:** a dispatcher-level unit test of the live tee — importing + `memory.ts` in isolation trips a pre-existing `memory.ts`↔`systemAgent.ts` + circular-init TDZ (`systemAgent` calls `getMemoryCommandHandlers()` at module + top level). The tee call sites are one-liners; coverage rests on the Copilot + test (seam), the full build (type threading), and the slice-2 rank tests. +- **Slice 4 — content search surface.** `@conversation search` command + + `searchConversation` action + `.agr`; CLI; group/rank by conversation. +- **Later** — unify the two commands if desired; standalone Shell support. + +## Open items / risks + +- Compaction cost when many conversations exist (`@copilot import`): rebuild is + O(all messages). Mitigate with threshold trigger; consider a per-conversation + summary index for routing if fan-out/rebuild ever hurts. +- Name resolution for tombstoned/renamed conversations in unified results must + come from the live registry, not the index. +- Privacy: tombstoned (deleted) content physically remains in the unified index + until compaction — ensure it is filtered from both search results and answer + generation immediately on delete. diff --git a/ts/packages/actionGrammar/src/grammarSourceMap.ts b/ts/packages/actionGrammar/src/grammarSourceMap.ts new file mode 100644 index 0000000000..024a869bc8 --- /dev/null +++ b/ts/packages/actionGrammar/src/grammarSourceMap.ts @@ -0,0 +1,253 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +// Source-map style side-car for compiled grammars. Like a `.js.map`, it is +// emitted next to `.ag.json` (as `.ag.map.json`) and lets a host +// recover the original `.agr` source text of the rule that matched a request - +// without shipping/re-parsing the `.agr` source (the runtime only has the +// compiled grammar). The compiler already collects everything needed in a +// `DebugInfoCollector`; this module serializes it and reads it back. + +import { DebugInfoCollector } from "./grammarCompiler.js"; +import { Grammar } from "./grammarTypes.js"; +import { matchGrammar } from "./grammarMatcher.js"; +import { TraceEvent } from "./traceEvents.js"; + +export type GrammarRuleRange = { + fileId: string; + start: number; + end: number; +}; + +export type GrammarSourceMap = { + version: 1; + // fileId (displayPath) -> original `.agr` source text. + files: Record; + // ruleId -> its source span [start, end) in the owning file. + rules: Record; + // partId -> owning ruleId, for parts that carry a source position (skips + // synthetic/spanless parts introduced by optimization). + parts: Record; +}; + +// One phrase word/run and the category it maps to (the `` a captured +// value references, e.g. "TrackPhrase"); empty category = literal keyword. +export type MatchedPhraseSegment = { text: string; category: string }; + +export type MatchedGrammarRule = { + // The matched rule's `.agr` source text. + text: string; + // The request broken into colored segments matching the rule's markers. + segments: MatchedPhraseSegment[]; +}; + +/** + * Build the side-car payload from the debug info the compiler collected. The + * collector records each rule's start offset only, so a rule's end is taken as + * the next rule's start in the same file (trailing whitespace is trimmed when + * the text is later sliced). + */ +export function buildGrammarSourceMap( + collector: DebugInfoCollector, +): GrammarSourceMap { + const files: Record = {}; + for (const [fileId, text] of collector.fileContents) { + files[fileId] = text; + } + + const startsByFile = new Map< + string, + { ruleId: string; offset: number }[] + >(); + for (const [ruleId, pos] of collector.rulePositions) { + const arr = startsByFile.get(pos.fileId) ?? []; + arr.push({ ruleId, offset: pos.offset }); + startsByFile.set(pos.fileId, arr); + } + + const rules: Record = {}; + for (const [fileId, arr] of startsByFile) { + arr.sort((a, b) => a.offset - b.offset); + const fileLength = files[fileId]?.length ?? 0; + for (let i = 0; i < arr.length; i++) { + const end = i + 1 < arr.length ? arr[i + 1].offset : fileLength; + rules[arr[i].ruleId] = { fileId, start: arr[i].offset, end }; + } + } + + const parts: Record = {}; + for (const [partId, ruleId] of collector.partRules) { + if (collector.partPositions.has(partId)) { + parts[partId] = ruleId; + } + } + + return { version: 1, files, rules, parts }; +} + +/** + * Recover the matched rule's `.agr` source text plus the request broken into + * per-category colored segments (so the phrase can be colored to match the + * rule's markers). Runs the matcher once with a trace hook: the winning rule is + * found by walking back the matched parts (preferring the rule that emits + * `actionName`), and phrase segments are built from the captured values, each + * colored by the `` its capture references. Returns undefined when the + * request doesn't match or the rule can't be pinpointed. + */ +export function findMatchedRule( + sourceMap: GrammarSourceMap, + grammar: Grammar, + request: string, + actionName?: string, +): MatchedGrammarRule | undefined { + const events: TraceEvent[] = []; + const results = matchGrammar(grammar, request, { + trace: (event) => events.push(event), + }); + if (results.length === 0) { + return undefined; + } + const text = recoverRuleText(events, sourceMap, actionName); + if (text === undefined) { + return undefined; + } + return { + text, + segments: buildPhraseSegments(results[0]?.match, request, text), + }; +} + +function recoverRuleText( + events: TraceEvent[], + sourceMap: GrammarSourceMap, + actionName: string | undefined, +): string | undefined { + let fallback: string | undefined; + for (let i = events.length - 1; i >= 0; i--) { + const event = events[i]; + if (event.kind !== "partMatched") { + continue; + } + const ruleId = sourceMap.parts[event.part]; + if (ruleId === undefined) { + continue; + } + const range = sourceMap.rules[ruleId]; + const source = range && sourceMap.files[range.fileId]; + if (source === undefined) { + continue; + } + const text = source.slice(range.start, range.end).trimEnd(); + if (actionName === undefined || emitsAction(text, actionName)) { + return text; + } + // Remember the last matched part's rule in case no rule on the parse + // explicitly emits the action (e.g. dispatched via a sub-rule). + fallback ??= text; + } + return fallback; +} + +function emitsAction(ruleText: string, actionName: string): boolean { + return new RegExp(`actionName:\\s*"${escapeRegExp(actionName)}"`).test( + ruleText, + ); +} + +// Collect a matched action value's leaf string values with the parameter name +// each sits under, e.g. { target: { trackName: "x", artists: ["y"] } } -> +// [{ name: "trackName", value: "x" }, { name: "artists", value: "y" }]. +function collectLeafValues( + value: unknown, + name: string, + leaves: { name: string; value: string }[], +) { + if (typeof value === "string") { + leaves.push({ name, value }); + } else if (Array.isArray(value)) { + for (const entry of value) collectLeafValues(entry, name, leaves); + } else if (value !== null && typeof value === "object") { + for (const [key, val] of Object.entries(value)) { + collectLeafValues(val, key, leaves); + } + } +} + +// Break `request` into colored segments: each captured value that references a +// `` in the matched rule becomes a segment with that category; the rest +// are literal keywords. The capture variables come from the rule's `$(var:)` +// markers and are linked to the matched action's leaf values by name (tolerating +// simple singular/plural, e.g. capture `artist` -> action param `artists`). +function buildPhraseSegments( + matchValue: unknown, + request: string, + ruleText: string, +): MatchedPhraseSegment[] { + const captureRe = /\$\(\s*(\w+)\s*:\s*<([^>]+)>/g; + const varCategory = new Map(); + let capture: RegExpExecArray | null; + while ((capture = captureRe.exec(ruleText)) !== null) { + if (!varCategory.has(capture[1])) { + varCategory.set(capture[1], capture[2]); + } + } + if (varCategory.size === 0) { + return []; + } + + const leaves: { name: string; value: string }[] = []; + collectLeafValues(matchValue, "", leaves); + + const lower = request.toLowerCase(); + const spans: { start: number; end: number; category: string }[] = []; + const usedValues = new Set(); + for (const [variable, category] of varCategory) { + const leaf = leaves.find( + (candidate) => + !usedValues.has(candidate.value.toLowerCase()) && + (candidate.name === variable || + candidate.name === `${variable}s` || + `${candidate.name}s` === variable), + ); + if (leaf === undefined) { + continue; + } + const key = leaf.value.toLowerCase(); + const start = lower.indexOf(key); + if (start === -1) { + continue; + } + usedValues.add(key); + spans.push({ start, end: start + leaf.value.length, category }); + } + if (spans.length === 0) { + return []; + } + spans.sort((a, b) => a.start - b.start); + + const segments: MatchedPhraseSegment[] = []; + const pushLiteral = (text: string) => { + const trimmed = text.trim(); + if (trimmed.length > 0) { + segments.push({ text: trimmed, category: "" }); + } + }; + let pos = 0; + for (const span of spans) { + if (span.start < pos) { + continue; // overlapping capture; keep the earlier one + } + pushLiteral(request.slice(pos, span.start)); + segments.push({ + text: request.slice(span.start, span.end), + category: span.category, + }); + pos = span.end; + } + pushLiteral(request.slice(pos)); + return segments; +} + +function escapeRegExp(text: string): string { + return text.replace(/[.*+?^${}()|[\]\\]/g, "\\$&"); +} diff --git a/ts/packages/actionGrammar/src/index.ts b/ts/packages/actionGrammar/src/index.ts index 87a7a15192..df6180cfd1 100644 --- a/ts/packages/actionGrammar/src/index.ts +++ b/ts/packages/actionGrammar/src/index.ts @@ -25,6 +25,13 @@ export type { FileLoader, } from "./grammarCompiler.js"; export { defaultFileLoader } from "./defaultFileLoader.js"; +export { buildGrammarSourceMap, findMatchedRule } from "./grammarSourceMap.js"; +export type { + GrammarSourceMap, + GrammarRuleRange, + MatchedGrammarRule, + MatchedPhraseSegment, +} from "./grammarSourceMap.js"; // Parser (for tooling — formatter, linters, etc.) export { parseGrammarRules } from "./grammarRuleParser.js"; diff --git a/ts/packages/actionGrammar/test/grammarSourceMap.spec.ts b/ts/packages/actionGrammar/test/grammarSourceMap.spec.ts new file mode 100644 index 0000000000..1b66656dcd --- /dev/null +++ b/ts/packages/actionGrammar/test/grammarSourceMap.spec.ts @@ -0,0 +1,98 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +import { loadGrammarRulesNoThrow } from "../src/grammarLoader.js"; +import { + buildGrammarSourceMap, + findMatchedRule, +} from "../src/grammarSourceMap.js"; +import type { DebugInfoCollector } from "../src/grammarCompiler.js"; + +function compileWithSourceMap(source: string) { + const collector: DebugInfoCollector = { + partPositions: new Map(), + rulePositions: new Map(), + partRules: new Map(), + partLabels: new Map(), + fileContents: new Map(), + filePaths: new Map(), + }; + const errors: string[] = []; + const grammar = loadGrammarRulesNoThrow("test.agr", source, errors, [], { + debugCollector: collector, + }); + if (grammar === undefined) { + throw new Error(errors.join("\n")); + } + return { grammar, sourceMap: buildGrammarSourceMap(collector) }; +} + +describe("grammar source map", () => { + it("recovers the exact source text of the matched rule", () => { + const source = ` = | ; + = pause -> { actionName: "pause" }; + = play $(track:string) -> { actionName: "play", parameters: { track } };`; + const { grammar, sourceMap } = compileWithSourceMap(source); + + const rule = findMatchedRule( + sourceMap, + grammar, + "play yesterday", + "play", + )?.text; + expect(rule).toContain(""); + expect(rule).toContain('actionName: "play"'); + expect(rule).not.toContain(""); + }); + + it("uses the action to skip a matched helper sub-rule", () => { + const source = ` = ; + = play $(n:number) -> { actionName: "playNum", parameters: { n } }; + = track | song;`; + const { grammar, sourceMap } = compileWithSourceMap(source); + + // The last matched token ("track") belongs to , but the action + // cross-check must return the emitting rule . + const rule = findMatchedRule( + sourceMap, + grammar, + "play track 3", + "playNum", + )?.text; + expect(rule).toContain(""); + expect(rule).toContain('actionName: "playNum"'); + expect(rule?.startsWith("")).toBe(false); + }); + + it("returns undefined when the request does not match", () => { + const source = ` = ; + = pause -> { actionName: "pause" };`; + const { grammar, sourceMap } = compileWithSourceMap(source); + + expect( + findMatchedRule(sourceMap, grammar, "fly to the moon", "pause") + ?.text, + ).toBeUndefined(); + }); + + it("colors the request phrase by the category of each capture", () => { + const source = ` = ; + = play $(track:) by $(artist:) -> { actionName: "play", parameters: { track, artist } }; + = $(x:wildcard); + = $(x:wildcard);`; + const { grammar, sourceMap } = compileWithSourceMap(source); + + const segments = findMatchedRule( + sourceMap, + grammar, + "play shake it off by taylor swift", + "play", + )?.segments; + expect(segments).toEqual([ + { text: "play", category: "" }, + { text: "shake it off", category: "TrackPhrase" }, + { text: "by", category: "" }, + { text: "taylor swift", category: "ArtistName" }, + ]); + }); +}); diff --git a/ts/packages/actionGrammarCompiler/src/commands/compile.ts b/ts/packages/actionGrammarCompiler/src/commands/compile.ts index 79274e2e21..c5dbfd2b6b 100644 --- a/ts/packages/actionGrammarCompiler/src/commands/compile.ts +++ b/ts/packages/actionGrammarCompiler/src/commands/compile.ts @@ -8,7 +8,9 @@ import { grammarToJson, loadGrammarRulesNoThrow, recommendedOptimizations, + buildGrammarSourceMap, SchemaLoader, + DebugInfoCollector, } from "@typeagent/action-grammar"; import { parseSchemaSource } from "@typeagent/action-schema"; import type { SchemaTypeDefinition } from "@typeagent/action-schema"; @@ -76,16 +78,27 @@ export default class Compile extends Command { const errors: string[] = []; const warnings: string[] = []; const schemaLoader = createFileSchemaLoader(); + // Collect source positions + text so we can emit a source-map side-car + // (partIds survive optimization, so they align with the emitted .ag.json). + const debugCollector: DebugInfoCollector = { + partPositions: new Map(), + rulePositions: new Map(), + partRules: new Map(), + partLabels: new Map(), + fileContents: new Map(), + filePaths: new Map(), + }; const grammar = loadGrammarRulesNoThrow( flags.input, undefined, errors, warnings, flags.debug - ? { startValueRequired: true, schemaLoader } + ? { startValueRequired: true, schemaLoader, debugCollector } : { startValueRequired: true, schemaLoader, + debugCollector, optimizations: recommendedOptimizations, }, ); @@ -108,5 +121,16 @@ export default class Compile extends Command { } fs.writeFileSync(flags.output, JSON.stringify(grammarToJson(grammar))); console.log(`Action grammar written: ${flags.output}`); + + // Emit the source-map side-car next to the compiled grammar. The + // compiled .ag.json is left untouched. + const mapOutput = flags.output.endsWith(".ag.json") + ? `${flags.output.slice(0, -".ag.json".length)}.ag.map.json` + : `${flags.output}.map.json`; + fs.writeFileSync( + mapOutput, + JSON.stringify(buildGrammarSourceMap(debugCollector)), + ); + this.log(`Grammar source map written: ${mapOutput}`); } } diff --git a/ts/packages/agentSdk/src/agentInterface.ts b/ts/packages/agentSdk/src/agentInterface.ts index 6047c2530c..051a72922d 100644 --- a/ts/packages/agentSdk/src/agentInterface.ts +++ b/ts/packages/agentSdk/src/agentInterface.ts @@ -65,7 +65,14 @@ export type SchemaContent = { content: string; config?: string | undefined; // for "ts" only }; -export type GrammarContent = { format: GrammarFormat; content: string }; +export type GrammarContent = { + format: GrammarFormat; + content: string; + // JSON text of the compiled grammar's source-map side-car (.ag.map.json), + // when present next to an "ag" grammar file. Lets a host recover matched-rule + // source text. + sourceMap?: string | undefined; +}; export type SchemaManifest = { description: string; diff --git a/ts/packages/agentServer/client/src/agentServerClient.ts b/ts/packages/agentServer/client/src/agentServerClient.ts index d4d7b36cd8..cb875e1ada 100644 --- a/ts/packages/agentServer/client/src/agentServerClient.ts +++ b/ts/packages/agentServer/client/src/agentServerClient.ts @@ -28,6 +28,7 @@ import { DispatcherConnectOptions, CreateConversationOptions, ConversationInfo, + ConversationMatch, JoinConversationResult, RenameConversationOptions, SpeechToken, @@ -115,6 +116,14 @@ export type AgentServerConnection = { options?: CreateConversationOptions, ): Promise; listConversations(name?: string): Promise; + /** + * Fuzzy-find conversations by name (lexical + embedding). Results are + * sorted by descending relevance score. + */ + findConversations( + query: string, + maxMatches?: number, + ): Promise; renameConversation( conversationId: string, newName: string, @@ -340,6 +349,13 @@ export function createAgentServerConnection( return rpc.invoke("listConversations", name); }, + async findConversations( + query: string, + maxMatches?: number, + ): Promise { + return rpc.invoke("findConversations", query, maxMatches); + }, + async renameConversation( conversationId: string, newName: string, diff --git a/ts/packages/agentServer/client/src/conversation/index.ts b/ts/packages/agentServer/client/src/conversation/index.ts index 7bec6c973a..68ae97941d 100644 --- a/ts/packages/agentServer/client/src/conversation/index.ts +++ b/ts/packages/agentServer/client/src/conversation/index.ts @@ -46,9 +46,11 @@ export { manageCycle, manageRename, manageDelete, + manageFind, type ManageConversationPayload, type ManageConversationContext, type ConversationActionResult, } from "./manage.js"; +export type { ConversationMatch } from "../index.js"; export { renderConversationActionResult } from "./render.js"; diff --git a/ts/packages/agentServer/client/src/conversation/manage.ts b/ts/packages/agentServer/client/src/conversation/manage.ts index 4efe629fa3..d2bc6c5a97 100644 --- a/ts/packages/agentServer/client/src/conversation/manage.ts +++ b/ts/packages/agentServer/client/src/conversation/manage.ts @@ -12,6 +12,7 @@ import type { AgentServerConnection, ConversationDispatcher, ConversationInfo, + ConversationMatch, } from "../index.js"; import { findConversationByName, @@ -38,9 +39,14 @@ export type ManageConversationPayload = { | "prev" | "next" | "rename" - | "delete"; + | "delete" + | "find"; name?: string; newName?: string; + /** Search term for the `find` subcommand. */ + query?: string; + /** Optional cap on `find` results. */ + maxMatches?: number; }; export type ManageConversationContext = { @@ -132,6 +138,12 @@ export type ConversationActionResult = conversations: ConversationInfo[]; currentConversationId: string | undefined; } + | { + kind: "matches"; + query: string; + matches: ConversationMatch[]; + currentConversationId: string | undefined; + } | { kind: "info"; conversationId: string; name: string } | { kind: "cancelled"; target: ConversationInfo }; @@ -153,6 +165,10 @@ function error(message: string, cause?: unknown): ConversationActionResult { return { kind: "error", message, cause }; } +// Minimum fuzzy score for `switch ` to accept a non-exact match and +// switch to it (find-then-switch). Below this, switch reports "not found". +const SWITCH_FUZZY_MIN_SCORE = 0.6; + async function performSwitch( connection: AgentServerConnection, clientIO: ClientIO, @@ -301,6 +317,32 @@ export async function manageList( }; } +/** + * `find` — fuzzy-find conversations by name (lexical + embedding). Returns a + * ranked `matches` result for the caller to render. + */ +export async function manageFind( + connection: AgentServerConnection, + ctx: ManageConversationContext, + query: string | undefined, + maxMatches?: number, +): Promise { + const trimmed = query?.trim(); + if (!trimmed) { + return warning("A search term is required to find conversations."); + } + const matches = await connection.findConversations(trimmed, maxMatches); + if (matches.length === 0) { + return warning(`No conversations matching "${trimmed}" found.`); + } + return { + kind: "matches", + query: trimmed, + matches, + currentConversationId: ctx.currentConversationId, + }; +} + /** `info` — show the current conversation id + name. */ export function manageInfo( ctx: ManageConversationContext, @@ -330,7 +372,17 @@ export async function manageSwitch( return warning("A conversation name is required to switch."); } const all = await connection.listConversations(); - const match = findConversationByName(all, trimmed); + let match = findConversationByName(all, trimmed); + let fuzzy = false; + if (match === undefined) { + // No exact name: fall back to a fuzzy find-then-switch on the best + // match, as long as it clears a confidence floor. + const found = await connection.findConversations(trimmed, 1); + if (found.length > 0 && found[0].score >= SWITCH_FUZZY_MIN_SCORE) { + match = found[0].conversation; + fuzzy = true; + } + } if (match === undefined) { return warning(`No conversation named "${trimmed}" found.`); } @@ -342,7 +394,9 @@ export async function manageSwitch( clientIO, ctx, match, - `Switched to conversation "${match.name}".`, + fuzzy + ? `Switched to conversation "${match.name}" (closest match for "${trimmed}").` + : `Switched to conversation "${match.name}".`, ); } @@ -582,6 +636,13 @@ export async function manageConversation( return await manageNew(connection, clientIO, ctx, payload.name); case "list": return await manageList(connection, ctx, payload.name); + case "find": + return await manageFind( + connection, + ctx, + payload.query, + payload.maxMatches, + ); case "info": return manageInfo(ctx); case "switch": diff --git a/ts/packages/agentServer/client/src/conversation/render.ts b/ts/packages/agentServer/client/src/conversation/render.ts index 651061e107..4408ca9949 100644 --- a/ts/packages/agentServer/client/src/conversation/render.ts +++ b/ts/packages/agentServer/client/src/conversation/render.ts @@ -80,5 +80,29 @@ export function renderConversationActionResult( ]; return createStructuredContent(blocks); } + case "matches": { + const items = result.matches.map((m) => { + const c = m.conversation; + const isCurrent = + c.conversationId === result.currentConversationId; + const pct = `${Math.round(m.score * 100)}% match`; + const messages = `${c.messageCount} message${ + c.messageCount === 1 ? "" : "s" + }`; + return { + text: isCurrent ? `${c.name} (current)` : c.name, + subtitle: `${pct} · ${messages}`, + }; + }); + const blocks: StructuredBlock[] = [ + { + kind: "heading", + level: 3, + text: `Matches for “${result.query}” (${result.matches.length})`, + }, + { kind: "list", items }, + ]; + return createStructuredContent(blocks); + } } } diff --git a/ts/packages/agentServer/client/src/index.ts b/ts/packages/agentServer/client/src/index.ts index 2b1f46f1ed..6d729ff245 100644 --- a/ts/packages/agentServer/client/src/index.ts +++ b/ts/packages/agentServer/client/src/index.ts @@ -23,6 +23,7 @@ export type { export type * from "@typeagent/dispatcher-rpc/types"; export type { ConversationInfo, + ConversationMatch, JoinConversationResult, DispatcherConnectOptions, SpeechToken, diff --git a/ts/packages/agentServer/client/test/conversation-manage.spec.ts b/ts/packages/agentServer/client/test/conversation-manage.spec.ts index 00a84e8620..c14211b79f 100644 --- a/ts/packages/agentServer/client/test/conversation-manage.spec.ts +++ b/ts/packages/agentServer/client/test/conversation-manage.spec.ts @@ -6,6 +6,7 @@ import { manageConversation, manageCycle, manageDelete, + manageFind, manageInfo, manageList, manageNew, @@ -184,6 +185,58 @@ describe("manageSwitch", () => { }); }); +describe("manageSwitch — fuzzy fallback", () => { + test("switches to the top fuzzy match when there is no exact name", async () => { + const conn = makeStubConnection({ + list: [ + makeInfo("a", "Workout Playlist"), + makeInfo("b", "Groceries"), + ], + }); + // "workout" is not an exact name, but the stub's findConversations + // returns it as a substring match, so switch should adopt it. + const result = await manageSwitch( + conn, + fakeClientIO, + ctx({ currentConversationId: "b" }), + "workout", + ); + expect(result.kind).toBe("ok"); + if (result.kind === "ok") { + expect(result.switched).toBe(true); + expect(result.conversation?.conversationId).toBe("a"); + expect(result.message).toMatch(/closest match/); + } + }); +}); + +describe("manageFind", () => { + test("returns ranked matches", async () => { + const conn = makeStubConnection({ + list: [ + makeInfo("a", "Workout Playlist"), + makeInfo("b", "Groceries"), + ], + }); + const result = await manageFind(conn, ctx(), "workout"); + expect(result.kind).toBe("matches"); + if (result.kind === "matches") { + expect(result.query).toBe("workout"); + expect(result.matches[0].conversation.conversationId).toBe("a"); + } + }); + test("warns when nothing matches", async () => { + const conn = makeStubConnection({ list: [makeInfo("a", "Workout")] }); + const result = await manageFind(conn, ctx(), "zzz-nonexistent"); + expect(result.kind).toBe("warning"); + }); + test("warns when the query is blank", async () => { + const conn = makeStubConnection({ list: [makeInfo("a", "A")] }); + const result = await manageFind(conn, ctx(), " "); + expect(result.kind).toBe("warning"); + }); +}); + describe("manageCycle", () => { test("next wraps around", async () => { const conn = makeStubConnection({ diff --git a/ts/packages/agentServer/client/test/conversation-render.spec.ts b/ts/packages/agentServer/client/test/conversation-render.spec.ts index ad9cbbb93a..5e7cadc6f6 100644 --- a/ts/packages/agentServer/client/test/conversation-render.spec.ts +++ b/ts/packages/agentServer/client/test/conversation-render.spec.ts @@ -49,3 +49,41 @@ describe("renderConversationActionResult — list", () => { expect(json).toContain("No conversations found."); }); }); + +describe("renderConversationActionResult — matches", () => { + test("shows query, name, and percent match", () => { + const json = render({ + kind: "matches", + query: "gym music", + currentConversationId: "id2", + matches: [ + { + conversation: { + conversationId: "id1", + name: "Workout Playlist", + clientCount: 0, + createdAt: "2026-01-01T00:00:00Z", + messageCount: 3, + }, + score: 0.92, + }, + { + conversation: { + conversationId: "id2", + name: "Beta", + clientCount: 0, + createdAt: "2026-01-02T00:00:00Z", + messageCount: 1, + }, + score: 0.71, + }, + ], + }); + expect(json).toContain("Matches for"); + expect(json).toContain("gym music"); + expect(json).toContain("Workout Playlist"); + expect(json).toContain("92% match"); + // The current conversation is marked. + expect(json).toContain("Beta (current)"); + }); +}); diff --git a/ts/packages/agentServer/client/test/conversation-stubConnection.ts b/ts/packages/agentServer/client/test/conversation-stubConnection.ts index 43eb11684a..1d3efbaad2 100644 --- a/ts/packages/agentServer/client/test/conversation-stubConnection.ts +++ b/ts/packages/agentServer/client/test/conversation-stubConnection.ts @@ -12,6 +12,7 @@ import type { AgentServerConnection, ConversationDispatcher, ConversationInfo, + ConversationMatch, } from "../src/index.js"; export function makeInfo( @@ -46,6 +47,11 @@ export type StubConnectionOptions = { name: string | undefined, callIndex: number, ) => ConversationInfo[] | Promise | undefined; + findConversations?: ( + query: string, + maxMatches: number | undefined, + callIndex: number, + ) => ConversationMatch[] | Promise | undefined; createConversation?: ( name: string, callIndex: number, @@ -120,6 +126,27 @@ export function makeStubConnection( return state.filter((c) => c.name.toLowerCase().includes(norm)); }, + async findConversations(query: string, maxMatches?: number) { + const idx = nextCount("findConversations"); + calls.push({ + method: "findConversations", + args: [query, maxMatches], + }); + const override = await opts.intercept?.findConversations?.( + query, + maxMatches, + idx, + ); + if (override !== undefined) return override; + const norm = query.trim().toLowerCase(); + const matched = state + .filter((c) => c.name.toLowerCase().includes(norm)) + .map((conversation) => ({ conversation, score: 1 })); + return maxMatches !== undefined + ? matched.slice(0, maxMatches) + : matched; + }, + async createConversation(name: string) { const idx = nextCount("createConversation"); calls.push({ method: "createConversation", args: [name] }); diff --git a/ts/packages/agentServer/protocol/src/index.ts b/ts/packages/agentServer/protocol/src/index.ts index 97a5904ea7..4bcd090cb3 100644 --- a/ts/packages/agentServer/protocol/src/index.ts +++ b/ts/packages/agentServer/protocol/src/index.ts @@ -13,6 +13,7 @@ export { createDiscoveryHandlers, CreateConversationOptions, ConversationInfo, + ConversationMatch, ConversationSource, ConversationNameCollisionOptions, ConversationNameCollisionBehavior, diff --git a/ts/packages/agentServer/protocol/src/protocol.ts b/ts/packages/agentServer/protocol/src/protocol.ts index a3cc46d0ce..e72f20e3a3 100644 --- a/ts/packages/agentServer/protocol/src/protocol.ts +++ b/ts/packages/agentServer/protocol/src/protocol.ts @@ -45,6 +45,15 @@ export type ConversationInfo = { readOnly?: boolean; }; +/** + * A conversation matched by fuzzy name search, with a relevance score in + * [0, 1] (higher is a closer match). + */ +export type ConversationMatch = { + conversation: ConversationInfo; + score: number; +}; + export type ConversationNameCollisionBehavior = "error" | "appendNumber"; export type ConversationNameCollisionOptions = { @@ -110,6 +119,16 @@ export type AgentServerInvokeFunctions = { options?: CreateConversationOptions, ) => Promise; listConversations: (name?: string) => Promise; + /** + * Fuzzy-find conversations by name. Blends lexical (exact / substring / + * edit-distance) with embedding similarity, so imprecise or semantically + * close queries still match. Results are sorted by descending score. Falls + * back to lexical-only matching when no embedding provider is configured. + */ + findConversations: ( + query: string, + maxMatches?: number, + ) => Promise; renameConversation: ( conversationId: string, newName: string, diff --git a/ts/packages/agentServer/server/package.json b/ts/packages/agentServer/server/package.json index 4941f055da..a7b4d2434a 100644 --- a/ts/packages/agentServer/server/package.json +++ b/ts/packages/agentServer/server/package.json @@ -41,11 +41,14 @@ "dependencies": { "@azure/identity": "^4.10.0", "@typeagent/agent-rpc": "workspace:*", + "@typeagent/agent-runtime": "workspace:*", "@typeagent/agent-sdk": "workspace:*", "@typeagent/agent-server-client": "workspace:*", "@typeagent/agent-server-protocol": "workspace:*", + "@typeagent/aiclient": "workspace:*", "@typeagent/common-utils": "workspace:*", "@typeagent/config": "workspace:*", + "@typeagent/conversation-memory": "workspace:*", "@typeagent/dispatcher-rpc": "workspace:*", "@typeagent/dispatcher-types": "workspace:*", "@typeagent/websocket-channel-server": "workspace:*", diff --git a/ts/packages/agentServer/server/src/connectionHandler.ts b/ts/packages/agentServer/server/src/connectionHandler.ts index 040295344c..2a5faa6b4b 100644 --- a/ts/packages/agentServer/server/src/connectionHandler.ts +++ b/ts/packages/agentServer/server/src/connectionHandler.ts @@ -451,6 +451,10 @@ export function createAgentServerConnectionHandler( return conversationManager.listConversations(name); }, + findConversations: async (query: string, maxMatches?: number) => { + return conversationManager.findConversations(query, maxMatches); + }, + renameConversation: async ( conversationId: string, newName: string, diff --git a/ts/packages/agentServer/server/src/conversationManager.ts b/ts/packages/agentServer/server/src/conversationManager.ts index 2749e57603..bf0e9e1a56 100644 --- a/ts/packages/agentServer/server/src/conversationManager.ts +++ b/ts/packages/agentServer/server/src/conversationManager.ts @@ -9,6 +9,7 @@ import { ConversationNameCollisionOptions, CreateConversationOptions, ConversationInfo, + ConversationMatch, ConversationSource, RenameConversationOptions, } from "@typeagent/agent-server-protocol"; @@ -23,6 +24,14 @@ import { createSharedDispatcher, SharedDispatcher, } from "./sharedDispatcher.js"; +import { + ConversationNameIndex, + createConversationNameIndex, +} from "./conversationNameIndex.js"; +import { + ConversationContentMatch, + createConversationSearchIndex, +} from "./conversationSearchIndex.js"; import { importCopilotSessions } from "./copilot/mirrorImporter.js"; import { lockInstanceDir } from "agent-dispatcher/internal"; @@ -152,6 +161,32 @@ export type ConversationManager = { connectionId: string, ): Promise; listConversations(name?: string): Promise; + /** + * Fuzzy-find conversations by name, blending lexical and embedding + * similarity. Sorted by descending relevance score. + */ + findConversations( + query: string, + maxMatches?: number, + ): Promise; + /** + * Index a conversation message into the unified content-search index + * (tagged by conversation id). Populated by callers as turns arrive. + */ + indexConversationMessage( + conversationId: string, + text: string, + sender?: string, + ): void; + /** + * Cross-conversation content search: rank conversations by how well their + * indexed messages match the query. Returns [] when the unified index has + * no model provider configured. + */ + searchConversationContent( + query: string, + maxMatches?: number, + ): Promise; renameConversation( conversationId: string, newName: string, @@ -265,6 +300,12 @@ export async function createConversationManager( const conversations = new Map(); + // Fuzzy index over conversation names (lexical + embedding), backing + // `findConversations`. Kept in sync as conversations are created, renamed, + // and deleted. + const conversationNameIndex: ConversationNameIndex = + createConversationNameIndex(); + // Single-flight lock for "auto-create the default conversation". Two // concurrent first-connects could both observe "no conversations exist" // and race; this serializes them so only one create happens. @@ -393,6 +434,23 @@ export async function createConversationManager( } } + // Build the wire-facing ConversationInfo for a record. Shared by + // listConversations and findConversations so the shape stays consistent. + async function toConversationInfo( + record: ConversationRecord, + ): Promise { + return { + conversationId: record.conversationId, + name: record.name ?? "", + clientCount: record.sharedDispatcher?.clientCount ?? 0, + createdAt: record.createdAt, + messageCount: await countUserMessages(record.conversationId), + ...(record.source !== undefined ? { source: record.source } : {}), + ...(record.readOnly !== undefined + ? { readOnly: record.readOnly } + : {}), + }; + } function ensureDispatcher( record: ConversationRecord, ): Promise { @@ -430,6 +488,17 @@ export async function createConversationManager( failed: r.failed, }), ), + // Tee this conversation's live turns into the unified + // content index, tagged by its id, so cross- + // conversation search can find it. Independent of the + // per-conversation memory (which connected mode leaves + // unextracted). + conversationContentSink: (text, sender) => + manager.indexConversationMessage( + record.conversationId, + text, + sender, + ), }), ) .then((dispatcher) => { @@ -630,6 +699,21 @@ export async function createConversationManager( } } + // Seed the fuzzy name index from the loaded registry (after the ephemeral + // sweep above). Names make lexical matching work immediately; embeddings + // are generated in the background so startup (and bulk imports) never block + // on embedding calls. + for (const record of conversations.values()) { + conversationNameIndex.update(record.conversationId, record.name); + } + void conversationNameIndex.prime(); + + // Unified content-search index across all conversations, tagged by + // conversation id. Inert when no model provider is configured. + const conversationSearchIndex = await createConversationSearchIndex( + path.join(conversationsDir, "_unified"), + ); + const manager: ConversationManager = { async createConversation( name: string, @@ -649,6 +733,7 @@ export async function createConversationManager( idleTimer: undefined, }; conversations.set(conversationId, record); + conversationNameIndex.update(conversationId, resolvedName); await saveMetadata(); debugConversation( `Conversation created: "${resolvedName}" (${conversationId})`, @@ -683,6 +768,10 @@ export async function createConversationManager( `Reconciling Copilot mirror name "${existing.name}" -> "${desiredName}" (${existing.conversationId})`, ); existing.name = desiredName; + conversationNameIndex.update( + existing.conversationId, + desiredName, + ); await saveMetadata(); renamed = true; } @@ -719,6 +808,7 @@ export async function createConversationManager( }, }; conversations.set(conversationId, record); + conversationNameIndex.update(conversationId, resolvedName); // Persist the synthesized display log so joining the conversation // replays the imported history through the normal replay path. @@ -931,25 +1021,56 @@ export async function createConversationManager( ) { continue; } + result.push(await toConversationInfo(record)); + } + return result; + }, + + async findConversations( + query: string, + maxMatches: number = 10, + ): Promise { + const matches = await conversationNameIndex.search( + query, + maxMatches, + ); + const result: ConversationMatch[] = []; + for (const match of matches) { + const record = conversations.get(match.conversationId); + // The index can briefly lag a concurrent delete; skip ids that + // no longer resolve to a live conversation. + if (record === undefined) { + continue; + } result.push({ - conversationId: record.conversationId, - name: recordName, - clientCount: record.sharedDispatcher?.clientCount ?? 0, - createdAt: record.createdAt, - messageCount: await countUserMessages( - record.conversationId, - ), - ...(record.source !== undefined - ? { source: record.source } - : {}), - ...(record.readOnly !== undefined - ? { readOnly: record.readOnly } - : {}), + conversation: await toConversationInfo(record), + score: match.score, }); } return result; }, + indexConversationMessage( + conversationId: string, + text: string, + sender?: string, + ): void { + conversationSearchIndex.addMessage(conversationId, text, sender); + }, + + async searchConversationContent( + query: string, + maxMatches: number = 10, + ): Promise { + // Drop hits for conversations that vanished from the registry + // (belt-and-suspenders; the index also tombstones on delete). + const matches = await conversationSearchIndex.search( + query, + maxMatches, + ); + return matches.filter((m) => conversations.has(m.conversationId)); + }, + async renameConversation( conversationId: string, newName: string, @@ -966,6 +1087,7 @@ export async function createConversationManager( conversationId, ); record.name = resolvedName; + conversationNameIndex.update(conversationId, resolvedName); await saveMetadata(); debugConversation( `Conversation renamed: "${resolvedName}" (${conversationId})`, @@ -987,6 +1109,8 @@ export async function createConversationManager( } conversations.delete(conversationId); + conversationNameIndex.remove(conversationId); + conversationSearchIndex.tombstone(conversationId); // Remove persist directory const persistDir = getConversationPersistDir(conversationId); @@ -1015,6 +1139,7 @@ export async function createConversationManager( } await Promise.all(promises); await saveMetadata(); + await conversationSearchIndex.close(); await unlockInstanceDir(); debugConversation("ConversationManager closed"); }, diff --git a/ts/packages/agentServer/server/src/conversationNameIndex.ts b/ts/packages/agentServer/server/src/conversationNameIndex.ts new file mode 100644 index 0000000000..cbe15047fc --- /dev/null +++ b/ts/packages/agentServer/server/src/conversationNameIndex.ts @@ -0,0 +1,272 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +import { + generateEmbedding, + indexesOfNearest, + NormalizedEmbedding, + SimilarityType, +} from "@typeagent/agent-runtime"; +import { + TextEmbeddingModel, + openai, + isEmbeddingAvailable, +} from "@typeagent/aiclient"; +import registerDebug from "debug"; + +const debug = registerDebug("agent-server:conversation:nameIndex"); +const debugError = registerDebug("agent-server:conversation:nameIndex:error"); + +/** A conversation matched by name, with a relevance score in [0, 1]. */ +export type ConversationNameMatch = { + conversationId: string; + score: number; +}; + +/** + * Fuzzy index over conversation names. Combines two signals so imprecise + * queries still match: + * - Lexical: exact / substring / edit-distance over the raw name. Always + * available; guarantees exact and substring hits rank at the top. + * - Embedding: cosine similarity over name embeddings. Catches semantically + * close names ("gym music" -> "workout playlist"). Skipped entirely when no + * embedding provider is configured, so the index degrades to lexical-only. + * + * The index owns embeddings (keyed by conversationId); the caller owns the + * conversation registry and drives {@link ConversationNameIndex.update} / + * {@link ConversationNameIndex.remove} as conversations are created, renamed, + * and deleted. Embeddings are generated lazily by {@link prime} so startup and + * bulk imports (e.g. `@copilot import`) never block on embedding calls. + */ +export interface ConversationNameIndex { + /** Add or update a conversation's name. Marks its embedding stale. */ + update(conversationId: string, name: string): void; + /** Drop a conversation from the index. */ + remove(conversationId: string): void; + /** Forget everything. */ + reset(): void; + /** Embed any entries whose embedding is missing or stale. Idempotent. */ + prime(): Promise; + /** + * Rank conversations against a query. Returns matches sorted by descending + * score, capped at maxMatches. Runs {@link prime} first so results reflect + * every known name. + */ + search(query: string, maxMatches: number): Promise; +} + +type NameEntry = { + name: string; + embedding: NormalizedEmbedding | undefined; +}; + +// Lexical scores are banded so exact/substring hits always outrank a fuzzy +// edit-distance hit, and edit-distance noise below the floor is dropped. +const SCORE_EXACT = 1; +const SCORE_NAME_CONTAINS_QUERY = 0.9; +const SCORE_QUERY_CONTAINS_NAME = 0.8; +const EDIT_SIMILARITY_FLOOR = 0.6; +const EDIT_SCORE_SCALE = 0.7; +// Embedding cosine below this is treated as noise (short strings sit high). +const EMBEDDING_SCORE_FLOOR = 0.78; + +export function createConversationNameIndex( + modelOverride?: TextEmbeddingModel, +): ConversationNameIndex { + const entries = new Map(); + + // Undefined when no embedding provider is configured; the index then does + // lexical-only matching instead of failing. + const embeddingModel: TextEmbeddingModel | undefined = + modelOverride ?? + (isEmbeddingAvailable() + ? openai.createEmbeddingModel( + openai.apiSettingsFromEnv(openai.ModelType.Embedding), + ) + : undefined); + + if (embeddingModel === undefined) { + debug( + "No embedding provider configured; conversation name search is lexical-only", + ); + } + + // Single-flight guard so concurrent prime() calls share one pass. + let primeInFlight: Promise | undefined; + + return { update, remove, reset, prime, search }; + + function update(conversationId: string, name: string): void { + const existing = entries.get(conversationId); + if (existing !== undefined && existing.name === name) { + return; + } + entries.set(conversationId, { name, embedding: undefined }); + } + + function remove(conversationId: string): void { + entries.delete(conversationId); + } + + function reset(): void { + entries.clear(); + } + + function prime(): Promise { + if (embeddingModel === undefined) { + return Promise.resolve(); + } + if (primeInFlight === undefined) { + primeInFlight = primeStaleEntries().finally(() => { + primeInFlight = undefined; + }); + } + return primeInFlight; + } + + async function primeStaleEntries(): Promise { + for (const [conversationId, entry] of entries) { + if (entry.embedding !== undefined) { + continue; + } + try { + entry.embedding = await generateEmbedding( + embeddingModel!, + entry.name, + ); + } catch (e: unknown) { + debugError( + `Could not embed name for ${conversationId}: ${e instanceof Error ? e.message : String(e)}`, + ); + } + } + } + + async function search( + query: string, + maxMatches: number, + ): Promise { + const trimmed = query.trim(); + if (trimmed.length === 0 || entries.size === 0) { + return []; + } + await prime(); + + const scores = new Map(); + const q = trimmed.toLowerCase(); + for (const [conversationId, entry] of entries) { + const score = lexicalScore(q, entry.name.trim().toLowerCase()); + if (score > 0) { + scores.set(conversationId, score); + } + } + + await addEmbeddingScores(trimmed, scores, maxMatches); + + const ranked = [...scores.entries()] + .map(([conversationId, score]) => ({ conversationId, score })) + .sort((a, b) => b.score - a.score); + return ranked.slice(0, maxMatches); + } + + async function addEmbeddingScores( + query: string, + scores: Map, + maxMatches: number, + ): Promise { + if (embeddingModel === undefined) { + return; + } + const ids: string[] = []; + const embeddings: NormalizedEmbedding[] = []; + for (const [conversationId, entry] of entries) { + if (entry.embedding !== undefined) { + ids.push(conversationId); + embeddings.push(entry.embedding); + } + } + if (embeddings.length === 0) { + return; + } + let queryEmbedding: NormalizedEmbedding; + try { + queryEmbedding = await generateEmbedding(embeddingModel, query); + } catch (e: unknown) { + debugError( + `Could not embed query "${query}": ${e instanceof Error ? e.message : String(e)}`, + ); + return; + } + // Pull a few extra candidates beyond maxMatches so the lexical/embedding + // merge has headroom before the final sort + slice. + const nearest = indexesOfNearest( + embeddings, + queryEmbedding, + Math.min(ids.length, maxMatches * 3), + SimilarityType.Dot, + ); + for (const match of nearest) { + if (match.score < EMBEDDING_SCORE_FLOOR) { + continue; + } + const conversationId = ids[Number(match.item)]; + const prev = scores.get(conversationId) ?? 0; + if (match.score > prev) { + scores.set(conversationId, match.score); + } + } + } +} + +// Lexical similarity of a lowercased query against a lowercased name. Returns 0 +// when there is no meaningful match so callers can drop the candidate. +function lexicalScore(query: string, name: string): number { + if (name.length === 0) { + return 0; + } + if (name === query) { + return SCORE_EXACT; + } + if (name.includes(query)) { + return SCORE_NAME_CONTAINS_QUERY; + } + if (query.includes(name)) { + return SCORE_QUERY_CONTAINS_NAME; + } + const distance = levenshtein(query, name); + const similarity = 1 - distance / Math.max(query.length, name.length); + return similarity >= EDIT_SIMILARITY_FLOOR + ? similarity * EDIT_SCORE_SCALE + : 0; +} + +// Iterative two-row Levenshtein edit distance. +function levenshtein(a: string, b: string): number { + if (a === b) { + return 0; + } + if (a.length === 0) { + return b.length; + } + if (b.length === 0) { + return a.length; + } + let prev = new Array(b.length + 1); + let curr = new Array(b.length + 1); + for (let j = 0; j <= b.length; j++) { + prev[j] = j; + } + for (let i = 1; i <= a.length; i++) { + curr[0] = i; + for (let j = 1; j <= b.length; j++) { + const cost = a[i - 1] === b[j - 1] ? 0 : 1; + curr[j] = Math.min( + prev[j] + 1, + curr[j - 1] + 1, + prev[j - 1] + cost, + ); + } + [prev, curr] = [curr, prev]; + } + return prev[b.length]; +} diff --git a/ts/packages/agentServer/server/src/conversationSearchIndex.ts b/ts/packages/agentServer/server/src/conversationSearchIndex.ts new file mode 100644 index 0000000000..6fc2584ab3 --- /dev/null +++ b/ts/packages/agentServer/server/src/conversationSearchIndex.ts @@ -0,0 +1,228 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +import { + ConversationMemory, + ConversationMessage, + ConversationMessageMeta, + createConversationMemory, +} from "@typeagent/conversation-memory"; +import registerDebug from "debug"; + +const debug = registerDebug("agent-server:conversation:searchIndex"); +const debugError = registerDebug("agent-server:conversation:searchIndex:error"); + +// Message tag carrying the owning conversation id. A plain-string tag (knowPro +// `MessageTag = string | StructuredTag`) is enough: cross-conversation search +// runs unscoped, then reads this tag back off each matched message to group +// hits by conversation. +const CONV_TAG_PREFIX = "conv:"; +const UNIFIED_MEMORY_BASENAME = "unifiedMemory"; + +/** A conversation whose content matched a query, with representative snippets. */ +export type ConversationContentMatch = { + conversationId: string; + /** Best (highest) matching-message score for this conversation. */ + score: number; + /** Top matching message texts, best first. */ + snippets: string[]; +}; + +/** + * A single knowPro index spanning every conversation's messages, each tagged + * with its owning conversation id. Backs cross-conversation content search + * ("which conversation was this discussed in?") without spinning up each + * conversation's own dispatcher. + * + * Deletes are handled by tombstoning the conversation id (filtered out of + * results) because knowPro collections are append-only; a later compaction + * pass rebuilds the index to reclaim the space. + */ +export interface ConversationSearchIndex { + /** Queue a conversation message for indexing, tagged by conversation id. */ + addMessage(conversationId: string, text: string, sender?: string): void; + /** Exclude a (deleted) conversation from future search results. */ + tombstone(conversationId: string): void; + /** Rank conversations by how well their content matches the query. */ + search( + query: string, + maxConversations?: number, + maxSnippetsPerConversation?: number, + ): Promise; + /** Await all queued indexing work (e.g. before search in tests). */ + waitForPendingTasks(): Promise; + close(): Promise; +} + +function conversationTag(conversationId: string): string { + return CONV_TAG_PREFIX + conversationId; +} + +function conversationIdFromTags( + tags: ReadonlyArray, +): string | undefined { + for (const tag of tags) { + if (typeof tag === "string" && tag.startsWith(CONV_TAG_PREFIX)) { + return tag.slice(CONV_TAG_PREFIX.length); + } + } + return undefined; +} + +/** + * Group scored message hits into per-conversation matches. Pure so it can be + * unit tested without a live model: `getMessage` yields each message's text and + * (tag-derived) conversation id, `isTombstoned` filters deleted conversations. + */ +export function rankConversationMatches( + messageMatches: ReadonlyArray<{ messageOrdinal: number; score: number }>, + getMessage: (ordinal: number) => { + text: string; + conversationId: string | undefined; + }, + isTombstoned: (conversationId: string) => boolean, + maxConversations: number, + maxSnippetsPerConversation: number, +): ConversationContentMatch[] { + const byConversation = new Map< + string, + { score: number; snippets: { text: string; score: number }[] } + >(); + for (const { messageOrdinal, score } of messageMatches) { + const { text, conversationId } = getMessage(messageOrdinal); + if (conversationId === undefined || isTombstoned(conversationId)) { + continue; + } + let entry = byConversation.get(conversationId); + if (entry === undefined) { + entry = { score: 0, snippets: [] }; + byConversation.set(conversationId, entry); + } + entry.score = Math.max(entry.score, score); + const snippet = text.trim(); + if (snippet.length > 0) { + entry.snippets.push({ text: snippet, score }); + } + } + const results: ConversationContentMatch[] = []; + for (const [conversationId, entry] of byConversation) { + const snippets = entry.snippets + .sort((a, b) => b.score - a.score) + .slice(0, maxSnippetsPerConversation) + .map((s) => s.text); + results.push({ conversationId, score: entry.score, snippets }); + } + results.sort((a, b) => b.score - a.score); + return results.slice(0, maxConversations); +} + +class ConversationSearchIndexImpl implements ConversationSearchIndex { + private readonly tombstoned = new Set(); + + constructor( + // Undefined when no model provider is configured; the index is then + // inert (addMessage no-ops, search returns nothing) rather than failing. + private readonly memory: ConversationMemory | undefined, + // Extract entities/topics per message for richer retrieval. Production + // leaves this on so memory search works well; tests turn it off. + private readonly extractKnowledge: boolean, + ) {} + + public addMessage( + conversationId: string, + text: string, + sender?: string, + ): void { + if (this.memory === undefined || text.trim().length === 0) { + return; + } + this.memory.queueAddMessage( + new ConversationMessage(text, new ConversationMessageMeta(sender), [ + conversationTag(conversationId), + ]), + undefined, + this.extractKnowledge, + ); + } + + public tombstone(conversationId: string): void { + this.tombstoned.add(conversationId); + } + + public async search( + query: string, + maxConversations: number = 10, + maxSnippetsPerConversation: number = 3, + ): Promise { + if (this.memory === undefined || query.trim().length === 0) { + return []; + } + const result = await this.memory.searchWithLanguage(query); + if (!result.success) { + debugError(`Unified content search failed: ${result.message}`); + return []; + } + const messageMatches = result.data.flatMap((r) => r.messageMatches); + const memory = this.memory; + return rankConversationMatches( + messageMatches, + (ordinal) => { + const message = memory.messages.get(ordinal); + return { + text: message?.textChunks?.join(" ") ?? "", + conversationId: conversationIdFromTags(message?.tags ?? []), + }; + }, + (conversationId) => this.tombstoned.has(conversationId), + maxConversations, + maxSnippetsPerConversation, + ); + } + + public async waitForPendingTasks(): Promise { + await this.memory?.waitForPendingTasks(); + } + + public async close(): Promise { + // Each queued add auto-saves; just drain any in-flight work. + await this.waitForPendingTasks(); + } +} + +/** Options for {@link createConversationSearchIndex}. */ +export type ConversationSearchIndexOptions = { + /** + * Extract entities/topics from each message as it is indexed, for richer + * memory retrieval. Defaults to true. Tests disable it to avoid the + * per-message extraction model call. + */ + extractKnowledge?: boolean; +}; + +/** + * Open (or create) the unified conversation search index under `dirPath`. + * Never throws: when no model provider is configured the returned index is + * inert, so callers can wire it unconditionally. + */ +export async function createConversationSearchIndex( + dirPath: string, + options?: ConversationSearchIndexOptions, +): Promise { + const extractKnowledge = options?.extractKnowledge ?? true; + let memory: ConversationMemory | undefined; + try { + memory = await createConversationMemory( + { dirPath, baseFileName: UNIFIED_MEMORY_BASENAME }, + false, + ); + } catch (e: unknown) { + debugError( + `Unified content search disabled (memory init failed): ${e instanceof Error ? e.message : String(e)}`, + ); + memory = undefined; + } + if (memory !== undefined) { + debug(`Unified conversation search index ready at ${dirPath}`); + } + return new ConversationSearchIndexImpl(memory, extractKnowledge); +} diff --git a/ts/packages/agentServer/server/src/copilot/mirrorImporter.ts b/ts/packages/agentServer/server/src/copilot/mirrorImporter.ts index 052becdd45..634b8fd914 100644 --- a/ts/packages/agentServer/server/src/copilot/mirrorImporter.ts +++ b/ts/packages/agentServer/server/src/copilot/mirrorImporter.ts @@ -146,6 +146,25 @@ export async function importCopilotSessions( if (res.created) { result.imported++; + // Read-only mirrors never replay through the live tee, so + // populate the unified content index explicitly here. Gated + // on `created` so re-imports don't double-index. + for (const turn of turns) { + if (turn.userMessage) { + conversationManager.indexConversationMessage( + res.conversationId, + turn.userMessage, + "user", + ); + } + if (turn.assistantResponse) { + conversationManager.indexConversationMessage( + res.conversationId, + turn.assistantResponse, + "assistant", + ); + } + } } else if (res.renamed) { // Existing mirror whose title we reconciled — count it // separately from unchanged skips. diff --git a/ts/packages/agentServer/server/test/conversationNameIndex.spec.ts b/ts/packages/agentServer/server/test/conversationNameIndex.spec.ts new file mode 100644 index 0000000000..0df6f530db --- /dev/null +++ b/ts/packages/agentServer/server/test/conversationNameIndex.spec.ts @@ -0,0 +1,143 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +import type { TextEmbeddingModel } from "@typeagent/aiclient"; +import { createConversationNameIndex } from "../src/conversationNameIndex.js"; + +// A tiny deterministic embedding space so tests can assert semantic matches +// without any network calls. Each recognized keyword maps to a dimension; +// unrecognized words land on a shared "other" dimension so unknown names stay +// orthogonal to the keyword dimensions. +const DIMS = 5; +const OTHER_DIM = 4; +const KEYWORD_DIMS: Record = { + workout: 0, + gym: 0, + exercise: 0, + fitness: 0, + playlist: 1, + music: 1, + song: 1, + songs: 1, + grocery: 2, + groceries: 2, + shopping: 2, + food: 2, + trip: 3, + travel: 3, + paris: 3, + france: 3, + vacation: 3, +}; + +function keywordVector(text: string): number[] { + const v = new Array(DIMS).fill(0); + let matched = false; + for (const word of text.toLowerCase().split(/[^a-z]+/)) { + const dim = KEYWORD_DIMS[word]; + if (dim !== undefined) { + v[dim] += 1; + matched = true; + } + } + if (!matched) { + v[OTHER_DIM] = 1; + } + return v; +} + +function keywordModel(): TextEmbeddingModel { + return { + generateEmbedding: async (text: string) => ({ + success: true as const, + data: keywordVector(text), + }), + maxBatchSize: 1, + } as unknown as TextEmbeddingModel; +} + +function failingModel(): TextEmbeddingModel { + return { + generateEmbedding: async () => ({ + success: false as const, + message: "no embedding provider", + }), + maxBatchSize: 1, + } as unknown as TextEmbeddingModel; +} + +const CONVERSATIONS: [string, string][] = [ + ["id-workout", "workout playlist setup"], + ["id-grocery", "grocery shopping list"], + ["id-paris", "trip to Paris"], +]; + +function seededIndex(model: TextEmbeddingModel) { + const index = createConversationNameIndex(model); + for (const [id, name] of CONVERSATIONS) { + index.update(id, name); + } + return index; +} + +describe("conversationNameIndex", () => { + it("ranks an exact name match first", async () => { + const index = seededIndex(keywordModel()); + const matches = await index.search("workout playlist setup", 10); + expect(matches[0].conversationId).toBe("id-workout"); + expect(matches[0].score).toBeGreaterThanOrEqual(0.99); + }); + + it("matches on a substring of the name", async () => { + const index = seededIndex(keywordModel()); + const matches = await index.search("grocery shopping", 10); + expect(matches[0].conversationId).toBe("id-grocery"); + }); + + it("surfaces a semantically related name via embeddings", async () => { + const index = seededIndex(keywordModel()); + // No lexical overlap with "workout playlist setup", but the same + // fitness + music concepts, so the embedding half must find it. + const matches = await index.search("gym music", 10); + expect(matches.length).toBeGreaterThan(0); + expect(matches[0].conversationId).toBe("id-workout"); + }); + + it("drops a conversation after remove()", async () => { + const index = seededIndex(keywordModel()); + index.remove("id-workout"); + const matches = await index.search("workout playlist setup", 10); + expect( + matches.find((m) => m.conversationId === "id-workout"), + ).toBeUndefined(); + }); + + it("reflects a renamed conversation", async () => { + const index = seededIndex(keywordModel()); + index.update("id-grocery", "trip to France"); + const matches = await index.search("trip to France", 10); + expect(matches[0].conversationId).toBe("id-grocery"); + }); + + it("degrades to lexical-only when embeddings are unavailable", async () => { + const index = seededIndex(failingModel()); + // Exact/substring still work without embeddings... + const exact = await index.search("workout playlist setup", 10); + expect(exact[0]?.conversationId).toBe("id-workout"); + // ...but a purely semantic query has nothing to match lexically. + const semantic = await index.search("gym music", 10); + expect(semantic).toHaveLength(0); + }); + + it("returns nothing for an empty query", async () => { + const index = seededIndex(keywordModel()); + expect(await index.search(" ", 10)).toHaveLength(0); + }); + + it("honors the maxMatches cap", async () => { + const index = seededIndex(keywordModel()); + // A query that lexically matches every name (each contains a space). + const matches = await index.search("i", 1); + expect(matches.length).toBeLessThanOrEqual(1); + }); +}); diff --git a/ts/packages/agentServer/server/test/conversationSearchIndex.spec.ts b/ts/packages/agentServer/server/test/conversationSearchIndex.spec.ts new file mode 100644 index 0000000000..2354cbe29a --- /dev/null +++ b/ts/packages/agentServer/server/test/conversationSearchIndex.spec.ts @@ -0,0 +1,100 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +import { rankConversationMatches } from "../src/conversationSearchIndex.js"; + +// Fake message store keyed by ordinal, mirroring what the real index derives +// from a matched message (its text + the conversation id read off its tag). +const MESSAGES: Record< + number, + { text: string; conversationId: string | undefined } +> = { + 0: { text: "workout playlist", conversationId: "A" }, + 1: { text: "more workout notes", conversationId: "A" }, + 2: { text: "grocery list", conversationId: "B" }, + 3: { text: "untagged message", conversationId: undefined }, +}; + +const getMessage = (ordinal: number) => MESSAGES[ordinal]; +const noTombstones = () => false; + +const MATCHES = [ + { messageOrdinal: 0, score: 0.9 }, + { messageOrdinal: 1, score: 0.5 }, + { messageOrdinal: 2, score: 0.8 }, + { messageOrdinal: 3, score: 0.3 }, +]; + +describe("rankConversationMatches", () => { + it("groups hits by conversation and keeps the best score", () => { + const result = rankConversationMatches( + MATCHES, + getMessage, + noTombstones, + 10, + 3, + ); + expect(result.map((m) => m.conversationId)).toEqual(["A", "B"]); + expect(result[0].score).toBeCloseTo(0.9); + // Snippets are ordered best-first within a conversation. + expect(result[0].snippets).toEqual([ + "workout playlist", + "more workout notes", + ]); + expect(result[1].score).toBeCloseTo(0.8); + }); + + it("skips messages with no conversation tag", () => { + const result = rankConversationMatches( + MATCHES, + getMessage, + noTombstones, + 10, + 3, + ); + // The untagged ordinal (3) must not create a phantom conversation. + expect(result.some((m) => m.conversationId === undefined)).toBe(false); + expect(result).toHaveLength(2); + }); + + it("excludes tombstoned conversations", () => { + const result = rankConversationMatches( + MATCHES, + getMessage, + (id) => id === "A", + 10, + 3, + ); + expect(result.map((m) => m.conversationId)).toEqual(["B"]); + }); + + it("caps snippets per conversation, best first", () => { + const result = rankConversationMatches( + MATCHES, + getMessage, + noTombstones, + 10, + 1, + ); + const a = result.find((m) => m.conversationId === "A")!; + expect(a.snippets).toEqual(["workout playlist"]); + }); + + it("caps the number of conversations returned", () => { + const result = rankConversationMatches( + MATCHES, + getMessage, + noTombstones, + 1, + 3, + ); + expect(result).toHaveLength(1); + expect(result[0].conversationId).toBe("A"); + }); + + it("returns nothing for no matches", () => { + expect( + rankConversationMatches([], getMessage, noTombstones, 10, 3), + ).toHaveLength(0); + }); +}); diff --git a/ts/packages/agentServer/server/test/copilotImport.spec.ts b/ts/packages/agentServer/server/test/copilotImport.spec.ts index a25fec0e0b..81a7026cbe 100644 --- a/ts/packages/agentServer/server/test/copilotImport.spec.ts +++ b/ts/packages/agentServer/server/test/copilotImport.spec.ts @@ -1,7 +1,7 @@ // Copyright (c) Microsoft Corporation. // Licensed under the MIT License. -import { afterEach, describe, expect, test } from "@jest/globals"; +import { afterEach, describe, expect, jest, test } from "@jest/globals"; import Database from "better-sqlite3"; import * as fs from "node:fs/promises"; import * as os from "node:os"; @@ -510,6 +510,55 @@ describe("importCopilotSessions", () => { } }); + test("indexes imported turns into the unified content search index", async () => { + const dbPath = await createSeededStore([ + { + id: "sess-idx", + summary: "gym playlist", + createdAt: "2026-06-01T10:00:00.000Z", + updatedAt: "2026-06-01T10:30:00.000Z", + turns: [ + { + turnIndex: 0, + userMessage: "build a workout playlist", + assistantResponse: "Added upbeat gym music.", + timestamp: "2026-06-01T10:00:00.000Z", + }, + { + turnIndex: 1, + userMessage: null, + assistantResponse: "Anything else?", + timestamp: "2026-06-01T10:05:00.000Z", + }, + ], + }, + ]); + const baseDir = await createTempDir(); + const manager = await createManager(baseDir); + const indexed: { + id: string; + text: string; + sender: string | undefined; + }[] = []; + jest.spyOn(manager, "indexConversationMessage").mockImplementation( + (id, text, sender) => indexed.push({ id, text, sender }), + ); + try { + const result = await importCopilotSessions(manager, { dbPath }); + expect(result.imported).toBe(1); + const id = (await manager.listConversations())[0].conversationId; + // Both roles indexed in turn order; the null user message is + // skipped, not indexed as empty. + expect(indexed).toEqual([ + { id, text: "build a workout playlist", sender: "user" }, + { id, text: "Added upbeat gym music.", sender: "assistant" }, + { id, text: "Anything else?", sender: "assistant" }, + ]); + } finally { + await manager.close(); + } + }); + test("is idempotent: re-importing skips existing mirrors", async () => { const dbPath = await createSeededStore([ { diff --git a/ts/packages/agents/browser/package.json b/ts/packages/agents/browser/package.json index e1fc8f7221..0dd6672263 100644 --- a/ts/packages/agents/browser/package.json +++ b/ts/packages/agents/browser/package.json @@ -132,7 +132,8 @@ "src/agent/browserSchema.agr" ], "outputGlobs": [ - "dist/agent/browserSchema.ag.json" + "dist/agent/browserSchema.ag.json", + "dist/agent/browserSchema.ag.map.json" ] } }, diff --git a/ts/packages/agents/browserExtension/src/extension/serviceWorker/dispatcherConnection.ts b/ts/packages/agents/browserExtension/src/extension/serviceWorker/dispatcherConnection.ts index 335c4449c7..33314e68cd 100644 --- a/ts/packages/agents/browserExtension/src/extension/serviceWorker/dispatcherConnection.ts +++ b/ts/packages/agents/browserExtension/src/extension/serviceWorker/dispatcherConnection.ts @@ -559,6 +559,10 @@ function makeConnectionAdapter(): AgentServerConnection { const { rpc } = requireFresh(); return rpc.invoke("listConversations", name); }, + findConversations: (query: string, maxMatches?: number) => { + const { rpc } = requireFresh(); + return rpc.invoke("findConversations", query, maxMatches); + }, renameConversation: (id: string, newName: string) => { const { rpc } = requireFresh(); return rpc.invoke( @@ -717,5 +721,21 @@ function renderActionResult( .join(""); return ok(`
    ${items}
`); } + case "matches": { + const items = result.matches + .map((m) => { + const c = m.conversation; + const cur = + c.conversationId === result.currentConversationId + ? "▸ " + : ""; + const pct = Math.round(m.score * 100); + return `
  • ${cur}${escapeHtml(c.name)} (${pct}%)
    • `; + }) + .join(""); + return ok( + `Matches for “${escapeHtml(result.query)}”:
      ${items}
    `, + ); + } } } diff --git a/ts/packages/agents/calendar/package.json b/ts/packages/agents/calendar/package.json index dc9c2f42e1..c1ef6b7dbe 100644 --- a/ts/packages/agents/calendar/package.json +++ b/ts/packages/agents/calendar/package.json @@ -65,7 +65,8 @@ "src/calendarSchema.agr" ], "outputGlobs": [ - "dist/calendarSchema.ag.json" + "dist/calendarSchema.ag.json", + "dist/calendarSchema.ag.map.json" ] } }, diff --git a/ts/packages/agents/code/package.json b/ts/packages/agents/code/package.json index 92c1f6cf91..1cced04d5e 100644 --- a/ts/packages/agents/code/package.json +++ b/ts/packages/agents/code/package.json @@ -84,7 +84,8 @@ "src/codeSchema.agr" ], "outputGlobs": [ - "dist/codeSchema.ag.json" + "dist/codeSchema.ag.json", + "dist/codeSchema.ag.map.json" ] } }, @@ -97,7 +98,8 @@ "src/vscode/debugActionsSchema.agr" ], "outputGlobs": [ - "dist/debugSchema.ag.json" + "dist/debugSchema.ag.json", + "dist/debugSchema.ag.map.json" ] } }, @@ -110,7 +112,8 @@ "src/vscode/displayActionsSchema.agr" ], "outputGlobs": [ - "dist/displaySchema.ag.json" + "dist/displaySchema.ag.json", + "dist/displaySchema.ag.map.json" ] } }, @@ -123,7 +126,8 @@ "src/vscode/extensionsActionsSchema.agr" ], "outputGlobs": [ - "dist/extensionSchema.ag.json" + "dist/extensionSchema.ag.json", + "dist/extensionSchema.ag.map.json" ] } }, @@ -136,7 +140,8 @@ "src/vscode/generalActionsSchema.agr" ], "outputGlobs": [ - "dist/generalSchema.ag.json" + "dist/generalSchema.ag.json", + "dist/generalSchema.ag.map.json" ] } }, @@ -149,7 +154,8 @@ "src/vscode/vscodeConversationActionsSchema.agr" ], "outputGlobs": [ - "dist/vscodeShellSchema.ag.json" + "dist/vscodeShellSchema.ag.json", + "dist/vscodeShellSchema.ag.map.json" ] } }, @@ -162,7 +168,8 @@ "src/vscode/workbenchCommandActionsSchema.agr" ], "outputGlobs": [ - "dist/workbenchSchema.ag.json" + "dist/workbenchSchema.ag.json", + "dist/workbenchSchema.ag.map.json" ] } }, diff --git a/ts/packages/agents/desktop/package.json b/ts/packages/agents/desktop/package.json index 730946cde9..a0d662ae4c 100644 --- a/ts/packages/agents/desktop/package.json +++ b/ts/packages/agents/desktop/package.json @@ -95,7 +95,8 @@ "src/desktopSchema.agr" ], "outputGlobs": [ - "dist/desktopSchema.ag.json" + "dist/desktopSchema.ag.json", + "dist/desktopSchema.ag.map.json" ] } }, @@ -108,7 +109,8 @@ "src/windows/displaySchema.agr" ], "outputGlobs": [ - "dist/displaySchema.ag.json" + "dist/displaySchema.ag.json", + "dist/displaySchema.ag.map.json" ] } }, @@ -121,7 +123,8 @@ "src/windows/inputSchema.agr" ], "outputGlobs": [ - "dist/inputSchema.ag.json" + "dist/inputSchema.ag.json", + "dist/inputSchema.ag.map.json" ] } }, @@ -134,7 +137,8 @@ "src/windows/personalizationSchema.agr" ], "outputGlobs": [ - "dist/personalizationSchema.ag.json" + "dist/personalizationSchema.ag.json", + "dist/personalizationSchema.ag.map.json" ] } }, @@ -147,7 +151,8 @@ "src/windows/powerSchema.agr" ], "outputGlobs": [ - "dist/powerSchema.ag.json" + "dist/powerSchema.ag.json", + "dist/powerSchema.ag.map.json" ] } }, @@ -160,7 +165,8 @@ "src/windows/privacySchema.agr" ], "outputGlobs": [ - "dist/privacySchema.ag.json" + "dist/privacySchema.ag.json", + "dist/privacySchema.ag.map.json" ] } }, @@ -173,7 +179,8 @@ "src/windows/systemSchema.agr" ], "outputGlobs": [ - "dist/systemSchema.ag.json" + "dist/systemSchema.ag.json", + "dist/systemSchema.ag.map.json" ] } }, @@ -186,7 +193,8 @@ "src/windows/taskbarSchema.agr" ], "outputGlobs": [ - "dist/taskbarSchema.ag.json" + "dist/taskbarSchema.ag.json", + "dist/taskbarSchema.ag.map.json" ] } }, diff --git a/ts/packages/agents/github-cli/package.json b/ts/packages/agents/github-cli/package.json index 54a87effbb..9e69117b10 100644 --- a/ts/packages/agents/github-cli/package.json +++ b/ts/packages/agents/github-cli/package.json @@ -56,7 +56,8 @@ "src/github-cliSchema.agr" ], "outputGlobs": [ - "dist/github-cliSchema.ag.json" + "dist/github-cliSchema.ag.json", + "dist/github-cliSchema.ag.map.json" ] } }, diff --git a/ts/packages/agents/list/package.json b/ts/packages/agents/list/package.json index 38124b3c0b..fef371b203 100644 --- a/ts/packages/agents/list/package.json +++ b/ts/packages/agents/list/package.json @@ -59,7 +59,8 @@ "src/listSchema.agr" ], "outputGlobs": [ - "dist/listSchema.ag.json" + "dist/listSchema.ag.json", + "dist/listSchema.ag.map.json" ] } }, diff --git a/ts/packages/agents/onboarding/package.json b/ts/packages/agents/onboarding/package.json index 548a774950..cd4753e6f4 100644 --- a/ts/packages/agents/onboarding/package.json +++ b/ts/packages/agents/onboarding/package.json @@ -82,7 +82,8 @@ "src/onboardingSchema.agr" ], "outputGlobs": [ - "dist/onboardingSchema.ag.json" + "dist/onboardingSchema.ag.json", + "dist/onboardingSchema.ag.map.json" ] } }, @@ -95,7 +96,8 @@ "src/discovery/discoverySchema.agr" ], "outputGlobs": [ - "dist/discoverySchema.ag.json" + "dist/discoverySchema.ag.json", + "dist/discoverySchema.ag.map.json" ] } }, @@ -108,7 +110,8 @@ "src/grammarGen/grammarGenSchema.agr" ], "outputGlobs": [ - "dist/grammarGenSchema.ag.json" + "dist/grammarGenSchema.ag.json", + "dist/grammarGenSchema.ag.map.json" ] } }, @@ -121,7 +124,8 @@ "src/packaging/packagingSchema.agr" ], "outputGlobs": [ - "dist/packagingSchema.ag.json" + "dist/packagingSchema.ag.json", + "dist/packagingSchema.ag.map.json" ] } }, @@ -134,7 +138,8 @@ "src/phraseGen/phraseGenSchema.agr" ], "outputGlobs": [ - "dist/phraseGenSchema.ag.json" + "dist/phraseGenSchema.ag.json", + "dist/phraseGenSchema.ag.map.json" ] } }, @@ -147,7 +152,8 @@ "src/scaffolder/scaffolderSchema.agr" ], "outputGlobs": [ - "dist/scaffolderSchema.ag.json" + "dist/scaffolderSchema.ag.json", + "dist/scaffolderSchema.ag.map.json" ] } }, @@ -160,7 +166,8 @@ "src/schemaGen/schemaGenSchema.agr" ], "outputGlobs": [ - "dist/schemaGenSchema.ag.json" + "dist/schemaGenSchema.ag.json", + "dist/schemaGenSchema.ag.map.json" ] } }, @@ -173,7 +180,8 @@ "src/testing/testingSchema.agr" ], "outputGlobs": [ - "dist/testingSchema.ag.json" + "dist/testingSchema.ag.json", + "dist/testingSchema.ag.map.json" ] } }, diff --git a/ts/packages/agents/osNotifications/package.json b/ts/packages/agents/osNotifications/package.json index e25ada7587..e95d594778 100644 --- a/ts/packages/agents/osNotifications/package.json +++ b/ts/packages/agents/osNotifications/package.json @@ -65,7 +65,8 @@ "src/osNotificationsSchema.agr" ], "outputGlobs": [ - "dist/osNotificationsSchema.ag.json" + "dist/osNotificationsSchema.ag.json", + "dist/osNotificationsSchema.ag.map.json" ] } }, diff --git a/ts/packages/agents/player/package.json b/ts/packages/agents/player/package.json index 9ae49ff7f8..2a04b59b36 100644 --- a/ts/packages/agents/player/package.json +++ b/ts/packages/agents/player/package.json @@ -68,7 +68,8 @@ "src/agent/playerSchema.agr" ], "outputGlobs": [ - "dist/agent/playerSchema.ag.json" + "dist/agent/playerSchema.ag.json", + "dist/agent/playerSchema.ag.map.json" ] } }, diff --git a/ts/packages/agents/playerLocal/package.json b/ts/packages/agents/playerLocal/package.json index 2ce93dab10..1c98eaa499 100644 --- a/ts/packages/agents/playerLocal/package.json +++ b/ts/packages/agents/playerLocal/package.json @@ -61,7 +61,8 @@ "src/agent/localPlayerSchema.agr" ], "outputGlobs": [ - "dist/agent/localPlayerSchema.ag.json" + "dist/agent/localPlayerSchema.ag.json", + "dist/agent/localPlayerSchema.ag.map.json" ] } }, diff --git a/ts/packages/agents/powershell/package.json b/ts/packages/agents/powershell/package.json index dad2640478..a1389e8640 100644 --- a/ts/packages/agents/powershell/package.json +++ b/ts/packages/agents/powershell/package.json @@ -78,7 +78,8 @@ "src/powershellSchema.agr" ], "outputGlobs": [ - "dist/powershellSchema.ag.json" + "dist/powershellSchema.ag.json", + "dist/powershellSchema.ag.map.json" ] } }, @@ -91,7 +92,8 @@ "src/namespaces/files/filesSchema.agr" ], "outputGlobs": [ - "dist/filesSchema.ag.json" + "dist/filesSchema.ag.json", + "dist/filesSchema.ag.map.json" ] } }, @@ -104,7 +106,8 @@ "src/namespaces/processes/processesSchema.agr" ], "outputGlobs": [ - "dist/processesSchema.ag.json" + "dist/processesSchema.ag.json", + "dist/processesSchema.ag.map.json" ] } }, @@ -117,7 +120,8 @@ "src/namespaces/system/systemSchema.agr" ], "outputGlobs": [ - "dist/systemSchema.ag.json" + "dist/systemSchema.ag.json", + "dist/systemSchema.ag.map.json" ] } }, @@ -130,7 +134,8 @@ "src/namespaces/services/servicesSchema.agr" ], "outputGlobs": [ - "dist/servicesSchema.ag.json" + "dist/servicesSchema.ag.json", + "dist/servicesSchema.ag.map.json" ] } }, @@ -143,7 +148,8 @@ "src/namespaces/network/networkSchema.agr" ], "outputGlobs": [ - "dist/networkSchema.ag.json" + "dist/networkSchema.ag.json", + "dist/networkSchema.ag.map.json" ] } }, @@ -156,7 +162,8 @@ "src/namespaces/data/dataSchema.agr" ], "outputGlobs": [ - "dist/dataSchema.ag.json" + "dist/dataSchema.ag.json", + "dist/dataSchema.ag.map.json" ] } }, @@ -169,7 +176,8 @@ "src/namespaces/archives/archivesSchema.agr" ], "outputGlobs": [ - "dist/archivesSchema.ag.json" + "dist/archivesSchema.ag.json", + "dist/archivesSchema.ag.map.json" ] } }, diff --git a/ts/packages/agents/screencapture/package.json b/ts/packages/agents/screencapture/package.json index 54dada2eca..cda368d185 100644 --- a/ts/packages/agents/screencapture/package.json +++ b/ts/packages/agents/screencapture/package.json @@ -63,7 +63,8 @@ "src/screencaptureSchema.agr" ], "outputGlobs": [ - "dist/screencaptureSchema.ag.json" + "dist/screencaptureSchema.ag.json", + "dist/screencaptureSchema.ag.map.json" ] } }, diff --git a/ts/packages/agents/timer/package.json b/ts/packages/agents/timer/package.json index cb404faffa..51e1541807 100644 --- a/ts/packages/agents/timer/package.json +++ b/ts/packages/agents/timer/package.json @@ -54,7 +54,8 @@ "src/timerSchema.agr" ], "outputGlobs": [ - "dist/timerSchema.ag.json" + "dist/timerSchema.ag.json", + "dist/timerSchema.ag.map.json" ] } }, diff --git a/ts/packages/agents/utility/package.json b/ts/packages/agents/utility/package.json index 62c79862cd..c67eaf95e1 100644 --- a/ts/packages/agents/utility/package.json +++ b/ts/packages/agents/utility/package.json @@ -67,7 +67,8 @@ "src/utilitySchema.agr" ], "outputGlobs": [ - "dist/utilitySchema.ag.json" + "dist/utilitySchema.ag.json", + "dist/utilitySchema.ag.map.json" ] } }, diff --git a/ts/packages/agents/vampire/package.json b/ts/packages/agents/vampire/package.json index b4af6101b1..b6af8a8dcc 100644 --- a/ts/packages/agents/vampire/package.json +++ b/ts/packages/agents/vampire/package.json @@ -50,7 +50,8 @@ "src/vampireSchema.agr" ], "outputGlobs": [ - "dist/vampireSchema.ag.json" + "dist/vampireSchema.ag.json", + "dist/vampireSchema.ag.map.json" ] } }, diff --git a/ts/packages/agents/visualStudio/package.json b/ts/packages/agents/visualStudio/package.json index e2aeb2c8d0..eaa23f8726 100644 --- a/ts/packages/agents/visualStudio/package.json +++ b/ts/packages/agents/visualStudio/package.json @@ -56,7 +56,8 @@ "src/visualStudioSchema.agr" ], "outputGlobs": [ - "dist/visualStudioSchema.ag.json" + "dist/visualStudioSchema.ag.json", + "dist/visualStudioSchema.ag.map.json" ] } }, diff --git a/ts/packages/agents/weather/package.json b/ts/packages/agents/weather/package.json index 65ebb7b7b2..4175069178 100644 --- a/ts/packages/agents/weather/package.json +++ b/ts/packages/agents/weather/package.json @@ -60,7 +60,8 @@ "src/weatherSchema.agr" ], "outputGlobs": [ - "dist/weatherSchema.ag.json" + "dist/weatherSchema.ag.json", + "dist/weatherSchema.ag.map.json" ] } }, diff --git a/ts/packages/cli/src/commands/conversations/find.ts b/ts/packages/cli/src/commands/conversations/find.ts new file mode 100644 index 0000000000..3ee9ba08fd --- /dev/null +++ b/ts/packages/cli/src/commands/conversations/find.ts @@ -0,0 +1,85 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +import { Args, Command, Flags } from "@oclif/core"; +import { + connectAgentServer, + AGENT_SERVER_DEFAULT_PORT, +} from "@typeagent/agent-server-client"; +import type { ConversationMatch } from "@typeagent/agent-server-client"; + +function formatTable(matches: ConversationMatch[]): string { + if (matches.length === 0) { + return "No matching conversations found."; + } + + const rows = matches.map((m) => ({ + score: m.score.toFixed(2), + id: m.conversation.conversationId, + name: m.conversation.name ?? "", + })); + + const scoreWidth = Math.max( + "SCORE".length, + ...rows.map((r) => r.score.length), + ); + const idWidth = Math.max( + "CONVERSATION ID".length, + ...rows.map((r) => r.id.length), + ); + const nameWidth = Math.max( + "NAME".length, + ...rows.map((r) => r.name.length), + ); + + const header = [ + "SCORE".padEnd(scoreWidth), + "CONVERSATION ID".padEnd(idWidth), + "NAME", + ].join(" "); + const separator = [ + "-".repeat(scoreWidth), + "-".repeat(idWidth), + "-".repeat(nameWidth), + ].join(" "); + const body = rows.map((r) => + [r.score.padEnd(scoreWidth), r.id.padEnd(idWidth), r.name].join(" "), + ); + return [header, separator, ...body].join("\n"); +} + +export default class ConversationsFind extends Command { + static description = + "Fuzzy-find conversations by name (lexical + embedding). Usage: conversations find [--max N]"; + static flags = { + port: Flags.integer({ + description: "Port for type agent server", + default: AGENT_SERVER_DEFAULT_PORT, + }), + max: Flags.integer({ + description: "Maximum number of matches to return", + default: 10, + }), + }; + static args = { + query: Args.string({ + description: "Name (or approximate name) to search for", + required: true, + }), + }; + + async run(): Promise { + const { args, flags } = await this.parse(ConversationsFind); + const url = `ws://localhost:${flags.port}`; + const connection = await connectAgentServer(url); + try { + const matches = await connection.findConversations( + args.query, + flags.max, + ); + this.log(formatTable(matches)); + } finally { + await connection.close(); + } + } +} diff --git a/ts/packages/cli/src/conversationCommands.ts b/ts/packages/cli/src/conversationCommands.ts index d4f08c2424..0bd83af0ef 100644 --- a/ts/packages/cli/src/conversationCommands.ts +++ b/ts/packages/cli/src/conversationCommands.ts @@ -127,6 +127,13 @@ function parseSlashCommand(args: string): ParsedCommand { : { subcommand: "list" }, }; } + case "find": { + const query = parseNameArg(subArgs); + if (!query) { + return { ok: false, usage: "@conversation find " }; + } + return { ok: true, payload: { subcommand: "find", query } }; + } case "info": return { ok: true, payload: { subcommand: "info" } }; case "prev": @@ -168,7 +175,7 @@ function parseSlashCommand(args: string): ParsedCommand { default: return { ok: false, - usage: `Unknown subcommand '${sub}'. Available: new, switch, list, info, rename, delete`, + usage: `Unknown subcommand '${sub}'. Available: new, switch, list, find, info, rename, delete`, }; } } @@ -245,6 +252,27 @@ function renderResult(result: ConversationActionResult): void { console.log(""); break; } + case "matches": { + // eslint-disable-next-line no-console + console.log( + chalk.bold(`\nMatches for '${chalk.green(result.query)}':`), + ); + const currentId = result.currentConversationId; + for (const m of result.matches) { + const c = m.conversation; + const isCurrent = c.conversationId === currentId; + const marker = isCurrent ? "\u25b8 " : " "; + const pct = chalk.dim(`(${Math.round(m.score * 100)}%)`); + const line = `${marker}${c.name} ${pct}${ + isCurrent ? " (current)" : "" + }`; + // eslint-disable-next-line no-console + console.log(isCurrent ? chalk.green(line) : line); + } + // eslint-disable-next-line no-console + console.log(""); + break; + } } } diff --git a/ts/packages/defaultAgentProvider/test/data/translate-mcpfs-e2e.json b/ts/packages/defaultAgentProvider/test/data/translate-mcpfs-e2e.json index 113262d79d..9125d87427 100644 --- a/ts/packages/defaultAgentProvider/test/data/translate-mcpfs-e2e.json +++ b/ts/packages/defaultAgentProvider/test/data/translate-mcpfs-e2e.json @@ -47,7 +47,7 @@ } }, { - "request": "can you move that file to /data/world.txt.", + "request": "move that file to /data/world.txt.", "expected": { "anyof": [ { diff --git a/ts/packages/dispatcher/dispatcher/src/context/appAgentManager.ts b/ts/packages/dispatcher/dispatcher/src/context/appAgentManager.ts index 8255170739..9fa481613f 100644 --- a/ts/packages/dispatcher/dispatcher/src/context/appAgentManager.ts +++ b/ts/packages/dispatcher/dispatcher/src/context/appAgentManager.ts @@ -46,6 +46,9 @@ import { compileGrammarToNFA, enrichGrammarWithCheckedVariables, loadGrammarRulesNoThrow, + findMatchedRule, + type GrammarSourceMap, + type MatchedGrammarRule, } from "@typeagent/action-grammar"; import fs from "node:fs"; import { FlowDefinition } from "../execute/flowInterpreter.js"; @@ -147,14 +150,25 @@ export const alwaysEnabledAgents = { commands: ["system"], }; -function loadGrammar(actionConfig: ActionConfig): Grammar | undefined { +function loadGrammar( + actionConfig: ActionConfig, +): { grammar: Grammar; sourceMap?: GrammarSourceMap | undefined } | undefined { const grammarContent = getGrammarContent(actionConfig); if (grammarContent === undefined) { return undefined; } if (grammarContent.format === "ag") { - return grammarFromJson(JSON.parse(grammarContent.content)); + const grammar = grammarFromJson(JSON.parse(grammarContent.content)); + let sourceMap: GrammarSourceMap | undefined; + if (grammarContent.sourceMap !== undefined) { + try { + sourceMap = JSON.parse(grammarContent.sourceMap); + } catch { + // Malformed side-car: skip rule-source recovery for this schema. + } + } + return { grammar, sourceMap }; } if (grammarContent.format === "agr") { // Parse raw .agr at load time; throw on errors so bad syntax fails loudly. @@ -169,7 +183,7 @@ function loadGrammar(actionConfig: ActionConfig): Grammar | undefined { `Failed to parse static grammar for ${actionConfig.schemaName}: ${errors.join(", ")}`, ); } - return grammar ?? undefined; + return grammar ? { grammar } : undefined; } throw new Error( `Unsupported grammar format '${(grammarContent as { format: string }).format}' for ${actionConfig.schemaName}`, @@ -186,6 +200,13 @@ export class AppAgentManager implements ActionConfigProvider { // (build once per agent version, reference from each manager) to avoid the // redundant rebuild on connect and on `@package update`. private readonly agents = new Map(); + // Raw compiled grammar + source-map side-car per schema (only for "ag" + // grammars that shipped a `.ag.map.json`). Used to recover the matched + // rule's `.agr` source text for the explained popover. + private readonly grammarSourceMaps = new Map< + string, + { grammar: Grammar; sourceMap: GrammarSourceMap } + >(); private readonly actionConfigs = new Map(); private readonly loadingSchemas = new Set(); private readonly flowRegistry = new Map(); @@ -907,7 +928,17 @@ export class AppAgentManager implements ActionConfigProvider { let g: Grammar | undefined = undefined; try { - g = loadGrammar(config); + const loaded = loadGrammar(config); + g = loaded?.grammar; + if (loaded?.grammar && loaded.sourceMap) { + // Keep the raw compiled grammar + side-car (partIds + // align) so the explain path can recover matched-rule + // source text, independent of the merged NFA grammar. + this.grammarSourceMaps.set(schemaName, { + grammar: loaded.grammar, + sourceMap: loaded.sourceMap, + }); + } } catch (e) { // Grammar file doesn't exist or failed to load debugError( @@ -1199,6 +1230,30 @@ export class AppAgentManager implements ActionConfigProvider { return config; } + /** + * Recover the grammar rule that matched `request` for `schemaName` (its + * `.agr` source text plus the request colored by category), using the + * compiled grammar's source-map side-car. Returns undefined when the schema + * shipped no side-car, the request doesn't match the static grammar, or the + * rule can't be pinpointed. + */ + public findMatchedGrammarRule( + schemaName: string, + request: string, + actionName?: string, + ): MatchedGrammarRule | undefined { + const entry = this.grammarSourceMaps.get(schemaName); + if (entry === undefined) { + return undefined; + } + return findMatchedRule( + entry.sourceMap, + entry.grammar, + request, + actionName, + ); + } + public getSchemaNames() { return Array.from(this.actionConfigs.keys()); } @@ -1510,7 +1565,7 @@ export class AppAgentManager implements ActionConfigProvider { if (!dynamicGrammar || dynamicGrammar.alternatives.length === 0) return; const config = this.actionConfigs.get(schemaName); - const staticGrammar = config ? loadGrammar(config) : undefined; + const staticGrammar = config ? loadGrammar(config)?.grammar : undefined; const merged: Grammar = { alternatives: [ diff --git a/ts/packages/dispatcher/dispatcher/src/context/commandHandlerContext.ts b/ts/packages/dispatcher/dispatcher/src/context/commandHandlerContext.ts index 0e61e1abf5..83ab546472 100644 --- a/ts/packages/dispatcher/dispatcher/src/context/commandHandlerContext.ts +++ b/ts/packages/dispatcher/dispatcher/src/context/commandHandlerContext.ts @@ -192,6 +192,19 @@ export type CopilotImporter = ( onProgress?: (progress: CopilotImportProgress) => void, ) => Promise; +/** + * Host-provided sink that mirrors each conversation turn (user request or + * assistant result) into a cross-conversation content index, tagged by the + * host's conversation id. Injected per-conversation by the agent-server; + * undefined for standalone hosts. Called independently of the knowledge- + * extraction flags (which connected mode disables), so the unified index still + * populates there. + */ +export type ConversationContentSink = ( + text: string, + sender: "user" | "assistant", +) => void; + // A request-scoped route chosen by the registry-first contextSelector tier // (§11.4) when the topical winner is a neighborhood sibling with no cache // MatchResult. Unlike `collisionOneShotPicks` (durable, cross-turn, explicit @@ -249,6 +262,12 @@ export type CommandHandlerContext = { * mode). */ readonly copilotImport?: CopilotImporter | undefined; + /** + * Host-provided sink mirroring each turn into the cross-conversation + * content index (see {@link ConversationContentSink}). Undefined for hosts + * without a unified index. + */ + readonly conversationContentSink?: ConversationContentSink | undefined; // Per activation configs developerMode?: boolean; // When true, each translated request is confirmed via the client @@ -486,6 +505,13 @@ export type DispatcherOptions = DeepPartialUndefined & { * agent-server; omitted by hosts without a ConversationManager. */ copilotImport?: CopilotImporter | undefined; + + /** + * Sink mirroring each turn into the host's cross-conversation content + * index (see {@link ConversationContentSink}). Injected per-conversation + * by the agent-server; omitted by standalone hosts. + */ + conversationContentSink?: ConversationContentSink | undefined; }; async function getSession( @@ -1089,6 +1115,7 @@ export async function initializeCommandHandlerContext( clientIO, getConversationList: options?.getConversationList, copilotImport: options?.copilotImport, + conversationContentSink: options?.conversationContentSink, // Runtime context commandLock: createLimiter(1), // Make sure we process one command at a time. diff --git a/ts/packages/dispatcher/dispatcher/src/context/dispatcher/handlers/requestCommandHandler.ts b/ts/packages/dispatcher/dispatcher/src/context/dispatcher/handlers/requestCommandHandler.ts index 6488d25041..fddbc17417 100644 --- a/ts/packages/dispatcher/dispatcher/src/context/dispatcher/handlers/requestCommandHandler.ts +++ b/ts/packages/dispatcher/dispatcher/src/context/dispatcher/handlers/requestCommandHandler.ts @@ -16,6 +16,10 @@ import type { ExplainedMapping, ExplainedSegment, } from "@typeagent/dispatcher-types"; +import { + loadGrammarRulesNoThrow, + matchGrammar, +} from "@typeagent/action-grammar"; import { type CommandHandlerContext, @@ -379,6 +383,56 @@ function buildSegments(explanationData: unknown): ExplainedSegment[] { return segments; } +// Recover the grammar rule that matched a grammar cache hit. The live matcher +// (NFA/DFA) doesn't report which rule matched, so look up the persisted rules +// for the matched action: when there's one it is the answer; when there are +// several, re-run the request against each to disambiguate. Falls back to the +// compiled grammar's source-map side-car for statically shipped rules (which +// aren't in the persisted store), which also yields colored phrase segments. +function findMatchedGrammarRule( + context: CommandHandlerContext, + requestAction: RequestAction, +): { rule?: string; segments?: ExplainedSegment[] } { + const primary = toFullActions(requestAction.actions)[0]; + if (primary === undefined) return {}; + + const store = context.persistedGrammarStore; + const candidates = + store + ?.getRulesForSchema(primary.schemaName) + .filter((rule) => rule.actionName === primary.actionName) ?? []; + if (candidates.length === 1) { + return { rule: candidates[0].grammarText }; + } + if (candidates.length > 1) { + const request = requestAction.request; + for (const rule of candidates) { + const errors: string[] = []; + const grammar = loadGrammarRulesNoThrow( + primary.schemaName, + rule.grammarText, + errors, + ); + if ( + grammar !== undefined && + matchGrammar(grammar, request).length > 0 + ) { + return { rule: rule.grammarText }; + } + } + // Couldn't pinpoint one (e.g. cross-rule references) — show the first. + return { rule: candidates[0].grammarText }; + } + + // No dynamically-learned rule: recover a statically shipped rule via the + // compiled grammar's source-map side-car. + const matched = context.agents.findMatchedGrammarRule( + primary.schemaName, + requestAction.request, + primary.actionName, + ); + return matched ? { rule: matched.text, segments: matched.segments } : {}; +} // Derive example same-meaning rephrasings from a V5 explanation, each broken // into per-category segments (so the client can color them like the phrase): // substitute each non-property sub-phrase's synonyms, and add the explainer's @@ -630,7 +684,6 @@ async function tryErrorReasoningFallback( return false; } } - async function requestExplain( context: CommandHandlerContext, attachments: CachedImageWithDetails[] | undefined, @@ -719,13 +772,22 @@ async function requestExplain( } if (fromCache && !fromUser) { // If it is from cache, and not from the user, explanation is not necessary. + // Recover the matched rule: the matched construction for construction + // hits, or (grammar mode) the grammar rule + colored phrase segments + // re-matched on demand. + let rule: string | undefined; + let segments: ExplainedSegment[] | undefined; + if (fromCache === "grammar") { + ({ rule, segments } = findMatchedGrammarRule( + context, + requestAction, + )); + } else { + rule = translationResult.ruleText; + } notifyExplained( undefined, - buildExplainedDetail( - fromCache, - requestAction, - translationResult.ruleText, - ), + buildExplainedDetail(fromCache, requestAction, rule, segments), ); return; } diff --git a/ts/packages/dispatcher/dispatcher/src/context/memory.ts b/ts/packages/dispatcher/dispatcher/src/context/memory.ts index b2492f302f..5e4dde02f8 100644 --- a/ts/packages/dispatcher/dispatcher/src/context/memory.ts +++ b/ts/packages/dispatcher/dispatcher/src/context/memory.ts @@ -105,6 +105,10 @@ export function addUserMessageToHistory( cachedAttachments?: CachedImageWithDetails[], ): void { context.chatHistory.addUserEntry(request, cachedAttachments); + // Mirror the user turn into the host's cross-conversation content index. + // Fires regardless of the knowledge-extraction flags (which connected mode + // disables), so the unified index still populates there. + context.conversationContentSink?.(request, "user"); } // Queue the user's turn for knowledge extraction into conversation memory. @@ -146,6 +150,10 @@ export function addResultToMemory( action, ); + // Mirror the assistant turn into the host's cross-conversation content + // index (ungated by knowledge extraction, like the user turn). + context.conversationContentSink?.(message, "assistant"); + if (context.actionResultKnowledgeExtraction) { if (context.conversationManager && entities) { const newEntities = entities.filter( diff --git a/ts/packages/dispatcher/dispatcher/src/context/system/action/conversationActionHandler.ts b/ts/packages/dispatcher/dispatcher/src/context/system/action/conversationActionHandler.ts index adf2dd8a15..6857153344 100644 --- a/ts/packages/dispatcher/dispatcher/src/context/system/action/conversationActionHandler.ts +++ b/ts/packages/dispatcher/dispatcher/src/context/system/action/conversationActionHandler.ts @@ -9,8 +9,16 @@ import { ActionContext, TypeAgentAction } from "@typeagent/agent-sdk"; // Quote a conversation name so the command parser keeps it as a single // argument; names often contain spaces (and, rarely, quotes). function quoteName(name: string): string { - const escaped = name.replace(/\\/g, "\\\\").replace(/"/g, '\\"'); - return `"${escaped}"`; + if (!name.includes('"')) { + return `"${name}"`; + } + if (!name.includes("'")) { + return `'${name}'`; + } + + // If the name contains both quote types, fall back to escaping for a + // double-quoted token. + return `"${name.replace(/\\/g, "\\\\").replace(/"/g, '\\"')}"`; } // Each conversation action runs its equivalent `@conversation` command, which @@ -42,6 +50,9 @@ export async function executeConversationAction( case "listConversation": command = "@conversation list"; break; + case "findConversation": + command = `@conversation find ${action.parameters.query}`; + break; case "showConversationInfo": command = "@conversation info"; break; diff --git a/ts/packages/dispatcher/dispatcher/src/context/system/handlers/conversationCommandHandlers.ts b/ts/packages/dispatcher/dispatcher/src/context/system/handlers/conversationCommandHandlers.ts index e929af6323..d5601b6d78 100644 --- a/ts/packages/dispatcher/dispatcher/src/context/system/handlers/conversationCommandHandlers.ts +++ b/ts/packages/dispatcher/dispatcher/src/context/system/handlers/conversationCommandHandlers.ts @@ -221,6 +221,29 @@ class ConversationDeleteCommandHandler implements CommandHandler { } } +class ConversationFindCommandHandler implements CommandHandler { + public readonly description = + "Fuzzy-find conversations by name (lexical + embedding)"; + public readonly action = "findConversation"; + public readonly parameters = { + args: { + query: { + description: "Name (or approximate name) to search for", + implicitQuotes: true, + }, + }, + } as const; + public async run( + context: ActionContext, + params: ParsedCommandParams, + ) { + dispatchManageConversation(context, { + subcommand: "find", + query: params.args.query, + }); + } +} + class ConversationHelpCommandHandler implements CommandHandlerNoParams { public readonly description = "Show conversation command help"; public async run(context: ActionContext) { @@ -235,6 +258,7 @@ export function getConversationCommandHandlers(): CommandHandlerTable { commands: { new: new ConversationNewCommandHandler(), list: new ConversationListCommandHandler(), + find: new ConversationFindCommandHandler(), info: new ConversationInfoCommandHandler(), switch: new ConversationSwitchCommandHandler(), prev: new ConversationPrevCommandHandler(), diff --git a/ts/packages/dispatcher/dispatcher/src/context/system/manageConversationPayload.ts b/ts/packages/dispatcher/dispatcher/src/context/system/manageConversationPayload.ts index 74d816587e..aeb5f7cc09 100644 --- a/ts/packages/dispatcher/dispatcher/src/context/system/manageConversationPayload.ts +++ b/ts/packages/dispatcher/dispatcher/src/context/system/manageConversationPayload.ts @@ -17,7 +17,12 @@ export type ManageConversationPayload = { | "next" | "rename" | "delete" + | "find" | "help"; name?: string; newName?: string; + /** Search term for the `find` subcommand. */ + query?: string; + /** Optional cap on `find` results. */ + maxMatches?: number; }; diff --git a/ts/packages/dispatcher/dispatcher/src/context/system/schema/conversationActionSchema.agr b/ts/packages/dispatcher/dispatcher/src/context/system/schema/conversationActionSchema.agr index b6e258ee89..5e1574b00a 100644 --- a/ts/packages/dispatcher/dispatcher/src/context/system/schema/conversationActionSchema.agr +++ b/ts/packages/dispatcher/dispatcher/src/context/system/schema/conversationActionSchema.agr @@ -26,6 +26,7 @@ = (conversation | chat | session | tab); = (conversations | chats | sessions | tabs); + = ( | ); // "create/start/make/open/begin a new conversation called/named X" = (create | start | make | open | begin) (a | an)? new (called | named | titled) $(name:wildcard) -> { @@ -68,7 +69,7 @@ // "switch conversation to X" was being matched as a browser webflow. = // "switch/go/change/open/load/resume to (the) conversation (called|named) X" - (switch | go | change | open | load | resume) to (the)? (called | named)? $(name:wildcard) -> { + (switch | go | change | open | load | resume) to (the)? (called | named | about | on | regarding | discussing | for)? $(name:wildcard) -> { actionName: "switchConversation", parameters: { name } } @@ -83,7 +84,7 @@ parameters: { name } } // "open/load/resume (the) conversation (called|named) X" - | (open | load | resume) (the)? (called | named)? $(name:wildcard) -> { + | (open | load | resume) (the)? (called | named | about | on | regarding | discussing | for)? $(name:wildcard) -> { actionName: "switchConversation", parameters: { name } }; @@ -132,9 +133,22 @@ parameters: { name } }; +// "find/search/locate (for)? (the|a|my)? conversation(s) (about|on|...)? X" +// "which/what conversation(s) (was|were)? (about|on|...) X" + = + (find | search | locate | look for) (me)? (the | a | my | our | any)? (about | on | regarding | with | containing | named | called | mentioning | discussing | for)? $(query:wildcard) -> { + actionName: "findConversation", + parameters: { query } + } + | (which | what) (was | were | is | are | did we)? (about | on | regarding | discussing | mentioned | covered | for) $(query:wildcard) -> { + actionName: "findConversation", + parameters: { query } + }; + = | | + | | | | diff --git a/ts/packages/dispatcher/dispatcher/src/context/system/schema/conversationActionSchema.ts b/ts/packages/dispatcher/dispatcher/src/context/system/schema/conversationActionSchema.ts index 3959ffe9a2..66d9970e28 100644 --- a/ts/packages/dispatcher/dispatcher/src/context/system/schema/conversationActionSchema.ts +++ b/ts/packages/dispatcher/dispatcher/src/context/system/schema/conversationActionSchema.ts @@ -4,6 +4,7 @@ export type ConversationAction = | NewConversationAction | ListConversationAction + | FindConversationAction | ShowConversationInfoAction | SwitchConversationAction | NextConversationAction @@ -33,6 +34,21 @@ export type ListConversationAction = { actionName: "listConversation"; }; +// Find existing conversations by approximate name or topic, WITHOUT switching. +// Use this when the user wants to find, search for, or locate a conversation by +// what it was about or roughly what it was named. +// Examples: "find the conversation about the workout playlist", "search my +// conversations for taxes", "which conversation was about the trip to Paris", +// "locate the chat where we discussed the budget". +// IMPORTANT: use switchConversation instead when the user wants to switch to it. +export type FindConversationAction = { + actionName: "findConversation"; + parameters: { + // The name or topic to search for + query: string; + }; +}; + // Show information about the current conversation. // Use this when the user asks about the current conversation info. // Examples: "show conversation info", "what conversation am I in", "current conversation info". diff --git a/ts/packages/dispatcher/dispatcher/src/translation/actionConfig.ts b/ts/packages/dispatcher/dispatcher/src/translation/actionConfig.ts index 25f5967ce2..6d6e10ab23 100644 --- a/ts/packages/dispatcher/dispatcher/src/translation/actionConfig.ts +++ b/ts/packages/dispatcher/dispatcher/src/translation/actionConfig.ts @@ -57,7 +57,15 @@ function loadGrammarFile(grammarFile: string): GrammarContent { const fullPath = getPackageFilePath(grammarFile); const content = fs.readFileSync(fullPath, "utf-8"); if (grammarFile.endsWith(".ag.json")) { - return { format: "ag", content }; + // Load the sibling source-map side-car if the build emitted one. + const mapPath = `${fullPath.slice(0, -".ag.json".length)}.ag.map.json`; + let sourceMap: string | undefined; + try { + sourceMap = fs.readFileSync(mapPath, "utf-8"); + } catch { + // Side-car is optional. + } + return { format: "ag", content, sourceMap }; } if (grammarFile.endsWith(".agr")) { // Raw grammar source; parsed at load time instead of via a build step. diff --git a/ts/packages/dispatcher/dispatcher/test/conversationActionHandler.spec.ts b/ts/packages/dispatcher/dispatcher/test/conversationActionHandler.spec.ts index cbe67a54bf..9a35d5d72f 100644 --- a/ts/packages/dispatcher/dispatcher/test/conversationActionHandler.spec.ts +++ b/ts/packages/dispatcher/dispatcher/test/conversationActionHandler.spec.ts @@ -70,12 +70,20 @@ describe("executeConversationAction delegates to @conversation commands", () => expectCommand('@conversation new "my work project"'); }); - it("escapes embedded quotes in conversation names", async () => { + it("uses single-quoted tokens when names contain double quotes", async () => { await run({ actionName: "newConversation", parameters: { name: 'fix "bug"' }, }); - expectCommand('@conversation new "fix \\"bug\\""'); + expectCommand(`@conversation new 'fix "bug"'`); + }); + + it("escapes when names contain both quote types", async () => { + await run({ + actionName: "newConversation", + parameters: { name: `Sam's "playlist"` }, + }); + expectCommand(`@conversation new "Sam's \\"playlist\\""`); }); it("listConversation runs list", async () => { diff --git a/ts/packages/dispatcher/dispatcher/test/conversationGrammar.spec.ts b/ts/packages/dispatcher/dispatcher/test/conversationGrammar.spec.ts index b569b8345c..cd4e3d78c8 100644 --- a/ts/packages/dispatcher/dispatcher/test/conversationGrammar.spec.ts +++ b/ts/packages/dispatcher/dispatcher/test/conversationGrammar.spec.ts @@ -98,6 +98,13 @@ describe("system.conversation grammar", () => { expect(r.parameters.name).toBe("research"); }); + it("matches 'switch to the conversation about X' (find-then-switch)", () => { + const r = match("switch to the conversation about gym music"); + expect(r).toBeDefined(); + expect(r.actionName).toBe("switchConversation"); + expect(r.parameters.name).toBe("gym music"); + }); + it("does NOT match bare 'switch to X' (no conversation anchor)", () => { // This avoids stealing matches from agents like browser/player that // legitimately use 'switch to X' for their own domain (e.g. @@ -194,4 +201,30 @@ describe("system.conversation grammar", () => { expect(r.actionName).toBe("showConversationInfo"); }); }); + + describe("findConversation", () => { + it.each([ + ["find the conversation about taxes", "taxes"], + ["search my conversations for taxes", "taxes"], + [ + "locate the chat about the workout playlist", + "the workout playlist", + ], + [ + "which conversation was about the trip to paris", + "the trip to paris", + ], + ["find conversation gym music", "gym music"], + ])("matches %j", (input, query) => { + const r = match(input); + expect(r).toBeDefined(); + expect(r.actionName).toBe("findConversation"); + expect(r.parameters.query).toBe(query); + }); + + it("does NOT hijack 'what conversation am i in'", () => { + const r = match("what conversation am i in"); + expect(r?.actionName).toBe("showConversationInfo"); + }); + }); }); diff --git a/ts/packages/shell/src/renderer/src/chatPanelBridge.ts b/ts/packages/shell/src/renderer/src/chatPanelBridge.ts index b364706885..05a6adeb19 100644 --- a/ts/packages/shell/src/renderer/src/chatPanelBridge.ts +++ b/ts/packages/shell/src/renderer/src/chatPanelBridge.ts @@ -1475,6 +1475,21 @@ export function createChatPanelClient( chatPanel.setDemoPaused(state === "paused"); }, reconnectStatusChanged(status: ConnectionStatus | undefined): void { + // Retract the "stale-build" notice when the connection drops + // (status defined), not when it returns. The notice describes the + // server we were connected to, so a stuck "Restarting..." toast + // must not outlive that link. Clearing on disconnect keeps the + // reconnected server's join-time push authoritative: a fresh + // successor stays silent (nothing to show) and a still-stale one + // re-pushes the notice, which arrives after this retract and + // renders correctly. Doing it here rather than on reconnect avoids + // racing that push - the Electron shell reuses its connection + // across reconnects (unlike vscode-shell's fresh connect), so the + // server's join-time retract can be dropped, and this makes the + // clear independent of it. Idempotent once the notice is gone. + if (status !== undefined) { + chatPanel.clearStatusNotice("stale-build"); + } chatPanel.setConnectionStatus(status, (action) => { // Manual recovery from the "stopped" banner — main owns the // retry / server-start logic. diff --git a/ts/pnpm-lock.yaml b/ts/pnpm-lock.yaml index d1baf5335e..1404dce90f 100644 --- a/ts/pnpm-lock.yaml +++ b/ts/pnpm-lock.yaml @@ -1661,6 +1661,9 @@ importers: '@typeagent/agent-rpc': specifier: workspace:* version: link:../../agentRpc + '@typeagent/agent-runtime': + specifier: workspace:* + version: link:../../typeagent '@typeagent/agent-sdk': specifier: workspace:* version: link:../../agentSdk @@ -1670,12 +1673,18 @@ importers: '@typeagent/agent-server-protocol': specifier: workspace:* version: link:../protocol + '@typeagent/aiclient': + specifier: workspace:* + version: link:../../aiclient '@typeagent/common-utils': specifier: workspace:* version: link:../../utils/commonUtils '@typeagent/config': specifier: workspace:* version: link:../../config + '@typeagent/conversation-memory': + specifier: workspace:* + version: link:../../memory/conversation '@typeagent/dispatcher-rpc': specifier: workspace:* version: link:../../dispatcher/rpc