diff --git a/CLAUDE.md b/CLAUDE.md index df689cd5..c98d7b5d 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -51,7 +51,7 @@ Bumping the version requires editing `releaseVersionName` and `defaultConfig.ver - `ANTHROPIC_MESSAGES` → `AnthropicMessagesProtocol` - `LOCAL_GGUF` → `LocalGgufProtocol` -Messages flow via `cn.lineai.ai.message.*` (`SystemModelMessage`, `UserModelMessage`, `AssistantModelMessage`, `ToolModelMessage`). `ModelClient` is the high-level entry point used by `MainCoordinator`. Tool-call text inside a stream is parsed by `ToolCallTextParser`; reasoning by `ThinkTagParser`. System prompts are assembled by `cn.lineai.ai.prompt.SystemPromptProvider` from `app/src/main/assets/prompts/*.txt` templates (`system-prompt-template.txt`, tone variants, `context-compaction-template.txt`, `memory-extraction-template.txt`, `skill-extraction-template.txt`, `work-directory-template.txt`, `learning-context-template.txt`, `model-identity-template.txt`) — modify templates there, not by hardcoding in Java. `SystemPromptProvider.build(..., ModelConfig model)` renders the model identity block (modelId / name / provider / protocol) from the currently selected model and injects it as `{{MODEL_IDENTITY}}` so the assistant can answer capability questions against the actual model id. +Messages flow via `cn.lineai.ai.message.*` (`SystemModelMessage`, `UserModelMessage`, `AssistantModelMessage`, `ToolModelMessage`). `ModelClient` is the high-level entry point used by `MainCoordinator`. Tool-call text inside a stream is parsed by `ToolCallTextParser`; reasoning by `ThinkTagParser`. System prompts are assembled by `cn.lineai.ai.prompt.SystemPromptProvider` from `app/src/main/assets/prompts/*.txt` templates (`system-prompt-template.txt`, tone variants, `context-compaction-template.txt`, `work-directory-template.txt`, `learning-context-template.txt`, `model-identity-template.txt`) — modify templates there, not by hardcoding in Java. `SystemPromptProvider.build(..., ModelConfig model)` renders the model identity block (modelId / name / provider / protocol) from the currently selected model and injects it as `{{MODEL_IDENTITY}}` so the assistant can answer capability questions against the actual model id. ### Model config & context size `ModelConfig` (`:core-model`) carries `contextSize` (tokens) as a first-class field, parsed by `ContextSizeParser` from user input like `128K`, `1m`, `128000`. UI uses a plain text field (`R.string.model_field_context_size`); the legacy `{id}[{size}]` suffix in `modelId` is still tolerated for old DB rows via `ModelContextParser.parse(ModelConfig)` / `apiModelId(ModelConfig)`. Protocols must call `apiModelId(ModelConfig)` rather than reading `modelId` directly, so the `[size]` suffix is stripped when needed. The `model_configs` SQLite table has a `context_size` column; `ModelRepository.ensureModelConfigColumns` auto-`ALTER`s old databases. @@ -68,7 +68,7 @@ Messages flow via `cn.lineai.ai.message.*` (`SystemModelMessage`, `UserModelMess - Compaction calls are dedicated model requests using the codex-style handoff-summary template (`assets/prompts/context-compaction-template.txt` + `context-compaction-summary-prefix.txt`), with retry + backoff (`MAX_COMPACT_RETRIES`) and usage recorded into `TokenUsageTracker` so the post-compact context becomes the new baseline. - Transcript is built segment-by-segment (`List`, each ≤ `TRANSCRIPT_SEGMENT_MAX_CHARS = 256KB`) to avoid single-`StringBuilder` OOM. **Do not reintroduce per-message or total-length truncation** — earlier `MAX_MESSAGE_CONTENT_CHARS` / `MAX_TRANSCRIPT_CHARS` limits were removed as harmful. `ContextCompactionController.startSoftContextCompaction` / `finishSoftContextCompaction` drive the soft flow; `ChatInteractionController` checks the soft trigger before each request when the hard trigger has not fired and the switch is enabled. `OutOfMemoryError` is caught in both the service and the controller as a last-resort fallback. `TokenUsageTracker` is reset on conversation switch (`ContextCompactionController.onConversationChanged`). - `AndroidManifest.xml` sets `android:largeHeap="true"` to support large transcripts. -- `MemoryExtractionService` and `LearningContextRepository` feed durable knowledge back into the system prompt. +- Durable knowledge (memories saved via the `memory_update` tool or the memory management screen, plus conversation/skill indexes) is reinjected into the system prompt by `LearningContextRepository` / `LearningContextService` when Learning Mode is enabled. ### Image attachments in chat `ComposerView` exposes a dedicated image button (right of the attach `+`) that opens the system image picker (`ACTION_OPEN_DOCUMENT`, `image/*`). Selected images are compressed (long edge ≤ 1568px, JPEG q=85, >3.5MB further downgraded) and base64-encoded. Send path goes through `ChatInteractionController.sendMessageWithImage` → `ImageInputPayload.rawInputJson(prompt, mimeType, base64)` → `ChatMessage.responseInputItemJson`. Protocols detect `ImageInputPayload.KIND` in `message.getRawInputJson()` and emit the correct shape per provider (OpenAI `image_url`, Anthropic `image.source.base64`, Codex `input_image`). No protocol changes are needed when adding new image entry points — reuse `ImageInputPayload`. @@ -144,7 +144,7 @@ The chat list renders Markdown through `:markdown` (`cn.lineai.ui.markdown.*`), - 所有用户可见字符串必须使用 R.string.* 资源，禁止在 Java 代码中硬编码中文或英文字符串 - 英文资源在 values/strings.xml，中文资源在 values-zh/strings.xml，俄文资源在 values-ru/strings.xml - 警惕 \uXXXX 转义的中文字符，这些绕过 grep 搜索但不违反规则 -- AI 提示词中的中文关键词（如 MemoryExtractionService 的记忆匹配词、SkillFileManager 的 Skill 模板）不属于用户可见字符串，不需要 i18n +- AI 提示词中的中文关键词（如 SkillFileManager 的 Skill 模板）不属于用户可见字符串，不需要 i18n - Repository 层通过 ResourceProvider.getString() 获取字符串资源，不直接使用 Context - Agent 提示词 fallback 通过 Context.getString() 读取资源，无 Context 时回退到硬编码英文 diff --git a/README.md b/README.md index 8825218b..d53cf04b 100644 --- a/README.md +++ b/README.md @@ -50,8 +50,8 @@ The application id is `cn.lineai` and the project is a multi-module Gradle proje - Streaming chat with **multiple model protocols** in the same UI: OpenAI-compatible HTTP APIs, Anthropic Messages, OpenAI Codex Responses, and a local GGUF runtime. - Reasoning blocks (`…`) are extracted and rendered separately from the final answer. - Tool-call text inside a stream is parsed and dispatched by `ToolCallTextParser`; everything the model asks to do is shown to you before it runs. -- System prompts are assembled from `feature-model/src/main/assets/prompts/*.txt` — tone variants (chat / coding), context-compaction, memory-extraction, skill-extraction, work-directory, learning-context, and model-identity templates. You can override the tone, the work directory, the identity block, and the prompt template from settings. -- Long conversations are summarised in the background by `ContextCompactionService` with **dynamic compaction** (50% soft trigger + 80% hard trigger) using the active model itself; durable knowledge is extracted by `MemoryExtractionService` and reinjected next session by `LearningContextRepository`. +- System prompts are assembled from `feature-model/src/main/assets/prompts/*.txt` — tone variants (chat / coding), context-compaction, work-directory, learning-context, and model-identity templates. You can override the tone, the work directory, the identity block, and the prompt template from settings. +- Long conversations are summarised in the background by `ContextCompactionService` with **dynamic compaction** (50% soft trigger + 80% hard trigger) using the active model itself; durable knowledge saved via the `memory_update` tool or the memory screen is reinjected next session by `LearningContextRepository`. - **Image attachments** — pick an image from the system picker; it is compressed, base64-encoded, and sent in the protocol-specific format (OpenAI `image_url`, Anthropic `image.source.base64`, Codex `input_image`). ### Tool execution diff --git a/README_CN.md b/README_CN.md index a4003867..8a115e9c 100644 --- a/README_CN.md +++ b/README_CN.md @@ -50,8 +50,8 @@ LineCode 不是一个轻量聊天客户端，它是一个**完整的编程工作 - 同一个聊天界面下支持**多种模型协议**：OpenAI 兼容 HTTP API、Anthropic Messages、OpenAI Codex Responses、本地 GGUF 推理。 - 推理块（`…`）会被 `ThinkTagParser` 单独抽出来，和最终回答分块渲染。 - 流中的工具调用文本由 `ToolCallTextParser` 解析并派发；模型请求做的每件事，在真正执行前都会先展示给你看。 -- 系统提示由 `feature-model/src/main/assets/prompts/*.txt` 中的模板拼装：语气（聊天 / 编程）变体、上下文压缩、记忆抽取、技能抽取、工作目录、学习上下文、模型身份。你可以在设置里覆盖语气、工作目录、身份块、提示模板。 -- 长对话由 `ContextCompactionService` 后台用**当前模型本身**做**动态压缩**（50% 软触发 + 80% 硬触发）；`MemoryExtractionService` 抽取长期知识，`LearningContextRepository` 在下一次会话中喂回上下文。 +- 系统提示由 `feature-model/src/main/assets/prompts/*.txt` 中的模板拼装：语气（聊天 / 编程）变体、上下文压缩、工作目录、学习上下文、模型身份。你可以在设置里覆盖语气、工作目录、身份块、提示模板。 +- 长对话由 `ContextCompactionService` 后台用**当前模型本身**做**动态压缩**（50% 软触发 + 80% 硬触发）；通过 `memory_update` 工具或记忆管理界面保存的长期知识，由 `LearningContextRepository` 在下一次会话中喂回上下文。 - **图片附件** - 在输入框右侧点图片按钮打开系统选择器；选中的图片经压缩后 base64 编码，按协议格式输出（OpenAI `image_url`、Anthropic `image.source.base64`、Codex `input_image`）。 ### 工具执行 diff --git a/app/build.gradle.kts b/app/build.gradle.kts index 5c3b77a2..fb887469 100644 --- a/app/build.gradle.kts +++ b/app/build.gradle.kts @@ -5,7 +5,7 @@ plugins { alias(libs.plugins.android.application) } -val releaseVersionName = "1.2.6-rc.2" +val releaseVersionName = "1.2.6" val releaseApkName = "LineCode Pro $releaseVersionName.APK" val releaseIdsigName = "$releaseApkName.idsig" val releaseSigningProperties = Properties() @@ -103,7 +103,7 @@ android { applicationId = "cn.lineai" minSdk = 26 targetSdk = 37 - versionCode = 30 + versionCode = 31 versionName = releaseVersionName } diff --git a/app/src/main/java/cn/lineai/context/MemoryExtractionService.java b/app/src/main/java/cn/lineai/context/MemoryExtractionService.java deleted file mode 100644 index 8b729610..00000000 --- a/app/src/main/java/cn/lineai/context/MemoryExtractionService.java +++ /dev/null @@ -1,587 +0,0 @@ -package cn.lineai.context; - -import cn.lineai.R; -import cn.lineai.ai.ModelClient; -import cn.lineai.ai.ModelCompletionResponse; -import cn.lineai.ai.message.ModelMessage; -import cn.lineai.ai.message.SystemModelMessage; -import cn.lineai.ai.message.UserModelMessage; -import cn.lineai.ai.prompt.StringTemplate; -import cn.lineai.data.repository.ExtensionStore; -import cn.lineai.data.repository.LearningContextStore; -import cn.lineai.data.repository.PromptTemplateRepository; -import cn.lineai.model.MemoryOverviewState; -import cn.lineai.model.ModelConfig; -import cn.lineai.resource.ResourceProvider; -import java.util.ArrayList; -import java.util.HashMap; -import java.util.LinkedHashMap; -import java.util.List; -import java.util.Locale; -import org.json.JSONArray; -import org.json.JSONObject; - -public final class MemoryExtractionService { - private static final int MAX_TRANSCRIPT_CHARS = 6000; - private static final int MAX_MEMORY_CHARS = 320; - private static final int MAX_MEMORIES = 3; - private static final int MAX_SKILLS = 2; - private static final int MAX_SKILL_CONTENT_CHARS = 8000; - private static final double MIN_KEEP_CONFIDENCE = 0.78; - private static final double RULE_CONFIDENCE = 0.88; - private static final double MODEL_DEFAULT_CONFIDENCE = 0.82; - - private final ResourceProvider resourceProvider; - private final LearningContextStore repository; - private final ExtensionStore extensionRepository; - private final PromptTemplateRepository promptTemplateRepository; - private final ModelClient modelClient = new ModelClient(); - - public MemoryExtractionService(ResourceProvider resourceProvider, LearningContextStore repository, ExtensionStore extensionRepository, PromptTemplateRepository promptTemplateRepository) { - this.resourceProvider = resourceProvider; - this.repository = repository; - this.extensionRepository = extensionRepository; - this.promptTemplateRepository = promptTemplateRepository; - } - - public void extractAndStore(ModelConfig selectedModel, String projectId, String userInput, String transcript) { - if (!hasDurableSignal(userInput, transcript)) { - extractAndStoreSkills(selectedModel, projectId, userInput, transcript); - return; - } - ArrayList candidates = new ArrayList<>(); - boolean modelAttempted = false; - if (selectedModel != null) { - modelAttempted = true; - try { - candidates.addAll(extractWithModel(selectedModel, projectId, userInput, transcript)); - } catch (Exception ignored) { - } - } - // Rules only fill gaps: skip when the model already returned durable candidates. - if (candidates.isEmpty()) { - candidates.addAll(ruleBasedCandidates(userInput, transcript)); - } else if (!modelAttempted) { - candidates.addAll(ruleBasedCandidates(userInput, transcript)); - } - for (ExtractedMemory memory : dedupe(candidates)) { - repository.saveExtractedMemory(memory.scope, projectId, memory.content, memory.confidence); - } - extractAndStoreSkills(selectedModel, projectId, userInput, transcript); - } - - private List extractWithModel( - ModelConfig selectedModel, - String projectId, - String userInput, - String transcript - ) throws Exception { - HashMap values = new HashMap<>(); - values.put("PROJECT_ID", safe(projectId)); - values.put("USER_INPUT", trimForPrompt(userInput, 1200)); - values.put("TRANSCRIPT", trimForPrompt(transcript, MAX_TRANSCRIPT_CHARS)); - String prompt = template().render(values); - ArrayList messages = new ArrayList<>(); - messages.add(new SystemModelMessage(prompt)); - messages.add(new UserModelMessage(resourceProvider.getString(R.string.memory_extraction_json_only))); - ModelCompletionResponse response = modelClient.complete(selectedModel, messages); - return parseCandidates(response.getText(), userInput); - } - - private void extractAndStoreSkills(ModelConfig selectedModel, String projectId, String userInput, String transcript) { - if (selectedModel == null || !hasSkillSignal(userInput, transcript)) { - return; - } - try { - HashMap values = new HashMap<>(); - values.put("PROJECT_ID", safe(projectId)); - values.put("USER_INPUT", trimForPrompt(userInput, 1200)); - values.put("TRANSCRIPT", trimForPrompt(transcript, MAX_TRANSCRIPT_CHARS)); - ArrayList messages = new ArrayList<>(); - messages.add(new SystemModelMessage(skillTemplate().render(values))); - messages.add(new UserModelMessage(resourceProvider.getString(R.string.memory_extraction_json_only))); - ModelCompletionResponse response = modelClient.complete(selectedModel, messages); - for (ExtractedSkill skill : parseSkills(response.getText())) { - if (skillExists(projectId, skill.name)) { - continue; - } - extensionRepository.createSkill(projectId, skill.location, skill.name, skill.description, skill.content); - } - } catch (Exception ignored) { - } - } - - private List parseSkills(String rawText) { - ArrayList skills = new ArrayList<>(); - String json = extractJson(rawText); - if (json.length() == 0) { - return skills; - } - try { - JSONObject object = new JSONObject(json); - JSONArray array = object.optJSONArray("skills"); - if (array == null && json.startsWith("[")) { - array = new JSONArray(json); - } - if (array == null) { - return skills; - } - for (int i = 0; i < array.length() && skills.size() < MAX_SKILLS; i++) { - JSONObject item = array.optJSONObject(i); - if (item == null) { - continue; - } - String name = sanitizeSkillName(item.optString("name")); - String description = normalizeContent(item.optString("description")); - String content = item.optString("content").trim(); - String location = "app".equals(item.optString("location")) ? "app" : "project"; - if (name.length() == 0 || content.length() < 80 || content.length() > MAX_SKILL_CONTENT_CHARS) { - continue; - } - if (shouldKeep(content, 0.80)) { - skills.add(new ExtractedSkill(name, description, location, content)); - } - } - } catch (Exception ignored) { - } - return skills; - } - - private boolean skillExists(String projectId, String name) { - String target = normalizedKey(name); - if (target.length() == 0) { - return true; - } - for (cn.lineai.model.SkillRecord skill : extensionRepository.getSkills(projectId)) { - if (normalizedKey(skill.getName()).equals(target)) { - return true; - } - } - return false; - } - - private boolean hasSkillSignal(String userInput, String transcript) { - String text = (safe(userInput) + "\n" + safe(transcript)).toLowerCase(Locale.ROOT); - return containsAny(text, "skill", "skills", "沉淀", "复用", "长期", "流程", "规范", "以后", "下次", "自动创建"); - } - - private List parseCandidates(String rawText, String userInput) { - ArrayList candidates = new ArrayList<>(); - String json = extractJson(rawText); - if (json.length() == 0) { - return candidates; - } - try { - JSONArray array; - if (json.startsWith("[")) { - array = new JSONArray(json); - } else { - JSONObject object = new JSONObject(json); - array = object.optJSONArray("memories"); - if (array == null) { - array = new JSONArray(); - } - } - for (int i = 0; i < array.length() && candidates.size() < MAX_MEMORIES; i++) { - JSONObject item = array.optJSONObject(i); - if (item == null) { - continue; - } - String content = normalizeContent(item.optString("content")); - String scope = resolveScope(item.optString("scope"), content, userInput); - if (scope.length() == 0) { - continue; - } - double confidence = item.optDouble("confidence", MODEL_DEFAULT_CONFIDENCE); - addIfValid(candidates, scope, content, confidence); - } - } catch (Exception ignored) { - } - return candidates; - } - - private String extractJson(String rawText) { - String value = safe(rawText).trim(); - if (value.startsWith("```")) { - value = value.replaceFirst("(?is)^```(?:json)?\\s*", ""); - value = value.replaceFirst("(?is)\\s*```$", "").trim(); - } - int objectStart = value.indexOf('{'); - int objectEnd = value.lastIndexOf('}'); - int arrayStart = value.indexOf('['); - int arrayEnd = value.lastIndexOf(']'); - if (objectStart >= 0 && objectEnd > objectStart && (arrayStart < 0 || objectStart < arrayStart)) { - return value.substring(objectStart, objectEnd + 1); - } - if (arrayStart >= 0 && arrayEnd > arrayStart) { - return value.substring(arrayStart, arrayEnd + 1); - } - return ""; - } - - static List ruleBasedCandidates(String userInput, String transcript) { - ArrayList candidates = new ArrayList<>(); - // Prefer explicit user statements; transcript only fills when user input is empty. - String primary = safeStatic(userInput).trim().length() > 0 ? userInput : transcript; - String normalized = compactSpaces(primary); - String lower = normalized.toLowerCase(Locale.ROOT); - - if (containsAny(lower, "androidx", "android x") - && containsAny(normalized, "不用", "不要用", "不能用", "不能使用", "禁止使用", "不能依赖") - && hasProjectCue(normalized)) { - addIfValid(candidates, MemoryOverviewState.Memory.SCOPE_PROJECT, "当前项目不能使用 AndroidX。", 0.95); - } - - for (String sentence : splitSentences(normalized)) { - if (candidates.size() >= MAX_MEMORIES) { - break; - } - if (!hasStrongConstraintCue(sentence)) { - continue; - } - String scope = inferScope(sentence); - if (scope.length() == 0) { - continue; - } - String content = normalizeContent(sentence); - if (content.length() == 0 || content.contains("比如") || isEphemeralContent(content)) { - continue; - } - addIfValid(candidates, scope, content, RULE_CONFIDENCE); - } - return candidates; - } - - static boolean hasDurableSignal(String userInput, String transcript) { - String text = (safeStatic(userInput) + "\n" + safeStatic(transcript)).toLowerCase(Locale.ROOT); - if (text.trim().length() == 0) { - return false; - } - return containsAny( - text, - "记住", - "记一下", - "以后都", - "以后一律", - "长期", - "始终", - "永远", - "不要再", - "别再", - "禁止", - "必须", - "只能", - "不能用", - "不要用", - "不能使用", - "偏好", - "习惯", - "所有项目", - "全局", - "这个项目", - "当前项目", - "本项目", - "该项目", - "remember", - "always", - "never", - "prefer", - "from now on", - "don't use", - "do not use", - "must not", - "must always" - ); - } - - private static String inferScope(String sentence) { - if (hasProjectCue(sentence)) { - return MemoryOverviewState.Memory.SCOPE_PROJECT; - } - if (hasEnvironmentCue(sentence)) { - return MemoryOverviewState.Memory.SCOPE_ENVIRONMENT; - } - if (hasUserCue(sentence)) { - return MemoryOverviewState.Memory.SCOPE_USER; - } - return ""; - } - - private static boolean hasProjectCue(String text) { - return containsAny(text, "这个项目", "当前项目", "本项目", "该项目", "这个 app", "当前 app", "工作区", "代码库", "仓库"); - } - - private static boolean hasStrongConstraintCue(String text) { - return containsAny( - text, - "不能", - "不要", - "禁止", - "必须", - "只能", - "以后都", - "以后一律", - "始终", - "永远", - "偏好", - "习惯", - "记住", - "记一下", - "always", - "never", - "must", - "prefer", - "remember" - ); - } - - private static boolean hasEnvironmentCue(String text) { - return containsAny(text, "我的手机", "当前设备", "本机", "Termux", "JDK", "镜像源", "adb", "环境变量"); - } - - private static boolean hasUserCue(String text) { - return containsAny(text, "我喜欢", "我偏好", "我的习惯", "以后都", "以后一律", "所有项目", "全局", "不要问我", "用中文回答", "始终用中文"); - } - - private static List splitSentences(String text) { - ArrayList sentences = new ArrayList<>(); - String[] parts = safeStatic(text).split("[。！？!?\\n]+"); - for (String part : parts) { - String value = compactSpaces(part); - if (value.length() >= 8) { - sentences.add(value); - } - } - return sentences; - } - - private static void addIfValid(List candidates, String scope, String content, double confidence) { - String normalizedScope = normalizeScope(scope); - String normalizedContent = normalizeContent(content); - if (!shouldKeep(normalizedContent, confidence)) { - return; - } - candidates.add(new ExtractedMemory(normalizedScope, normalizedContent, confidence)); - } - - private List dedupe(List candidates) { - LinkedHashMap unique = new LinkedHashMap<>(); - for (ExtractedMemory candidate : candidates) { - if (candidate == null || !shouldKeep(candidate.content, candidate.confidence)) { - continue; - } - String key = candidate.scope + ":" + normalizedKey(candidate.content); - if (!unique.containsKey(key)) { - unique.put(key, candidate); - } - if (unique.size() >= MAX_MEMORIES) { - break; - } - } - return new ArrayList<>(unique.values()); - } - - private static boolean shouldKeep(String content, double confidence) { - String value = safeStatic(content).trim(); - if (confidence < MIN_KEEP_CONFIDENCE || value.length() < 10 || value.length() > MAX_MEMORY_CHARS) { - return false; - } - if (isEphemeralContent(value)) { - return false; - } - String lower = value.toLowerCase(Locale.ROOT); - if (containsAny(lower, "api key", "apikey", "token", "password", "passwd", "secret", "cookie", "私钥", "密码", "密钥", "sk-")) { - return false; - } - return true; - } - - private static boolean isEphemeralContent(String content) { - String value = safeStatic(content); - String lower = value.toLowerCase(Locale.ROOT); - return containsAny( - value, - "本轮任务", - "这次任务", - "刚才的输出", - "报错日志", - "安装到手机", - "继续写", - "下一步干嘛", - "先实现", - "先修复", - "刚才说", - "当前这轮", - "这轮对话", - "临时", - "试一下", - "试试看" - ) || containsAny( - lower, - "this turn", - "for now", - "right now", - "stacktrace", - "traceback", - "todo:", - "next step" - ); - } - - private static String resolveScope(String requestedScope, String content, String userInput) { - String safeContent = safeStatic(content); - if (hasProjectCue(safeContent)) { - return MemoryOverviewState.Memory.SCOPE_PROJECT; - } - if (hasEnvironmentCue(safeContent)) { - return MemoryOverviewState.Memory.SCOPE_ENVIRONMENT; - } - if (hasUserCue(safeContent)) { - return MemoryOverviewState.Memory.SCOPE_USER; - } - String normalizedRequest = normalizeScopeOrEmpty(requestedScope); - if (normalizedRequest.length() > 0) { - return normalizedRequest; - } - String lowerContent = safeContent.toLowerCase(Locale.ROOT); - if (hasProjectCue(userInput) && containsAny(lowerContent, "androidx", "android x")) { - return MemoryOverviewState.Memory.SCOPE_PROJECT; - } - // Unknown scope is discarded for auto-extract (do not default to user). - return ""; - } - - private static String normalizeScope(String scope) { - String value = normalizeScopeOrEmpty(scope); - return value.length() == 0 ? MemoryOverviewState.Memory.SCOPE_USER : value; - } - - private static String normalizeScopeOrEmpty(String scope) { - String value = safeStatic(scope).trim().toLowerCase(Locale.ROOT); - if (MemoryOverviewState.Memory.SCOPE_PROJECT.equals(value)) { - return MemoryOverviewState.Memory.SCOPE_PROJECT; - } - if (MemoryOverviewState.Memory.SCOPE_ENVIRONMENT.equals(value)) { - return MemoryOverviewState.Memory.SCOPE_ENVIRONMENT; - } - if (MemoryOverviewState.Memory.SCOPE_USER.equals(value)) { - return MemoryOverviewState.Memory.SCOPE_USER; - } - return ""; - } - - private static String normalizeContent(String content) { - String value = compactSpaces(content); - while (value.startsWith("-") || value.startsWith("，") || value.startsWith(",") || value.startsWith(":") || value.startsWith("：")) { - value = value.substring(1).trim(); - } - if (value.length() > MAX_MEMORY_CHARS) { - value = value.substring(0, MAX_MEMORY_CHARS - 1).trim() + "。"; - } - return value; - } - - private static String compactSpaces(String text) { - String value = safeStatic(text).replace('\r', ' ').replace('\n', ' ').trim(); - while (value.contains(" ")) { - value = value.replace(" ", " "); - } - return value; - } - - private static boolean containsAny(String text, String... needles) { - String value = safeStatic(text); - for (String needle : needles) { - if (value.contains(needle)) { - return true; - } - } - return false; - } - - private static String normalizedKey(String content) { - String value = safeStatic(content).toLowerCase(Locale.ROOT); - StringBuilder builder = new StringBuilder(); - for (int i = 0; i < value.length(); i++) { - char ch = value.charAt(i); - if (Character.isLetterOrDigit(ch) || (ch >= '\u4e00' && ch <= '\u9fff')) { - builder.append(ch); - } - } - return builder.toString(); - } - - private String trimForPrompt(String value, int maxChars) { - String text = safe(value); - if (text.length() <= maxChars) { - return text; - } - return text.substring(0, maxChars - 3) + "..."; - } - - private StringTemplate template() { - return new StringTemplate(promptTemplateRepository.getTemplateText(PromptTemplateRepository.ID_MEMORY_EXTRACTION)); - } - - private StringTemplate skillTemplate() { - return new StringTemplate(promptTemplateRepository.getTemplateText(PromptTemplateRepository.ID_SKILL_EXTRACTION)); - } - - private String sanitizeSkillName(String name) { - String value = safe(name).trim().toLowerCase(Locale.ROOT); - StringBuilder builder = new StringBuilder(); - for (int i = 0; i < value.length() && builder.length() < 64; i++) { - char ch = value.charAt(i); - if ((ch >= 'a' && ch <= 'z') || (ch >= '0' && ch <= '9') || ch == '-' || ch == '_') { - builder.append(ch); - } else if (Character.isWhitespace(ch)) { - builder.append('-'); - } - } - String clean = builder.toString(); - while (clean.contains("--")) { - clean = clean.replace("--", "-"); - } - while (clean.startsWith("-")) { - clean = clean.substring(1); - } - while (clean.endsWith("-")) { - clean = clean.substring(0, clean.length() - 1); - } - return clean; - } - - private String safe(String value) { - return value == null ? "" : value; - } - - private static String safeStatic(String value) { - return value == null ? "" : value; - } - - static final class ExtractedMemory { - final String scope; - final String content; - final double confidence; - - ExtractedMemory(String scope, String content, double confidence) { - this.scope = normalizeScope(scope); - this.content = normalizeContent(content); - this.confidence = confidence; - } - } - - private static final class ExtractedSkill { - final String name; - final String description; - final String location; - final String content; - - ExtractedSkill(String name, String description, String location, String content) { - this.name = name == null ? "" : name; - this.description = description == null ? "" : description; - this.location = "app".equals(location) ? "app" : "project"; - this.content = content == null ? "" : content; - } - } -} diff --git a/app/src/main/java/cn/lineai/data/repository/ToolSettingsRepository.java b/app/src/main/java/cn/lineai/data/repository/ToolSettingsRepository.java index 3acf3c41..f9cc2add 100644 --- a/app/src/main/java/cn/lineai/data/repository/ToolSettingsRepository.java +++ b/app/src/main/java/cn/lineai/data/repository/ToolSettingsRepository.java @@ -110,6 +110,12 @@ private List buildDefaultConfigs() { true, new String[] {ToolNames.WEB_SEARCH, ToolNames.WEB_FETCH}, MODE_ALL, "web_search")); + configs.add(new McpToolConfig("memory", + resourceProvider.getString(R.string.tool_group_memory_name), + resourceProvider.getString(R.string.tool_group_memory_desc), + true, + new String[] {ToolNames.MEMORY_UPDATE}, + MODE_ALL, "memory")); return configs; } @@ -281,9 +287,17 @@ public synchronized Set getEnabledToolNames() { } } } + boolean learningMode = settingsRepository.getBoolean(AiBehaviorSettingsRepository.KEY_LEARNING_MODE, false); + applyLearningModeGate(enabled, learningMode); return enabled; } + static void applyLearningModeGate(Set enabled, boolean learningModeEnabled) { + if (!learningModeEnabled && enabled != null) { + enabled.remove(ToolNames.MEMORY_UPDATE); + } + } + @Override public synchronized Set getEnabledToolNames(Collection implementedTools) { HashSet enabled = new HashSet<>(getEnabledToolNames()); diff --git a/app/src/main/java/cn/lineai/data/service/SkillFileManager.java b/app/src/main/java/cn/lineai/data/service/SkillFileManager.java index 25948145..d6adf840 100644 --- a/app/src/main/java/cn/lineai/data/service/SkillFileManager.java +++ b/app/src/main/java/cn/lineai/data/service/SkillFileManager.java @@ -366,7 +366,7 @@ public String buildSkillMarkdown(String name, String description, String content public String readUtf8(File file, int maxChars) { try { - String text = readStream(new FileInputStream(file)); + String text = readStream(new FileInputStream(file), (long) maxChars + 4); return text.length() <= maxChars ? text : text.substring(0, maxChars); } catch (Exception ignored) { return ""; @@ -390,16 +390,19 @@ public void writeUtf8(File file, String content) { } } - private String readStream(InputStream input) throws Exception { + private String readStream(InputStream input, long maxBytes) throws Exception { if (input == null) { return ""; } try { ByteArrayOutputStream output = new ByteArrayOutputStream(); byte[] buffer = new byte[8192]; + long total = 0; int read; - while ((read = input.read(buffer)) != -1) { - output.write(buffer, 0, read); + while (total < maxBytes && (read = input.read(buffer)) != -1) { + int toWrite = (int) Math.min(read, maxBytes - total); + output.write(buffer, 0, toWrite); + total += toWrite; } return output.toString(StandardCharsets.UTF_8.name()); } finally { diff --git a/app/src/main/java/cn/lineai/mvp/ConversationPersistenceController.java b/app/src/main/java/cn/lineai/mvp/ConversationPersistenceController.java index f2e9db76..d14db032 100644 --- a/app/src/main/java/cn/lineai/mvp/ConversationPersistenceController.java +++ b/app/src/main/java/cn/lineai/mvp/ConversationPersistenceController.java @@ -10,6 +10,10 @@ import cn.lineai.model.ChatMessage; import cn.lineai.model.InputAttachment; import java.util.ArrayList; +import java.util.concurrent.CountDownLatch; +import java.util.concurrent.ExecutorService; +import java.util.concurrent.Executors; +import java.util.concurrent.TimeUnit; import org.json.JSONArray; import org.json.JSONObject; @@ -29,6 +33,21 @@ interface Host { private final AiBehaviorSettingsRepository aiBehaviorSettingsRepository; private final LearningContextStore learningContextStore; private final Host host; + /** + * 会话持久化走进程级共享的单线程后台执行器，避免在主线程热路径（每次收发/流式步骤/工具批次后）同步写库造成 ANR。 + * 采用 latest-wins 合并：若已有一次在途写入，则仅更新待写快照，始终持久化最新状态，避免任务堆积。 + * + *

静态共享而非实例字段：控制器若因 Activity 重建被重新创建，不会重复创建线程导致泄漏； + * daemon 线程不阻塞进程退出，无需显式 shutdown。 + */ + private static final ExecutorService PERSIST_EXECUTOR = Executors.newSingleThreadExecutor(runnable -> { + Thread thread = new Thread(runnable, "linecode-conversation-persist"); + thread.setDaemon(true); + return thread; + }); + private final Object persistLock = new Object(); + private boolean persistScheduled; + private ConversationRecord persistSnapshot; ConversationPersistenceController( Context context, @@ -65,6 +84,8 @@ void loadConversation(String id) { } void applyConversation(ConversationRecord conversation) { + // 加载/切换会话前先落库所有已排队的异步持久化，确保后续同步写入不会被过期异步写覆盖。 + awaitPendingPersist(); ConversationResumeSanitizer.Result result = ConversationResumeSanitizer.sanitize( conversation, host.interruptedGenerationMessage(context) @@ -120,9 +141,51 @@ void persistCurrentConversation() { "", records ); - conversationStore.saveConversation(conversation); - if (aiBehaviorSettingsRepository.get().isLearningModeEnabled()) { - learningContextStore.indexConversation(projectPath, conversation); + // 消息列表在这里已快照为不可变的 ConversationRecord，DB 写入在线程上异步执行， + // 不阻塞主线程热路径，也避免后续对 messages 的并发修改造成数据竞争。 + boolean learningEnabled = aiBehaviorSettingsRepository.get().isLearningModeEnabled(); + synchronized (persistLock) { + persistSnapshot = conversation; + if (persistScheduled) { + return; // 已有写入在途，latest-wins：保留最新快照即可 + } + persistScheduled = true; + } + PERSIST_EXECUTOR.execute(() -> runPersist(learningEnabled)); + } + + private void runPersist(boolean learningEnabled) { + ConversationRecord snapshot; + synchronized (persistLock) { + snapshot = persistSnapshot; + persistScheduled = false; + persistSnapshot = null; + } + if (snapshot == null) { + return; + } + conversationStore.saveConversation(snapshot); + if (learningEnabled) { + try { + learningContextStore.indexConversation(snapshot.getProjectId(), snapshot); + } catch (Exception ignored) { + } + } + } + + /** 阻塞等待所有已排队的持久化任务落库，用于加载/切换会话前避免过期异步写入覆盖新状态。 */ + private void awaitPendingPersist() { + final CountDownLatch latch = new CountDownLatch(1); + synchronized (persistLock) { + if (!persistScheduled) { + return; + } + PERSIST_EXECUTOR.execute(latch::countDown); + } + try { + latch.await(10L, TimeUnit.SECONDS); + } catch (InterruptedException e) { + Thread.currentThread().interrupt(); } } diff --git a/app/src/main/java/cn/lineai/mvp/ExtensionController.java b/app/src/main/java/cn/lineai/mvp/ExtensionController.java index 2134ed1f..2e5090dd 100644 --- a/app/src/main/java/cn/lineai/mvp/ExtensionController.java +++ b/app/src/main/java/cn/lineai/mvp/ExtensionController.java @@ -24,13 +24,13 @@ public interface ExtensionController { List onMcpToolsQuery(String url, List headers) throws Exception; - SkillRecord onSkillCreated(String location, String name, String description, String content); + void onSkillCreated(String location, String name, String description, String content); - SkillRecord onSkillInstalled(String location, String sourcePath, String name) throws Exception; + void onSkillInstalled(String location, String sourcePath, String name) throws Exception; - SkillRecord onSkillInstalledFromUri(String location, String uri, String displayName) throws Exception; + void onSkillInstalledFromUri(String location, String uri, String displayName) throws Exception; - SkillRecord onSkillInstalledFromGitHub(String location, String githubUrl) throws Exception; + void onSkillInstalledFromGitHub(String location, String githubUrl) throws Exception; void onExtensionEnabledChanged(String kind, String id, boolean enabled); diff --git a/app/src/main/java/cn/lineai/mvp/ExtensionManagementController.java b/app/src/main/java/cn/lineai/mvp/ExtensionManagementController.java index 3bec6913..b8d8ac72 100644 --- a/app/src/main/java/cn/lineai/mvp/ExtensionManagementController.java +++ b/app/src/main/java/cn/lineai/mvp/ExtensionManagementController.java @@ -21,22 +21,30 @@ interface Host { void refreshVisibleScreen(String screenId); void render(); + + void showSkillError(String message); } private final ExtensionStore extensionRepository; private final IpcProviderStore ipcProviderRepository; private final ToolRegistry toolRegistry; + private final BackgroundTaskRunner backgroundTasks; + private final MainThreadDispatcher mainThread; private final Host host; ExtensionManagementController( ExtensionStore extensionRepository, IpcProviderStore ipcProviderRepository, ToolRegistry toolRegistry, + BackgroundTaskRunner backgroundTasks, + MainThreadDispatcher mainThread, Host host ) { this.extensionRepository = extensionRepository; this.ipcProviderRepository = ipcProviderRepository; this.toolRegistry = toolRegistry; + this.backgroundTasks = backgroundTasks; + this.mainThread = mainThread; this.host = host; } @@ -68,32 +76,57 @@ List queryMcpTools(String url, List headers) t return extensionRepository.queryMcpTools(url, headers); } - SkillRecord createSkill(String location, String name, String description, String content) { - SkillRecord skill = extensionRepository.createSkill(host.projectPath(), location, name, description, content); - host.returnToScreen("extension:skills"); - host.render(); - return skill; + void createSkill(String location, String name, String description, String content) { + backgroundTasks.execute("skill-create", () -> { + try { + extensionRepository.createSkill(host.projectPath(), location, name, description, content); + mainThread.dispatch(this::completeSkillInstall); + } catch (Exception e) { + mainThread.dispatch(() -> host.showSkillError(errorMessage(e))); + } + }); } - SkillRecord installSkill(String location, String sourcePath, String name) throws Exception { - SkillRecord skill = extensionRepository.installSkill(host.projectPath(), location, sourcePath, name); - host.returnToScreen("extension:skills"); - host.render(); - return skill; + void installSkill(String location, String sourcePath, String name) { + backgroundTasks.execute("skill-install", () -> { + try { + extensionRepository.installSkill(host.projectPath(), location, sourcePath, name); + mainThread.dispatch(this::completeSkillInstall); + } catch (Exception e) { + mainThread.dispatch(() -> host.showSkillError(errorMessage(e))); + } + }); } - SkillRecord installSkillFromUri(String location, String uri, String displayName) throws Exception { - SkillRecord skill = extensionRepository.installSkillFromUri(host.projectPath(), location, uri, displayName); - host.returnToScreen("extension:skills"); - host.render(); - return skill; + void installSkillFromUri(String location, String uri, String displayName) { + backgroundTasks.execute("skill-install-from-uri", () -> { + try { + extensionRepository.installSkillFromUri(host.projectPath(), location, uri, displayName); + mainThread.dispatch(this::completeSkillInstall); + } catch (Exception e) { + mainThread.dispatch(() -> host.showSkillError(errorMessage(e))); + } + }); } - SkillRecord installSkillFromGitHub(String location, String githubUrl) throws Exception { - SkillRecord skill = extensionRepository.installSkillFromGitHub(host.projectPath(), location, githubUrl); + void installSkillFromGitHub(String location, String githubUrl) { + backgroundTasks.execute("skill-install-from-github", () -> { + try { + extensionRepository.installSkillFromGitHub(host.projectPath(), location, githubUrl); + mainThread.dispatch(this::completeSkillInstall); + } catch (Exception e) { + mainThread.dispatch(() -> host.showSkillError(errorMessage(e))); + } + }); + } + + private void completeSkillInstall() { host.returnToScreen("extension:skills"); host.render(); - return skill; + } + + private String errorMessage(Exception e) { + return e == null ? null : e.getMessage(); } void deleteExtensions(String kind, List ids) { diff --git a/app/src/main/java/cn/lineai/mvp/GenerationFlowController.java b/app/src/main/java/cn/lineai/mvp/GenerationFlowController.java index 2017bd89..2abe2751 100644 --- a/app/src/main/java/cn/lineai/mvp/GenerationFlowController.java +++ b/app/src/main/java/cn/lineai/mvp/GenerationFlowController.java @@ -357,6 +357,11 @@ void startInitialModelRequest( ModelCancellationToken cancellationToken, String userInput ) { + // 每次用户发起的新一轮生成开始时，清零 Agent 内部工具调用累计计数， + // 使全局工具上限只统计本轮实际执行的工具调用。 + if (agentExecutionController != null) { + agentExecutionController.resetExecutedAgentToolCalls(); + } if (cancellationToken != null && cancellationToken.isCancelled()) { return; } @@ -622,9 +627,9 @@ private void finishGeneration( messages.set(index, message.withContent(finalText, finalReasoning, false) .withToolCalls(toolCalls, false)); if (hasToolCalls) { - if (!generationController.canExecuteToolCalls(selectedModel, usedToolCallCount, toolCalls.size())) { + if (!generationController.canExecuteToolCalls(selectedModel, effectiveUsedToolCalls(usedToolCallCount), toolCalls.size())) { messages.add(new ChatMessage(host.nextId(), ChatMessage.Role.ASSISTANT, - generationController.toolLimitMessage(selectedModel, usedToolCallCount, toolCalls.size()), false)); + generationController.toolLimitMessage(selectedModel, effectiveUsedToolCalls(usedToolCallCount), toolCalls.size()), false)); for (ToolCall call : toolCalls) { addOrReplaceToolResult(ToolResult.of( call.getId(), @@ -644,7 +649,7 @@ private void finishGeneration( generationId, selectedModel, toolCalls, - usedToolCallCount + toolCalls.size(), + usedToolCallCount, cancellationToken ); return; @@ -706,6 +711,7 @@ private void handleToolExecutionBatch( ToolExecutionBatch batch ) { toolMessageController.addOrReplaceToolResults(batch.getCompletedResults()); + int executedCount = usedToolCallCount + batch.getCompletedResults().size(); if (batch.getPendingCall() != null) { ToolResult pendingResult = ToolResult.withReview( batch.getPendingCall().getId(), @@ -722,7 +728,7 @@ private void handleToolExecutionBatch( selectedModel, batch.getPendingCall(), batch.getRemainingCalls(), - usedToolCallCount, + executedCount, homePath, cancellationToken )); @@ -731,7 +737,7 @@ private void handleToolExecutionBatch( return; } host.persistCurrentConversation(); - continueModelAfterTools(generationId, selectedModel, usedToolCallCount, cancellationToken); + continueModelAfterTools(generationId, selectedModel, executedCount, cancellationToken); } private void continueModelAfterTools( @@ -769,6 +775,16 @@ void continueToolLoop( continueModelAfterTools(generationId, selectedModel, usedToolCallCount, cancellationToken); } + /** + * 全局工具预算的有效已用次数 = 主流程已执行的工具调用数 + Agent 内部已执行的工具调用数。 + * Agent 内部的工具调用通过 {@link AgentExecutionController#executedAgentToolCalls()} 累计， + * 主流程在上限判断处都应计入，否则重复发起 Agent 会重复占用相同份额导致全局上限被绕过。 + */ + private int effectiveUsedToolCalls(int mainFlowUsedToolCallCount) { + int agentCalls = agentExecutionController == null ? 0 : agentExecutionController.executedAgentToolCalls(); + return mainFlowUsedToolCallCount + agentCalls; + } + private ToolContext toolContext( String homePath, ModelConfig selectedModel, @@ -776,7 +792,7 @@ private ToolContext toolContext( int generationId, int usedToolCallCount ) { - final int[] counter = new int[]{Math.max(0, usedToolCallCount)}; + final int[] counter = new int[]{Math.max(0, effectiveUsedToolCalls(usedToolCallCount))}; return ToolContext.builder() .homePath(homePath) .extraWriteRoots(extensionRepository.skillWriteRoots(homePath)) @@ -850,11 +866,12 @@ private void executeAcceptedPendingTool(PendingToolExecution pending) { addOrReplaceToolResult(finalResult); host.persistCurrentConversation(); host.render(); + // 被确认的暂停工具刚刚实际执行过，续跑时计数 +1，使其占用对应的主流程工具预算。 continueToolExecution( pending.getGenerationId(), pending.getSelectedModel(), pending.getRemainingCalls(), - pending.getUsedToolCallCount(), + pending.getUsedToolCallCount() + 1, pending.getHomePath(), pending.getCancellationToken() ); diff --git a/app/src/main/java/cn/lineai/mvp/MainControllerInitializer.java b/app/src/main/java/cn/lineai/mvp/MainControllerInitializer.java index d8625665..4d15d26d 100644 --- a/app/src/main/java/cn/lineai/mvp/MainControllerInitializer.java +++ b/app/src/main/java/cn/lineai/mvp/MainControllerInitializer.java @@ -1,6 +1,7 @@ package cn.lineai.mvp; import android.content.Context; +import android.widget.Toast; import cn.lineai.R; import cn.lineai.ai.ModelCancellationToken; import cn.lineai.context.ContextCompactionService; @@ -369,6 +370,8 @@ public String interruptedGenerationMessage(Context ctx) { extensionRepository, ipcProviderRepository, toolRegistry, + backgroundTasks, + mainThread, new ExtensionManagementController.Host() { @Override public String projectPath() { @@ -389,6 +392,14 @@ public void refreshVisibleScreen(String screenId) { public void render() { coordinator.render(); } + + @Override + public void showSkillError(String message) { + String text = message == null || message.trim().length() == 0 + ? context.getString(R.string.skill_install_failed) + : message; + Toast.makeText(context, text, Toast.LENGTH_LONG).show(); + } } ); coordinator.modelPromptController = new ModelPromptController( diff --git a/app/src/main/java/cn/lineai/mvp/MainCoordinator.java b/app/src/main/java/cn/lineai/mvp/MainCoordinator.java index feb3b3ae..03ae9456 100644 --- a/app/src/main/java/cn/lineai/mvp/MainCoordinator.java +++ b/app/src/main/java/cn/lineai/mvp/MainCoordinator.java @@ -760,23 +760,23 @@ public List onMcpToolsQuery(String url, List h } @Override - public SkillRecord onSkillCreated(String location, String name, String description, String content) { - return extensionManagementController.createSkill(location, name, description, content); + public void onSkillCreated(String location, String name, String description, String content) { + extensionManagementController.createSkill(location, name, description, content); } @Override - public SkillRecord onSkillInstalled(String location, String sourcePath, String name) throws Exception { - return extensionManagementController.installSkill(location, sourcePath, name); + public void onSkillInstalled(String location, String sourcePath, String name) throws Exception { + extensionManagementController.installSkill(location, sourcePath, name); } @Override - public SkillRecord onSkillInstalledFromUri(String location, String uri, String displayName) throws Exception { - return extensionManagementController.installSkillFromUri(location, uri, displayName); + public void onSkillInstalledFromUri(String location, String uri, String displayName) throws Exception { + extensionManagementController.installSkillFromUri(location, uri, displayName); } @Override - public SkillRecord onSkillInstalledFromGitHub(String location, String githubUrl) throws Exception { - return extensionManagementController.installSkillFromGitHub(location, githubUrl); + public void onSkillInstalledFromGitHub(String location, String githubUrl) throws Exception { + extensionManagementController.installSkillFromGitHub(location, githubUrl); } @Override diff --git a/app/src/main/java/cn/lineai/mvp/MainDependencies.java b/app/src/main/java/cn/lineai/mvp/MainDependencies.java index be33a895..1f252f34 100644 --- a/app/src/main/java/cn/lineai/mvp/MainDependencies.java +++ b/app/src/main/java/cn/lineai/mvp/MainDependencies.java @@ -9,7 +9,6 @@ import cn.lineai.ai.protocol.OpenAiResponsesCompactionProtocol; import cn.lineai.context.ContextCompactionService; import cn.lineai.context.ContextManager; -import cn.lineai.context.MemoryExtractionService; import cn.lineai.context.TokenUsageTracker; import cn.lineai.data.db.LineCodeDatabase; import cn.lineai.data.importer.LineCodeArchiveService; @@ -92,7 +91,6 @@ public final class MainDependencies { final ProjectStore projectRepository; final LearningContextStore learningContextRepository; final LearningContextService learningContextService; - final MemoryExtractionService memoryExtractionService; final ToolSettingsStore toolSettingsRepository; final ExtensionStore extensionRepository; final IpcProviderStore ipcProviderRepository; @@ -179,7 +177,6 @@ public String buildExtensionPrompt(String skillName, String skillContent, String return sb.toString(); } }); - memoryExtractionService = new MemoryExtractionService(resourceProvider, learningContextRepository, extensionRepository, promptTemplateRepository); ipcProviderRepository = new IpcProviderRepository(database); ipcProviderScanner = new IpcProviderScanner(); ipcProviderManager = new IpcProviderManager(context); diff --git a/app/src/main/java/cn/lineai/mvp/ShareController.java b/app/src/main/java/cn/lineai/mvp/ShareController.java index 8bf90d53..d9462ade 100644 --- a/app/src/main/java/cn/lineai/mvp/ShareController.java +++ b/app/src/main/java/cn/lineai/mvp/ShareController.java @@ -2,6 +2,8 @@ import android.app.AlertDialog; import android.content.Context; +import android.os.Handler; +import android.os.Looper; import android.widget.Toast; import cn.lineai.R; import cn.lineai.model.ChatMessage; @@ -37,10 +39,14 @@ public void showFormatPicker(Context context, List selected) { .setTitle(R.string.dialog_export_format_title) .setItems(names, (dialog, which) -> { ExportFormat format = resolver.get(which); - ExportResult result = format.execute(context, selected); - if (result != null) { - handleResult(context, result, format); - } + new Thread(() -> { + ExportResult result = format.execute(context, selected); + new Handler(Looper.getMainLooper()).post(() -> { + if (result != null) { + handleResult(context, result, format); + } + }); + }, "linecode-share-export").start(); }) .show(); } diff --git a/app/src/main/java/cn/lineai/mvp/agent/AgentExecutionController.java b/app/src/main/java/cn/lineai/mvp/agent/AgentExecutionController.java index e325fdec..d5eef1c5 100644 --- a/app/src/main/java/cn/lineai/mvp/agent/AgentExecutionController.java +++ b/app/src/main/java/cn/lineai/mvp/agent/AgentExecutionController.java @@ -67,6 +67,7 @@ public final class AgentExecutionController { private final AgentPromptBuilder promptBuilder; private final PipelineDependencyResolver dependencyResolver; private final AgentResultRegistry agentResultRegistry = new AgentResultRegistry(); + private final AtomicInteger executedAgentToolCalls = new AtomicInteger(); private Context context; private ToolReviewAwaiter toolReviewAwaiter; private java.util.function.BooleanSupplier bypassPathProtectionSupplier = () -> false; @@ -75,6 +76,24 @@ public AgentResultRegistry getAgentResultRegistry() { return agentResultRegistry; } + /** + * 主流程工具预算应计入 Agent 内部执行过的工具调用次数（累计），以便全局工具上限被正确约束。 + * 该计数跨多次 Agent 调用累计，由主流程在每次生成开始时通过 {@link #resetExecutedAgentToolCalls()} 清零。 + */ + public int executedAgentToolCalls() { + return executedAgentToolCalls.get(); + } + + public void resetExecutedAgentToolCalls() { + executedAgentToolCalls.set(0); + } + + private void accumulateExecutedAgentToolCalls(int count) { + if (count > 0) { + executedAgentToolCalls.addAndGet(count); + } + } + public interface Host { String projectPath(); @@ -228,6 +247,7 @@ public ToolResult runAgentTool( customMcpIds, toolCallBudget ); + accumulateExecutedAgentToolCalls(asyncResult.getToolCallCount()); finishAgentWithCompact( agentId, toolCallId, AgentTool.NAME, progress, asyncResult, host); }); @@ -248,6 +268,7 @@ public ToolResult runAgentTool( customMcpIds, toolCallBudget ); + accumulateExecutedAgentToolCalls(result.getToolCallCount()); return finishAgentWithCompact(agentId, toolCallId, AgentTool.NAME, progress, result, host); } @@ -372,6 +393,7 @@ public ToolResult runAgentPipelineTool( AgentRunResult result = outcome.result; results.put(agent.getId(), result); totalToolCalls += result.getToolCallCount(); + accumulateExecutedAgentToolCalls(result.getToolCallCount()); hasError = hasError || result.isError(); summary.append("\n\n## ").append(agent.getId()).append(" · ").append(agent.getDescription()) .append('\n').append("类型: ").append(agent.getType()) diff --git a/app/src/main/java/cn/lineai/service/LineCodeAccessibilityService.java b/app/src/main/java/cn/lineai/service/LineCodeAccessibilityService.java index 8f6b204d..95703da3 100644 --- a/app/src/main/java/cn/lineai/service/LineCodeAccessibilityService.java +++ b/app/src/main/java/cn/lineai/service/LineCodeAccessibilityService.java @@ -205,7 +205,8 @@ private boolean clickFirstClickableNode(List nodes) { } boolean clicked = false; for (AccessibilityNodeInfo node : nodes) { - if (node.isClickable() && node.performAction(AccessibilityNodeInfo.ACTION_CLICK)) { + if (!clicked && node.isClickable() + && node.performAction(AccessibilityNodeInfo.ACTION_CLICK)) { clicked = true; } node.recycle(); diff --git a/app/src/main/java/cn/lineai/ui/MainChatView.java b/app/src/main/java/cn/lineai/ui/MainChatView.java index f541769a..37e04990 100644 --- a/app/src/main/java/cn/lineai/ui/MainChatView.java +++ b/app/src/main/java/cn/lineai/ui/MainChatView.java @@ -131,6 +131,7 @@ public interface DocumentCreateCallback { private String shellCommandText = ""; private String currentScreenId = ""; private final ScreenRegistry screenRegistry = new ScreenRegistry(); + private static final int SCREEN_CACHE_MAX = 12; private final LinkedHashMap screenCache = new LinkedHashMap<>(); private int screenAnimationGeneration; private boolean screenClosing; @@ -660,6 +661,16 @@ public void showScreen(String screenId, boolean forward, boolean animate) { nextView = buildScreen(currentScreenId); if (currentScreenId.length() > 0 && nextView != null) { screenCache.put(currentScreenId, nextView); + while (screenCache.size() > SCREEN_CACHE_MAX) { + java.util.Map.Entry eldest = + screenCache.entrySet().iterator().next(); + String eldestKey = eldest.getKey(); + View eldestView = eldest.getValue(); + screenCache.remove(eldestKey); + if (eldestView != null && eldestView.getParent() instanceof ViewGroup) { + ((ViewGroup) eldestView.getParent()).removeView(eldestView); + } + } } } if (nextView != null && nextView.getParent() == null) { diff --git a/app/src/main/java/cn/lineai/ui/component/AgentExtensionEditScreenView.java b/app/src/main/java/cn/lineai/ui/component/AgentExtensionEditScreenView.java index 320538e1..945d5d94 100644 --- a/app/src/main/java/cn/lineai/ui/component/AgentExtensionEditScreenView.java +++ b/app/src/main/java/cn/lineai/ui/component/AgentExtensionEditScreenView.java @@ -127,7 +127,7 @@ private void renderToolRows() { toolsSection.removeAllRows(); toolsSection.setTitle(getContext().getString(R.string.screen_agent_tools_count, getContext().getString(R.string.screen_agent_section_tools), - getContext().getString(R.string.screen_agent_let_ai_button), + getContext().getString(R.string.screen_agent_tools_selected), selectedTools.size())); if (availableTools.isEmpty()) { toolsSection.addRow(empty(getContext().getString(R.string.screen_agent_tools_empty)), false); @@ -149,7 +149,7 @@ private void renderMcpRows() { mcpSection.removeAllRows(); mcpSection.setTitle(getContext().getString(R.string.screen_agent_tools_count, getContext().getString(R.string.screen_agent_section_mcps), - getContext().getString(R.string.screen_agent_let_ai_button), + getContext().getString(R.string.screen_agent_tools_selected), selectedMcps.size())); ArrayList options = mcpOptions(); if (options.isEmpty()) { diff --git a/app/src/main/java/cn/lineai/ui/component/McpExtensionEditScreenView.java b/app/src/main/java/cn/lineai/ui/component/McpExtensionEditScreenView.java index 67e3ebcc..c0f48ca3 100644 --- a/app/src/main/java/cn/lineai/ui/component/McpExtensionEditScreenView.java +++ b/app/src/main/java/cn/lineai/ui/component/McpExtensionEditScreenView.java @@ -100,9 +100,10 @@ private void queryTools() { querying = true; renderQuery(); renderTools(); + List snapshot = headers(); new Thread(() -> { try { - List rawTools = listener.onQueryTools(url, headers()); + List rawTools = listener.onQueryTools(url, snapshot); final List tools = rawTools != null ? rawTools : java.util.Collections.emptyList(); mainHandler.post(() -> { querying = false; diff --git a/app/src/main/java/cn/lineai/ui/component/ModelAddScreenView.java b/app/src/main/java/cn/lineai/ui/component/ModelAddScreenView.java index 2dde07cc..b65df485 100644 --- a/app/src/main/java/cn/lineai/ui/component/ModelAddScreenView.java +++ b/app/src/main/java/cn/lineai/ui/component/ModelAddScreenView.java @@ -266,8 +266,8 @@ public void onStateChanged() { updateSaveState(); } }, - effectiveBaseUrl(), - ModelFormHelper.value(apiKeyInput) + this::effectiveBaseUrl, + () -> ModelFormHelper.value(apiKeyInput) ); content.addView(compressionSection, new LinearLayout.LayoutParams(LayoutParams.MATCH_PARENT, LayoutParams.WRAP_CONTENT)); } diff --git a/app/src/main/java/cn/lineai/ui/component/ModelCompressionSectionView.java b/app/src/main/java/cn/lineai/ui/component/ModelCompressionSectionView.java index 71d5673c..19b3e961 100644 --- a/app/src/main/java/cn/lineai/ui/component/ModelCompressionSectionView.java +++ b/app/src/main/java/cn/lineai/ui/component/ModelCompressionSectionView.java @@ -22,6 +22,10 @@ public interface CompressionListener { void onStateChanged(); } + public interface ValueSource { + String value(); + } + private Switch compressionEnabledSwitch; private Switch compressionAutoSwitch; private Switch compressionCustomIdSwitch; @@ -37,18 +41,18 @@ public interface CompressionListener { private boolean fetchingCompressionModels; private final ModelProtocolType[] protocolType; private final CompressionListener listener; - private final String effectiveBaseUrl; - private final String effectiveApiKey; + private final ValueSource baseUrlSource; + private final ValueSource apiKeySource; public ModelCompressionSectionView(Context context, ModelProtocolType[] protocolType, boolean enabled, boolean auto, String modelId, CompressionListener listener, - String effectiveBaseUrl, String effectiveApiKey) { + ValueSource baseUrlSource, ValueSource apiKeySource) { super(context); this.protocolType = protocolType; this.listener = listener; - this.effectiveBaseUrl = effectiveBaseUrl; - this.effectiveApiKey = effectiveApiKey; + this.baseUrlSource = baseUrlSource; + this.apiKeySource = apiKeySource; setOrientation(VERTICAL); LinearLayout enabledHeader = new LinearLayout(context); @@ -204,7 +208,7 @@ private void updateVisibility() { } private boolean canQuery() { - return isEnabled() && !isAuto() && effectiveBaseUrl.length() > 0 && effectiveApiKey.length() > 0; + return isEnabled() && !isAuto() && baseUrlSource.value().length() > 0 && apiKeySource.value().length() > 0; } private void renderModelIdInput(boolean custom) { @@ -292,7 +296,7 @@ private void fetchCatalog() { updateQueryState(); new Thread(() -> { try { - List rawIds = listener.onFetchCompressionCatalog(protocolType[0], effectiveBaseUrl, effectiveApiKey); + List rawIds = listener.onFetchCompressionCatalog(protocolType[0], baseUrlSource.value(), apiKeySource.value()); final List ids = rawIds != null ? rawIds : java.util.Collections.emptyList(); post(() -> { fetchingCompressionModels = false; diff --git a/app/src/main/java/cn/lineai/ui/component/StorageManagementScreenView.java b/app/src/main/java/cn/lineai/ui/component/StorageManagementScreenView.java index 0ff666b2..be97bf0a 100644 --- a/app/src/main/java/cn/lineai/ui/component/StorageManagementScreenView.java +++ b/app/src/main/java/cn/lineai/ui/component/StorageManagementScreenView.java @@ -12,6 +12,9 @@ import cn.lineai.R; import cn.lineai.model.StorageStatsUiModel; +import android.os.Handler; +import android.os.Looper; + public final class StorageManagementScreenView extends ScreenScaffoldView { public interface Listener { void onBack(); @@ -22,6 +25,8 @@ public interface Listener { private final Context context; private final Listener listener; + private final RefreshCwButtonView refreshButton; + private final Handler mainHandler = new Handler(Looper.getMainLooper()); private TextView totalSizeView; private TextView diffSizeView; private TextView diffCountView; @@ -33,9 +38,11 @@ public interface Listener { private TextView homeCountView; public StorageManagementScreenView(Context context, Listener listener) { - super(context, context.getString(R.string.screen_storage_title), listener::onBack, refreshButton(context, listener)); + super(context, context.getString(R.string.screen_storage_title), listener::onBack, createRefreshButton(context)); this.context = context; this.listener = listener; + this.refreshButton = (RefreshCwButtonView) getRightAction(); + this.refreshButton.setOnClickListener(v -> loadStats()); LinearLayout content = getContent(); LineTheme.padding(content, LineTheme.LG, LineTheme.LG, LineTheme.LG, 100); @@ -80,10 +87,8 @@ public StorageManagementScreenView(Context context, Listener listener) { loadStats(); } - private static View refreshButton(Context context, Listener listener) { - RefreshCwButtonView button = new RefreshCwButtonView(context, 18); - button.setOnClickListener(v -> listener.onBack()); - return button; + private static View createRefreshButton(Context context) { + return new RefreshCwButtonView(context, 18); } private LinearLayout createStorageRow(int iconType, String title, String desc) { @@ -135,8 +140,10 @@ private LinearLayout.LayoutParams createRowParams() { } private void loadStats() { - StorageStatsUiModel stats = listener.onLoadStats(); - updateViews(stats); + new Thread(() -> { + StorageStatsUiModel stats = listener.onLoadStats(); + mainHandler.post(() -> updateViews(stats)); + }, "linecode-storage-stats").start(); } private void updateViews(StorageStatsUiModel stats) { diff --git a/app/src/main/res/values-ru/strings.xml b/app/src/main/res/values-ru/strings.xml index 02b92746..c6e0132c 100644 --- a/app/src/main/res/values-ru/strings.xml +++ b/app/src/main/res/values-ru/strings.xml @@ -242,6 +242,7 @@ Неверный URL GitHub Не удалось скачать Skill с GitHub: %1$s В репозитории не найден SKILL.md + Не удалось установить SKILL Рабочая область LineCode Не удалось открыть рабочую папку @@ -594,6 +595,7 @@ Условие запуска Доступные инструменты Доступные MCP + выбрано Пусть AI напишет Agent Опишите, что должен делать Agent Опишите роль и поведение Agent @@ -1127,6 +1129,8 @@ Выполнение shell-команд через SSH Веб-поиск Поиск в интернете и просмотр веб-страниц + Память + Сохранение и управление долгосрочными воспоминаниями; требуется включённый режим обучения Выполнение shell-команд через терминальный провайдер IPC @@ -1166,9 +1170,6 @@ Чтение SSH файлов через SFTP… Чтение каталога терминального провайдера через IPC… - - Верните только JSON. - Agent завершён: Agent pipeline завершён: diff --git a/app/src/main/res/values-zh/strings.xml b/app/src/main/res/values-zh/strings.xml index b0275b8b..5c8ff546 100644 --- a/app/src/main/res/values-zh/strings.xml +++ b/app/src/main/res/values-zh/strings.xml @@ -242,6 +242,7 @@ GitHub 地址无效 从 GitHub 下载 Skill 失败: %1$s 仓库中未找到 SKILL.md + SKILL 安装失败 LineCode 工作区 无法打开工作区目录 @@ -596,6 +597,7 @@ 触发条件 可使用的工具 可使用的 MCP + 已选 让 AI 写 Agent 描述这个 Agent 应该做什么 描述 Agent 的角色与行为 @@ -1130,6 +1132,8 @@ 通过 SSH 执行 shell 命令 网页搜索 搜索互联网并查看网页内容 + 记忆 + 保存和管理长期记忆；需要开启学习模式 通过终端提供者 IPC 执行 shell 命令 @@ -1169,9 +1173,6 @@ 正在通过 SFTP 读取 SSH 文件... 正在通过 IPC 读取终端提供者目录... - - 请仅返回 JSON。 - Agent 完成: Agent 流水线完成: diff --git a/app/src/main/res/values/strings.xml b/app/src/main/res/values/strings.xml index e0e4b0e1..af1a60f4 100644 --- a/app/src/main/res/values/strings.xml +++ b/app/src/main/res/values/strings.xml @@ -242,6 +242,7 @@ Invalid GitHub URL Failed to download Skill from GitHub: %1$s No SKILL.md found in the repository + Failed to install SKILL LineCode workspace Unable to open workspace folder @@ -594,6 +595,7 @@ Trigger condition Available tools Available MCPs + selected Let AI write Agent Describe what the Agent should do Describe the agent\'s role and behavior @@ -1129,6 +1131,8 @@ Run shell commands via SSH Web search Search the internet and view web pages + Memory + Save and manage long-term memories; requires Learning Mode to be enabled Run shell commands via the terminal provider IPC @@ -1168,9 +1172,6 @@ Reading SSH files via SFTP… Reading terminal provider directory via IPC… - - Return JSON only. - Agent completed: Agent pipeline completed: diff --git a/app/src/test/java/cn/lineai/context/MemoryExtractionServiceTest.java b/app/src/test/java/cn/lineai/context/MemoryExtractionServiceTest.java deleted file mode 100644 index 800479e7..00000000 --- a/app/src/test/java/cn/lineai/context/MemoryExtractionServiceTest.java +++ /dev/null @@ -1,33 +0,0 @@ -package cn.lineai.context; - -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertTrue; - -import cn.lineai.model.MemoryOverviewState; -import java.util.List; -import org.junit.Test; - -public final class MemoryExtractionServiceTest { - @Test - public void projectAndroidXConstraintUsesProjectScope() { - List candidates = MemoryExtractionService.ruleBasedCandidates( - "做自动提取，另外区分作用域，比如这个项目不能用AndroidX，就保存到项目，而不是全局", - "" - ); - - assertTrue(candidates.size() > 0); - assertEquals(MemoryOverviewState.Memory.SCOPE_PROJECT, candidates.get(0).scope); - assertEquals("当前项目不能使用 AndroidX。", candidates.get(0).content); - } - - @Test - public void userPreferenceUsesUserScope() { - List candidates = MemoryExtractionService.ruleBasedCandidates( - "我偏好默认用中文，回答要简洁直接。", - "" - ); - - assertTrue(candidates.size() > 0); - assertEquals(MemoryOverviewState.Memory.SCOPE_USER, candidates.get(0).scope); - } -} diff --git a/app/src/test/java/cn/lineai/data/repository/MemoryRankerRagInjectionTest.java b/app/src/test/java/cn/lineai/data/repository/MemoryRankerRagInjectionTest.java new file mode 100644 index 00000000..0b32729d --- /dev/null +++ b/app/src/test/java/cn/lineai/data/repository/MemoryRankerRagInjectionTest.java @@ -0,0 +1,112 @@ +package cn.lineai.data.repository; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; + +import java.util.Arrays; +import java.util.List; +import org.junit.Test; + +/** + * 证明 RAG 记忆注入的排序行为：MemoryRanker.rank 会选择关键词命中的候选， + * 在无命中时通过 recency fallback 选择最近候选，并在关闭 fallback 时返回空列表。 + */ +public final class MemoryRankerRagInjectionTest { + + @Test + public void matchingMemoryCandidateIsRankedFirstWithPositiveRelevance() { + MemoryRanker.Candidate unrelated = new MemoryRanker.Candidate( + "u1", "天气预报和网页搜索配置。", System.currentTimeMillis(), ""); + MemoryRanker.Candidate match = new MemoryRanker.Candidate( + "m1", "当前项目不能使用 AndroidX，必须保持 Java 原生 View。", + System.currentTimeMillis(), ""); + + List result = MemoryRanker.rank( + Arrays.asList(unrelated, match), "项目 AndroidX", 10, false, 0.0); + + assertFalse(result.isEmpty()); + assertEquals("m1", result.get(0).id); + assertTrue(match.relevanceScore > 0.0); + } + + @Test + public void rankWithNoMatchesAndRecentFallbackReturnsRecentCandidates() { + long now = System.currentTimeMillis(); + MemoryRanker.Candidate recent = new MemoryRanker.Candidate( + "r1", "天气预报和网页搜索配置。", now - 60_000L, ""); + MemoryRanker.Candidate old = new MemoryRanker.Candidate( + "r2", "天气预报和网页搜索配置。", now - 90L * 86_400_000L, ""); + + List result = MemoryRanker.rank( + Arrays.asList(recent, old), "项目 AndroidX", 5, true, 0.0); + + assertFalse(result.isEmpty()); + assertEquals("r1", result.get(0).id); + } + + @Test + public void rankWithNoMatchesAndNoFallbackReturnsEmpty() { + long now = System.currentTimeMillis(); + MemoryRanker.Candidate recent = new MemoryRanker.Candidate( + "r1", "天气预报和网页搜索配置。", now - 60_000L, ""); + MemoryRanker.Candidate old = new MemoryRanker.Candidate( + "r2", "天气预报和网页搜索配置。", now - 90L * 86_400_000L, ""); + + List result = MemoryRanker.rank( + Arrays.asList(recent, old), "项目 AndroidX", 5, false, 0.0); + + assertTrue(result.isEmpty()); + } + + @Test + public void maxCountLimitsRankedCandidatesAndPreservesOrdering() { + long now = System.currentTimeMillis(); + MemoryRanker.Candidate oldUnrelated = new MemoryRanker.Candidate( + "c1", "很久以前的无关记忆：天气预报和网页搜索配置。", now - 1000L * 60 * 60, ""); + MemoryRanker.Candidate recentUnrelated = new MemoryRanker.Candidate( + "c2", "刚刚发生的无关记忆：午饭吃了什么。", now - 1000L * 60, ""); + MemoryRanker.Candidate highRelevanceOlder = new MemoryRanker.Candidate( + "c3", "当前项目不能使用 AndroidX，必须保持 Java 原生 View。", now - 1000L * 30, ""); + MemoryRanker.Candidate highRelevanceNewest = new MemoryRanker.Candidate( + "c4", "项目升级到 AndroidX 后，需要更新所有传统 View 的适配。", now, ""); + + List result = MemoryRanker.rank( + Arrays.asList(oldUnrelated, recentUnrelated, highRelevanceOlder, highRelevanceNewest), + "AndroidX 项目 View", + 2, + true, + 0.0); + + // 截断到 maxCount=2：无关候选被过滤，只保留两个相关候选（排序由相关度/新鲜度启发式决定）。 + assertEquals(2, result.size()); + for (MemoryRanker.Candidate candidate : result) { + assertTrue("unrelated candidate must be filtered: " + candidate.id, + "c3".equals(candidate.id) || "c4".equals(candidate.id)); + assertTrue(candidate.relevanceScore > 0.0); + } + } + + @Test + public void matchingRankFiltersOutIrrelevantCandidates() { + long now = System.currentTimeMillis(); + MemoryRanker.Candidate unrelated = new MemoryRanker.Candidate( + "u1", "天气预报和网页搜索配置，还有一些生活琐事。", now - 1000L * 60 * 60, ""); + MemoryRanker.Candidate recentUnrelated = new MemoryRanker.Candidate( + "u2", "刚刚发生的无关记忆：午饭吃了什么。", now - 1000L * 60, ""); + MemoryRanker.Candidate strongMatch = new MemoryRanker.Candidate( + "s1", "当前项目已经切换到 AndroidX，需要更新所有旧的 View 实现。", now, ""); + + List result = MemoryRanker.rank( + Arrays.asList(unrelated, recentUnrelated, strongMatch), + "AndroidX 项目 View 升级", + 10, + true, + 0.0); + + // 有匹配时，相关性为 0 的候选即使很新也会被过滤，只保留强匹配候选。 + assertEquals(1, result.size()); + assertEquals("s1", result.get(0).id); + assertTrue(result.get(0).relevanceScore > 0.0); + } +} diff --git a/app/src/test/java/cn/lineai/data/repository/PromptTemplateRepositoryTest.java b/app/src/test/java/cn/lineai/data/repository/PromptTemplateRepositoryTest.java index 6eb671e1..7241caea 100644 --- a/app/src/test/java/cn/lineai/data/repository/PromptTemplateRepositoryTest.java +++ b/app/src/test/java/cn/lineai/data/repository/PromptTemplateRepositoryTest.java @@ -13,8 +13,6 @@ public void templateIdsIncludeUserEditablePromptTemplates() { Assert.assertTrue(ids.contains(PromptTemplateRepository.ID_TONE_CODING)); Assert.assertTrue(ids.contains(PromptTemplateRepository.ID_TONE_CHAT)); Assert.assertTrue(ids.contains(PromptTemplateRepository.ID_LEARNING_CONTEXT)); - Assert.assertTrue(ids.contains(PromptTemplateRepository.ID_MEMORY_EXTRACTION)); - Assert.assertTrue(ids.contains(PromptTemplateRepository.ID_SKILL_EXTRACTION)); Assert.assertTrue(ids.contains(PromptTemplateRepository.ID_CONTEXT_COMPACTION)); Assert.assertTrue(ids.contains(PromptTemplateRepository.ID_CHAT_MODE_CHAT)); Assert.assertTrue(ids.contains(PromptTemplateRepository.ID_CHAT_MODE_PLAN)); diff --git a/app/src/test/java/cn/lineai/data/repository/ToolSettingsRepositoryTest.java b/app/src/test/java/cn/lineai/data/repository/ToolSettingsRepositoryTest.java index 59aa2fe7..62da5621 100644 --- a/app/src/test/java/cn/lineai/data/repository/ToolSettingsRepositoryTest.java +++ b/app/src/test/java/cn/lineai/data/repository/ToolSettingsRepositoryTest.java @@ -10,6 +10,7 @@ import cn.lineai.tool.ToolDisplayCategory; import cn.lineai.tool.ToolDisplayResolver; import cn.lineai.tool.ToolInfo; +import cn.lineai.tool.ToolNames; import cn.lineai.tool.ToolRegistry; import java.lang.reflect.Field; import java.util.LinkedHashMap; @@ -407,6 +408,24 @@ public void terminalProviderToolPromptEmptyEnabledReturnsNoToolsMessage() { Assert.assertTrue(prompt.contains("No tools are available")); } + @Test + public void learningModeGateRemovesMemoryUpdateWhenDisabled() { + Set enabled = new LinkedHashSet<>(); + enabled.add("file_read"); + enabled.add(ToolNames.MEMORY_UPDATE); + ToolSettingsRepository.applyLearningModeGate(enabled, false); + Assert.assertFalse(enabled.contains(ToolNames.MEMORY_UPDATE)); + } + + @Test + public void learningModeGateKeepsMemoryUpdateWhenEnabled() { + Set enabled = new LinkedHashSet<>(); + enabled.add("file_read"); + enabled.add(ToolNames.MEMORY_UPDATE); + ToolSettingsRepository.applyLearningModeGate(enabled, true); + Assert.assertTrue(enabled.contains(ToolNames.MEMORY_UPDATE)); + } + private static final class DummyCustomMcpTool extends BaseTool { @Override public String getName() { diff --git a/core-security/src/main/java/cn/lineai/security/SimpleHttpClient.java b/core-security/src/main/java/cn/lineai/security/SimpleHttpClient.java index 8b534c80..c762cd9d 100644 --- a/core-security/src/main/java/cn/lineai/security/SimpleHttpClient.java +++ b/core-security/src/main/java/cn/lineai/security/SimpleHttpClient.java @@ -15,6 +15,8 @@ */ public final class SimpleHttpClient { + public static final long MAX_RESPONSE_BODY_BYTES = 32L * 1024 * 1024; + private SimpleHttpClient() { } @@ -60,7 +62,7 @@ public static String get(String url, int connectTimeoutMs, int readTimeoutMs, } public static DownloadResult download(String url, int connectTimeoutMs, int readTimeoutMs) throws Exception { - return download(url, connectTimeoutMs, readTimeoutMs, Integer.MAX_VALUE); + return download(url, connectTimeoutMs, readTimeoutMs, (int) MAX_RESPONSE_BODY_BYTES); } public static DownloadResult download(String url, int connectTimeoutMs, int readTimeoutMs, int maxBytes) throws Exception { @@ -147,8 +149,13 @@ public static String readStream(InputStream input) throws Exception { try { ByteArrayOutputStream output = new ByteArrayOutputStream(); byte[] buffer = new byte[8192]; + long total = 0; int read; while ((read = input.read(buffer)) != -1) { + total += read; + if (total > MAX_RESPONSE_BODY_BYTES) { + throw new Exception("Response body too large, current limit is " + (MAX_RESPONSE_BODY_BYTES / 1024 / 1024) + " MB."); + } output.write(buffer, 0, read); } return output.toString(StandardCharsets.UTF_8.name()); @@ -164,7 +171,7 @@ public static byte[] readBytes(InputStream input, int maxBytes) throws Exception try { ByteArrayOutputStream output = new ByteArrayOutputStream(); byte[] buffer = new byte[8192]; - int total = 0; + long total = 0; int read; while ((read = input.read(buffer)) >= 0) { total += read; diff --git a/core-security/src/main/java/cn/lineai/security/UrlPolicy.java b/core-security/src/main/java/cn/lineai/security/UrlPolicy.java index 74093cc6..6fc67464 100644 --- a/core-security/src/main/java/cn/lineai/security/UrlPolicy.java +++ b/core-security/src/main/java/cn/lineai/security/UrlPolicy.java @@ -29,9 +29,51 @@ private void registerDefaultCleartextHosts() { } private void registerDefaultPrivateNetworkPredicates() { - addPrivateNetworkPredicate(host -> host.startsWith("192.168.")); - addPrivateNetworkPredicate(host -> host.startsWith("10.")); - addPrivateNetworkPredicate(host -> host.matches("172\\.(1[6-9]|2[0-9]|3[01])\\..*")); + // Classify as private network ONLY for a literal IPv4 address (dotted-quad) + // in an RFC1918 range or loopback. Public DNS names such as "10.evil.com" + // or "192.168.attacker.example" must NOT be treated as private. + addPrivateNetworkPredicate(UrlPolicy::isPrivateIpv4); + } + + private static boolean isPrivateIpv4(String host) { + String[] parts = host.split("\\."); + if (parts.length != 4) { + return false; + } + int[] octets = new int[4]; + for (int i = 0; i < 4; i++) { + String part = parts[i]; + if (part.length() == 0 || part.length() > 3) { + return false; + } + int value; + try { + value = Integer.parseInt(part); + } catch (NumberFormatException ignored) { + return false; + } + if (value < 0 || value > 255) { + return false; + } + octets[i] = value; + } + if (octets[0] == 10) { + // 10.0.0.0/8 (also covers the 10.0.2.2 emulator alias) + return true; + } + if (octets[0] == 172 && octets[1] >= 16 && octets[1] <= 31) { + // 172.16.0.0/12 + return true; + } + if (octets[0] == 192 && octets[1] == 168) { + // 192.168.0.0/16 + return true; + } + if (octets[0] == 127) { + // loopback 127.0.0.0/8 + return true; + } + return false; } public void addCleartextHost(String host) { diff --git a/data/src/main/java/cn/lineai/data/importer/LineCodeImportMapper.java b/data/src/main/java/cn/lineai/data/importer/LineCodeImportMapper.java index 3d4cbb68..76116ca3 100644 --- a/data/src/main/java/cn/lineai/data/importer/LineCodeImportMapper.java +++ b/data/src/main/java/cn/lineai/data/importer/LineCodeImportMapper.java @@ -183,10 +183,9 @@ private JSONObject resolveConversationObject( throw new IllegalStateException("LineCode chat file requires payloadDir: " + id); } String fileName = stored.optString("fileName", safeConversationFileName(id)); - File target = new File(new File(payloadDir, CONVERSATION_FILES_DIR), fileName); - if (!target.isFile()) { - File backup = new File(target.getAbsolutePath() + ".bak"); - target = backup.isFile() ? backup : target; + File target = resolveConversationFile(id, fileName, payloadDir); + if (target == null) { + throw new IllegalStateException("LineCode chat file escapes payload directory: " + id); } return new JSONObject(readUtf8(target)); } @@ -260,6 +259,68 @@ private String safeConversationFileName(String id) { return (safeId.length() == 0 ? "conversation" : safeId) + ".json"; } + /** + * Resolve the conversation file inside the payload conversations directory. The incoming + * {@code fileName} originates from an untrusted imported archive and may contain path + * separators or {@code ..} fragments; sanitize it (falling back to a safe derived name) + * and verify via the canonical path that the resolved target stays inside the directory. + * + * @return the resolved existing file (primary or ".bak" fallback), or {@code null} if no + * file can be resolved safely inside the conversations directory. + */ + private File resolveConversationFile(String id, String fileName, File payloadDir) { + File conversationsDir = new File(payloadDir, CONVERSATION_FILES_DIR); + File target = buildConversationTarget(conversationsDir, safeFileName(fileName, id)); + if (target == null) { + return null; + } + if (target.isFile()) { + return target; + } + File backup = new File(target.getPath() + ".bak"); + if (backup.isFile()) { + return backup; + } + return null; + } + + private File buildConversationTarget(File conversationsDir, String fileName) { + File target = new File(conversationsDir, fileName); + File canonicalTarget = canonical(target); + if (canonicalTarget == null) { + return null; + } + File canonicalDir = canonical(conversationsDir); + if (canonicalDir == null || !startsWith(canonicalTarget, canonicalDir)) { + return null; + } + return canonicalTarget; + } + + private boolean startsWith(File path, File prefix) { + String pathName = path.getPath(); + String prefixName = prefix.getPath(); + if (prefixName.length() == 0) { + return true; + } + return pathName.startsWith(prefixName) && (pathName.length() == prefixName.length() || pathName.charAt(prefixName.length()) == File.separatorChar); + } + + private static File canonical(File file) { + try { + return file.getCanonicalFile(); + } catch (Exception ignored) { + return null; + } + } + + private String safeFileName(String fileName, String id) { + if (fileName == null || fileName.length() == 0 || fileName.contains("/") || fileName.contains("\\") || fileName.contains("..")) { + return safeConversationFileName(id); + } + return fileName; + } + private String readUtf8(File file) throws Exception { FileInputStream input = new FileInputStream(file); try { diff --git a/data/src/main/java/cn/lineai/data/repository/PromptTemplateRepository.java b/data/src/main/java/cn/lineai/data/repository/PromptTemplateRepository.java index 4b10d0f9..5be6cd22 100644 --- a/data/src/main/java/cn/lineai/data/repository/PromptTemplateRepository.java +++ b/data/src/main/java/cn/lineai/data/repository/PromptTemplateRepository.java @@ -17,8 +17,6 @@ public final class PromptTemplateRepository { public static final String ID_TONE_CODING = "toneCoding"; public static final String ID_TONE_CHAT = "toneChat"; public static final String ID_LEARNING_CONTEXT = "learningContext"; - public static final String ID_MEMORY_EXTRACTION = "memoryExtraction"; - public static final String ID_SKILL_EXTRACTION = "skillExtraction"; public static final String ID_CONTEXT_COMPACTION = "contextCompaction"; public static final String ID_CHAT_MODE_CHAT = "chatModeChat"; public static final String ID_CHAT_MODE_PLAN = "chatModePlan"; @@ -202,20 +200,6 @@ private static List buildDefinitions() { "WORKING_MEMORY_SECTION", "MEMORY_SECTION", "HISTORY_SECTION", "SKILL_PATHS_SECTION", "SKILLS_SECTION", "PRIVATE_BOUNDARY_SECTION" )); - definitions.add(new Definition( - ID_MEMORY_EXTRACTION, - R.string.prompt_template_memory_extraction_title, - R.string.prompt_template_memory_extraction_description, - "prompts/memory-extraction-template.txt", - "PROJECT_ID", "USER_INPUT", "TRANSCRIPT" - )); - definitions.add(new Definition( - ID_SKILL_EXTRACTION, - R.string.prompt_template_skill_extraction_title, - R.string.prompt_template_skill_extraction_description, - "prompts/skill-extraction-template.txt", - "PROJECT_ID", "USER_INPUT", "TRANSCRIPT" - )); definitions.add(new Definition( ID_CONTEXT_COMPACTION, R.string.prompt_template_context_compaction_title, diff --git a/data/src/main/java/cn/lineai/log/ErrorLogRepository.java b/data/src/main/java/cn/lineai/log/ErrorLogRepository.java index 23ff852a..b20107c4 100644 --- a/data/src/main/java/cn/lineai/log/ErrorLogRepository.java +++ b/data/src/main/java/cn/lineai/log/ErrorLogRepository.java @@ -38,7 +38,7 @@ public File record(String type, String summary, Throwable throwable, String deta builder.append("LineCode Error Log\n"); builder.append("Time: ").append(DISPLAY_TIME_FORMAT.format(new Date(now))).append('\n'); builder.append("Type: ").append(type == null ? "error" : type).append('\n'); - builder.append("Summary: ").append(summary == null ? "" : summary).append("\n\n"); + builder.append("Summary: ").append(summary == null ? "" : ErrorLogRedactor.redact(summary)).append("\n\n"); if (details != null && details.length() > 0) { builder.append("Details:\n").append(ErrorLogRedactor.redact(details)).append("\n\n"); } diff --git a/data/src/main/res/values-zh/strings.xml b/data/src/main/res/values-zh/strings.xml index 79add864..aa6f83d3 100644 --- a/data/src/main/res/values-zh/strings.xml +++ b/data/src/main/res/values-zh/strings.xml @@ -10,8 +10,6 @@ 会话 Agent 模式提示词 会话 Control 模式提示词 学习模式上下文模板 - 长期记忆提取模板 - Skills 沉淀模板 上下文压缩模板 模型身份提示词 TODO 状态模板 @@ -35,8 +33,6 @@ 顶部会话模式选择 Agent 时注入，允许按权限读取、修改、执行和验证任务。 顶部会话模式选择 Control 时注入，仅允许手机控制工具，禁止文件操作、Shell 和 Agent。 学习模式开启时，把短期记忆、长期记忆、聊天检索和 Skills 检索结果渲染进 system prompt。 - 学习模式保存记忆时使用，指导模型从本轮对话中提取 user/project/environment 记忆。 - 学习模式判断是否生成可复用 Skill 时使用，约束返回 JSON 和 Skill 内容格式。 上下文过长时用于总结旧对话，要求模型输出可恢复任务状态的压缩摘要。 把当前模型的 modelId、名称、提供方和协议注入到 system prompt，让模型在回答自身能力相关问题时以模型标识为依据。 把当前 TODO 列表注入到 system prompt，引导模型按顺序推进并及时更新状态。 diff --git a/data/src/main/res/values/strings.xml b/data/src/main/res/values/strings.xml index 08428f30..0b5a523a 100644 --- a/data/src/main/res/values/strings.xml +++ b/data/src/main/res/values/strings.xml @@ -10,8 +10,6 @@ Session Agent mode prompt Session Control mode prompt Learning mode context template - Long-term memory extraction template - Skills extraction template Context compaction template Model identity prompt TODO state template @@ -35,8 +33,6 @@ Injected when the top session mode is set to Agent; allows reading, modifying, executing and verifying tasks per permissions. Injected when the top session mode is set to Control; only phone control tools allowed, file operations, Shell and Agent forbidden. When learning mode is on, renders short-term memory, long-term memory, chat retrieval and Skills retrieval results into the system prompt. - Used when learning mode saves memories; instructs the model to extract user/project/environment memories from the current conversation. - Used when learning mode decides whether to generate a reusable Skill; constrains JSON output and Skill content format. Used to summarize old conversation when context gets too long; requires the model to output a compaction summary that can recover task state. Injects the current model\'s modelId, name, provider and protocol into the system prompt so the model answers capability questions based on the model identifier. Injects the current TODO list into the system prompt, guiding the model to advance in order and update state in time. diff --git a/feature-model/src/main/assets/prompts/memory-extraction-template.txt b/feature-model/src/main/assets/prompts/memory-extraction-template.txt deleted file mode 100644 index 09762de5..00000000 --- a/feature-model/src/main/assets/prompts/memory-extraction-template.txt +++ /dev/null @@ -1,40 +0,0 @@ -You are LineCode's long-term memory extractor. Your task is to extract a small number of long-term reusable memories from the current conversation and classify them by scope. - -Only output JSON; do not output Markdown, explanations, code blocks, or natural language wrappers. - -## Writable Scopes -- user: Cross-project long-term user preferences, stable habits, general requirements. -- project: Constraints, architecture, technology choices, UI alignment requirements, and prohibitions for the current project/workspace. -- environment: Stable environmental facts about the current device, system, build environment, paths, permissions, network sources, etc. - -## Strict Rules -- Do not save entire conversation segments, one-time tasks, temporary progress, error logs, command outputs, or PR/commit numbers as long-term memories. -- Do not save sensitive information such as API keys, tokens, passwords, private keys, cookies, phone numbers, ID numbers, or complete key fragments. -- Do not save content the user merely gave as an example, asked rhetorically, or was uncertain about, unless the semantics clearly indicate a long-term constraint. -- Constraints related to "this project / current project / the project / codebase / repository / workspace / app" must use project scope, not user scope. -- Facts related to "my phone / current device / Android / Termux / Gradle / JDK / paths / permissions / mirror sources" must use environment scope. -- Memory content should be rewritten as an independent, clear, long-term effective statement. -- Return at most 5 items. - -## JSON Format -{ - "memories": [ - { - "scope": "user|project|environment", - "content": "A long-term reusable memory", - "confidence": 0.0 - } - ] -} - -If there are no long-term memories worth saving, return: -{"memories":[]} - -## Current Project -{{PROJECT_ID}} - -## Current User Input -{{USER_INPUT}} - -## Current Conversation Segment -{{TRANSCRIPT}} diff --git a/feature-model/src/main/assets/prompts/skill-extraction-template.txt b/feature-model/src/main/assets/prompts/skill-extraction-template.txt deleted file mode 100644 index f6ad0997..00000000 --- a/feature-model/src/main/assets/prompts/skill-extraction-template.txt +++ /dev/null @@ -1,27 +0,0 @@ -You are LineCode's Skills distiller. Please determine whether the current conversation has produced any long-term reusable workflows, project conventions, or troubleshooting methods. - -Only create a Skill when the content is stable, reusable, and not one-time task progress. Do not save API keys, tokens, passwords, private keys, one-time paths, temporary error fragments, or chat transcripts. - -Project path: {{PROJECT_ID}} -User input: -{{USER_INPUT}} - -Conversation summary: -{{TRANSCRIPT}} - -Please return only JSON in the following format: -{ - "skills": [ - { - "name": "kebab-case-english-name", - "description": "One-sentence description", - "location": "project or app", - "content": "# Skill Title\n\n## Trigger Conditions\n- ...\n\n## Steps\n- ...\n\n## Verification\n- ..." - } - ] -} - -Rules: -- Project conventions, specific repository constraints, build/test workflows default to location=project. -- Only use location=app for user global preferences or cross-project methods. -- When there is nothing worth distilling, return {"skills":[]}. diff --git a/feature-model/src/main/java/cn/lineai/ai/protocol/OpenAiCompatibleProtocol.java b/feature-model/src/main/java/cn/lineai/ai/protocol/OpenAiCompatibleProtocol.java index 1a88615b..f4753076 100644 --- a/feature-model/src/main/java/cn/lineai/ai/protocol/OpenAiCompatibleProtocol.java +++ b/feature-model/src/main/java/cn/lineai/ai/protocol/OpenAiCompatibleProtocol.java @@ -14,6 +14,7 @@ import cn.lineai.ai.protocol.reasoning.DashscopeReasoningStrategy; import cn.lineai.ai.protocol.reasoning.DeepseekReasoningStrategy; import cn.lineai.ai.protocol.reasoning.DefaultReasoningStrategy; +import cn.lineai.ai.protocol.reasoning.KimiReasoningStrategy; import cn.lineai.ai.protocol.reasoning.MinimaxReasoningStrategy; import cn.lineai.ai.protocol.reasoning.MoonshotReasoningStrategy; import cn.lineai.ai.protocol.reasoning.ReasoningDeltaExtractor; @@ -37,6 +38,7 @@ private static ReasoningStrategyRegistry createDefaultRegistry() { registry.register(new DashscopeReasoningStrategy()); registry.register(new MinimaxReasoningStrategy()); registry.register(new DeepseekReasoningStrategy()); + registry.register(new KimiReasoningStrategy()); registry.register(new MoonshotReasoningStrategy()); registry.register(new DefaultReasoningStrategy()); return registry; @@ -65,6 +67,7 @@ public ModelCompletionResponse complete(ModelConfig config, List m body.put("model", ModelContextParser.apiModelId(config)); body.put("messages", messageSerializer.messagesJson(messages)); body.put("temperature", 0.2); + applyReasoningRequest(config, body, ModelRequestOptions.defaults()); HashMap headers = new HashMap<>(); headers.put("Authorization", "Bearer " + config.getApiKey()); diff --git a/feature-model/src/main/java/cn/lineai/ai/protocol/reasoning/KimiReasoningStrategy.java b/feature-model/src/main/java/cn/lineai/ai/protocol/reasoning/KimiReasoningStrategy.java new file mode 100644 index 00000000..21b62e35 --- /dev/null +++ b/feature-model/src/main/java/cn/lineai/ai/protocol/reasoning/KimiReasoningStrategy.java @@ -0,0 +1,24 @@ +package cn.lineai.ai.protocol.reasoning; + +import cn.lineai.ai.protocol.ReasoningRequestContext; +import cn.lineai.ai.protocol.ReasoningRequestStrategy; +import org.json.JSONObject; + +public final class KimiReasoningStrategy implements ReasoningRequestStrategy { + @Override + public boolean matches(String baseUrl, String modelId) { + return baseUrl.contains("moonshot") || baseUrl.contains("kimi") || modelId.contains("kimi") + || modelId.contains("moonshot"); + } + + @Override + public void apply(JSONObject body, ReasoningRequestContext context) throws Exception { + JSONObject thinking = new JSONObject().put("type", context.isEnabled() ? "enabled" : "disabled"); + if (context.isPreserveReasoning()) { + thinking.put("keep", "all"); + } + body.put("thinking", thinking); + // Kimi 官方要求 temperature 必须 >= 1.0，低于该值会报错。 + body.put("temperature", Math.max(1.0, body.optDouble("temperature", 0.2))); + } +} diff --git a/feature-model/src/main/java/cn/lineai/ai/protocol/reasoning/MoonshotReasoningStrategy.java b/feature-model/src/main/java/cn/lineai/ai/protocol/reasoning/MoonshotReasoningStrategy.java index c757feb0..ab1eeb0b 100644 --- a/feature-model/src/main/java/cn/lineai/ai/protocol/reasoning/MoonshotReasoningStrategy.java +++ b/feature-model/src/main/java/cn/lineai/ai/protocol/reasoning/MoonshotReasoningStrategy.java @@ -7,8 +7,7 @@ public final class MoonshotReasoningStrategy implements ReasoningRequestStrategy { @Override public boolean matches(String baseUrl, String modelId) { - return baseUrl.contains("moonshot") || baseUrl.contains("kimi") || modelId.contains("kimi") - || baseUrl.contains("bigmodel") || baseUrl.contains("zhipu") || modelId.contains("glm") + return baseUrl.contains("bigmodel") || baseUrl.contains("zhipu") || modelId.contains("glm") || baseUrl.contains("mimo") || baseUrl.contains("xiaomi") || modelId.contains("mimo"); } @@ -17,9 +16,6 @@ public void apply(JSONObject body, ReasoningRequestContext context) throws Excep JSONObject thinking = new JSONObject().put("type", context.isEnabled() ? "enabled" : "disabled"); String base = context.getBaseUrl(); String model = context.getModelId(); - if (context.isPreserveReasoning() && (base.contains("moonshot") || base.contains("kimi") || model.contains("kimi"))) { - thinking.put("keep", "all"); - } body.put("thinking", thinking); if (context.isPreserveReasoning() && (base.contains("bigmodel") || base.contains("zhipu") || model.contains("glm"))) { body.put("clear_thinking", false); diff --git a/feature-model/src/main/java/cn/lineai/context/ContextCompactionService.java b/feature-model/src/main/java/cn/lineai/context/ContextCompactionService.java index e29a22d9..bbfa113d 100644 --- a/feature-model/src/main/java/cn/lineai/context/ContextCompactionService.java +++ b/feature-model/src/main/java/cn/lineai/context/ContextCompactionService.java @@ -251,6 +251,13 @@ private ContextCompactionResult compactWithResponsesApi( if (cancellationToken != null && cancellationToken.isCancelled()) { return new ContextCompactionResult("", ""); } + // Codex/OpenAI-Responses 压缩调用返回的是字符串压缩项，不携带 token usage， + // 因此不会像 streamSummaryWithRetry 那样把压缩后的 usage 记入 tracker。 + // 这里显式重置 tracker，避免残留压缩前的旧 usage 让后续 shouldCompact/ + // shouldSoftCompact 继续以旧的大数值触发过度压缩；重置后基线回退到本地估算。 + if (tokenUsageTracker != null) { + tokenUsageTracker.reset(); + } return new ContextCompactionResult(createResponsesCompactFallbackContent(), compactItem); } diff --git a/feature-model/src/test/java/cn/lineai/ai/protocol/OpenAiCompatibleProtocolTest.java b/feature-model/src/test/java/cn/lineai/ai/protocol/OpenAiCompatibleProtocolTest.java index 9ec81fdc..caa8a0ee 100644 --- a/feature-model/src/test/java/cn/lineai/ai/protocol/OpenAiCompatibleProtocolTest.java +++ b/feature-model/src/test/java/cn/lineai/ai/protocol/OpenAiCompatibleProtocolTest.java @@ -4,6 +4,7 @@ import static org.junit.Assert.assertEquals; import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; import cn.lineai.ai.ModelCompletionResponse; import cn.lineai.ai.ImageInputPayload; @@ -12,6 +13,8 @@ import cn.lineai.ai.message.ModelMessage; import cn.lineai.ai.message.ToolModelMessage; import cn.lineai.ai.message.UserModelMessage; +import cn.lineai.ai.protocol.reasoning.KimiReasoningStrategy; +import cn.lineai.ai.protocol.reasoning.MoonshotReasoningStrategy; import cn.lineai.model.AiBehaviorSettings; import cn.lineai.model.ModelConfig; import cn.lineai.model.ModelProtocolType; @@ -151,6 +154,101 @@ public void nvidiaGatewayDoesNotSendUnsupportedThinkingParameters() throws Excep assertFalse(body.has("reasoning")); } + @Test + public void kimiGatewayClampsTemperatureAndEnablesThinking() throws Exception { + ModelConfig config = ModelConfig.builder( + "kimi-k2", + "Kimi K2", + ModelProtocolType.OPENAI_COMPATIBLE, + "Moonshot", + "https://api.moonshot.cn/v1", + "sk-test", + "kimi-k2-0711-preview").build(); + + JSONObject body = new OpenAiCompatibleProtocol().reasoningRequestBodyForTest( + config, + new ModelRequestOptions(AiBehaviorSettings.REASONING_HIGH, false) + ); + + assertEquals(1.0, body.optDouble("temperature", 0.2), 0.0001); + assertTrue("enabled".equals(body.getJSONObject("thinking").getString("type"))); + } + + @Test + public void kimiPreserveReasoningKeepsThinking() throws Exception { + ModelConfig config = ModelConfig.builder( + "kimi-k2", + "Kimi K2", + ModelProtocolType.OPENAI_COMPATIBLE, + "Moonshot", + "https://api.moonshot.cn/v1", + "sk-test", + "kimi-k2-0711-preview").build(); + + JSONObject body = new OpenAiCompatibleProtocol().reasoningRequestBodyForTest( + config, + new ModelRequestOptions(AiBehaviorSettings.REASONING_HIGH, true) + ); + + assertTrue("all".equals(body.getJSONObject("thinking").getString("keep"))); + } + + @Test + public void kimiStrategyClampsPreExistingTemperature() throws Exception { + JSONObject body = new JSONObject().put("temperature", 0.2); + new KimiReasoningStrategy().apply( + body, + new ReasoningRequestContext(true, "high", true, "api.moonshot.cn", "kimi-k2", 0) + ); + + assertEquals(1.0, body.getDouble("temperature"), 0.0001); + assertTrue("all".equals(body.getJSONObject("thinking").getString("keep"))); + } + + @Test + public void glmConfigStillHandledByMoonshotStrategy() throws Exception { + String baseUrl = "https://open.bigmodel.cn/api/paas/v4"; + String modelId = "glm-4-plus"; + ModelConfig config = ModelConfig.builder( + "glm-4", + "GLM-4 Plus", + ModelProtocolType.OPENAI_COMPATIBLE, + "Zhipu", + baseUrl, + "sk-test", + modelId).build(); + + JSONObject body = new OpenAiCompatibleProtocol().reasoningRequestBodyForTest( + config, + new ModelRequestOptions(AiBehaviorSettings.REASONING_HIGH, true) + ); + + assertTrue("enabled".equals(body.getJSONObject("thinking").getString("type"))); + assertTrue(body.has("clear_thinking")); + assertFalse(body.getBoolean("clear_thinking")); + assertFalse(new KimiReasoningStrategy().matches(baseUrl, modelId)); + assertTrue(new MoonshotReasoningStrategy().matches(baseUrl, modelId)); + } + + @Test + public void deepseekTemperatureRemainsUnclamped() throws Exception { + ModelConfig config = ModelConfig.builder( + "deepseek", + "DeepSeek Chat", + ModelProtocolType.OPENAI_COMPATIBLE, + "DeepSeek", + "https://api.deepseek.com", + "sk-test", + "deepseek-chat").build(); + + JSONObject body = new OpenAiCompatibleProtocol().reasoningRequestBodyForTest( + config, + new ModelRequestOptions(AiBehaviorSettings.REASONING_OFF, false) + ); + + assertEquals(0.2, body.optDouble("temperature", 0.2), 0.0001); + } + private static JSONObject chunk(JSONObject delta, String finishReason) throws Exception { return new JSONObject() .put("id", "chunk_1") diff --git a/feature-share/src/main/java/cn/lineai/share/format/PdfRenderer.java b/feature-share/src/main/java/cn/lineai/share/format/PdfRenderer.java index c217b704..05d62a9f 100644 --- a/feature-share/src/main/java/cn/lineai/share/format/PdfRenderer.java +++ b/feature-share/src/main/java/cn/lineai/share/format/PdfRenderer.java @@ -75,9 +75,11 @@ public void render(PdfDocument doc, List messages) { } // Handle newlines + boolean newlineFound = false; int newlineIdx = content.indexOf('\n', start); if (newlineIdx >= start && newlineIdx < end) { end = newlineIdx; + newlineFound = true; } String line = content.substring(start, end); @@ -91,7 +93,9 @@ public void render(PdfDocument doc, List messages) { canvas.drawText(line, margin, y, bodyPaint); y += BODY_SIZE + LINE_SPACING; } - start = end + 1; + // 若本段是因为换行符中断，则跳过换行符；否则退出的 end 是首个放不下的字符， + // 必须作为下一行起点重新处理，避免丢字符。 + start = newlineFound ? end + 1 : end; } y += LINE_SPACING; } diff --git a/feature-ssh/src/main/java/cn/lineai/ssh/SshConnectionPool.java b/feature-ssh/src/main/java/cn/lineai/ssh/SshConnectionPool.java index ad35ed9c..9e053751 100644 --- a/feature-ssh/src/main/java/cn/lineai/ssh/SshConnectionPool.java +++ b/feature-ssh/src/main/java/cn/lineai/ssh/SshConnectionPool.java @@ -96,7 +96,38 @@ public Session getOrCreate(SshConfig config, int timeoutMs) throws Exception { Entry created = new Entry(session); created.lock.lock(); created.inUse = true; - entries.put(key, created); + Entry prior = entries.putIfAbsent(key, created); + if (prior != null) { + // 竞争失败：另一个线程已抢先放入同 key 的 entry，本次新建的会话被丢弃。 + // 有限次尝试借用已有 entry；都不行则回退为用新建 entry 强制替换。 + Entry candidate = prior; + for (int attempt = 0; attempt < 2 && candidate != null; attempt++) { + if (candidate.closed || !candidate.lock.tryLock()) { + if (candidate.closed) { + entries.remove(key, candidate); + } + // 无效或正在被占用的 entry，读取下次尝试的最新 entry。 + candidate = entries.get(key); + continue; + } + if (!candidate.closed && candidate.session.isConnected()) { + // 借用成功，丢弃本次新建的 entry。 + candidate.inUse = true; + candidate.lastUsedAtMs = System.currentTimeMillis(); + created.inUse = false; + created.lock.unlock(); + closeQuietly(created); + return candidate.session; + } + // 已有 entry 失效：解锁移除，读取下次尝试的最新 entry。 + candidate.lock.unlock(); + entries.remove(key, candidate); + candidate = entries.get(key); + } + // 回退：用新建 entry 强制替换（保持原有的 put 语义）。 + entries.put(key, created); + return created.session; + } return session; } @@ -121,7 +152,9 @@ public void release(Session session, SshConfig config) { entries.remove(keyOf(config), entry); } } finally { - entry.lock.unlock(); + if (entry.lock.isHeldByCurrentThread()) { + entry.lock.unlock(); + } } } diff --git a/feature-tool/src/main/java/cn/lineai/tool/ToolRegistry.java b/feature-tool/src/main/java/cn/lineai/tool/ToolRegistry.java index 3d7cc7be..713f6284 100644 --- a/feature-tool/src/main/java/cn/lineai/tool/ToolRegistry.java +++ b/feature-tool/src/main/java/cn/lineai/tool/ToolRegistry.java @@ -15,6 +15,7 @@ import java.util.List; import java.util.Map; import java.util.Set; +import java.util.concurrent.locks.ReentrantReadWriteLock; import org.json.JSONArray; public final class ToolRegistry { @@ -23,6 +24,7 @@ public final class ToolRegistry { private final Map tools = new LinkedHashMap<>(); private final Map displayCategoryCache = new LinkedHashMap<>(); + private final ReentrantReadWriteLock lock = new ReentrantReadWriteLock(); private final Context context; private ExtensionStore extensionStore; @@ -46,48 +48,78 @@ public ToolRegistry(Context context, cn.lineai.ipc.IpcProviderManager ipcProvide } public void setExtensionStore(ExtensionStore extensionStore) { - this.extensionStore = extensionStore; + lock.writeLock().lock(); + try { + this.extensionStore = extensionStore; + } finally { + lock.writeLock().unlock(); + } } public void register(BaseTool tool) { if (tool != null) { - tools.put(tool.getName(), tool); - displayCategoryCache.put(tool.getName(), tool.getDisplayCategory()); + lock.writeLock().lock(); + try { + tools.put(tool.getName(), tool); + displayCategoryCache.put(tool.getName(), tool.getDisplayCategory()); + } finally { + lock.writeLock().unlock(); + } } } public BaseTool get(String name) { - return tools.get(name); + lock.readLock().lock(); + try { + return tools.get(name); + } finally { + lock.readLock().unlock(); + } } public ToolDisplayCategory getCachedDisplayCategory(String name) { - ToolDisplayCategory category = displayCategoryCache.get(name); - return category != null ? category : ToolDisplayCategory.GENERIC; + lock.readLock().lock(); + try { + ToolDisplayCategory category = displayCategoryCache.get(name); + return category != null ? category : ToolDisplayCategory.GENERIC; + } finally { + lock.readLock().unlock(); + } } public List getAll() { - return new ArrayList<>(tools.values()); + lock.readLock().lock(); + try { + return new ArrayList<>(tools.values()); + } finally { + lock.readLock().unlock(); + } } public void reloadExtensions() { - removeExtensionTools(); - if (extensionStore == null) { - return; - } - for (ExtensionMcpConfig mcp : extensionStore.getMcpExtensions()) { - if (!mcp.isEnabled()) { - continue; + lock.writeLock().lock(); + try { + removeExtensionTools(); + if (extensionStore == null) { + return; } - for (McpToolSummary tool : mcp.getTools()) { - if (tool.isEnabled()) { - register(new CustomMcpHttpTool(customMcpToolName(mcp, tool), mcp, tool)); + for (ExtensionMcpConfig mcp : extensionStore.getMcpExtensions()) { + if (!mcp.isEnabled()) { + continue; + } + for (McpToolSummary tool : mcp.getTools()) { + if (tool.isEnabled()) { + register(new CustomMcpHttpTool(customMcpToolName(mcp, tool), mcp, tool)); + } } } - } - for (ExtensionAgentConfig agent : extensionStore.getAgentExtensions()) { - if (agent.isEnabled()) { - register(new CustomAgentExtensionTool(customAgentToolName(agent), agent)); + for (ExtensionAgentConfig agent : extensionStore.getAgentExtensions()) { + if (agent.isEnabled()) { + register(new CustomAgentExtensionTool(customAgentToolName(agent), agent)); + } } + } finally { + lock.writeLock().unlock(); } } @@ -96,10 +128,15 @@ public List getByNameSet(Set names) { if (names == null || names.isEmpty()) { return selected; } - for (BaseTool tool : tools.values()) { - if (names.contains(tool.getName())) { - selected.add(tool); + lock.readLock().lock(); + try { + for (BaseTool tool : tools.values()) { + if (names.contains(tool.getName())) { + selected.add(tool); + } } + } finally { + lock.readLock().unlock(); } return selected; } @@ -110,10 +147,15 @@ public List getToolInfoByNameSet(Set names) { if (names == null || names.isEmpty()) { return selected; } - for (BaseTool tool : tools.values()) { - if (names.contains(tool.getName())) { - selected.add(tool); + lock.readLock().lock(); + try { + for (BaseTool tool : tools.values()) { + if (names.contains(tool.getName())) { + selected.add(tool); + } } + } finally { + lock.readLock().unlock(); } return selected; } @@ -169,26 +211,36 @@ public Set mcpToolNamesForIds(List mcpIds) { if (mcpIds == null || mcpIds.isEmpty() || extensionStore == null) { return names; } - for (ExtensionMcpConfig mcp : extensionStore.getMcpExtensions()) { - if (!mcpIds.contains(mcp.getId())) { - continue; - } - for (McpToolSummary tool : mcp.getTools()) { - if (tool.isEnabled()) { - names.add(customMcpToolName(mcp, tool)); + lock.readLock().lock(); + try { + for (ExtensionMcpConfig mcp : extensionStore.getMcpExtensions()) { + if (!mcpIds.contains(mcp.getId())) { + continue; + } + for (McpToolSummary tool : mcp.getTools()) { + if (tool.isEnabled()) { + names.add(customMcpToolName(mcp, tool)); + } } } + } finally { + lock.readLock().unlock(); } return names; } private void removeExtensionTools() { - ArrayList names = new ArrayList<>(tools.keySet()); - for (String name : names) { - if (isExtensionToolName(name)) { - tools.remove(name); - displayCategoryCache.remove(name); + lock.writeLock().lock(); + try { + ArrayList names = new ArrayList<>(tools.keySet()); + for (String name : names) { + if (isExtensionToolName(name)) { + tools.remove(name); + displayCategoryCache.remove(name); + } } + } finally { + lock.writeLock().unlock(); } } diff --git a/feature-tool/src/main/java/cn/lineai/tool/builtin/FileReadTool.java b/feature-tool/src/main/java/cn/lineai/tool/builtin/FileReadTool.java index 35355fe8..43f0637d 100644 --- a/feature-tool/src/main/java/cn/lineai/tool/builtin/FileReadTool.java +++ b/feature-tool/src/main/java/cn/lineai/tool/builtin/FileReadTool.java @@ -16,6 +16,8 @@ public final class FileReadTool extends BaseTool { public static final String NAME = "file_read"; private static final long LARGE_FILE_THRESHOLD_BYTES = 50L * 1024L; + /** 单次 KB 读取的最大跨度（KB），防止超大的 end_kb 一次性申请过多内存。 */ + private static final int MAX_KB_RANGE = 1024; private static final int MAX_DIRECTORY_ITEMS = 400; @Override @@ -25,7 +27,7 @@ public String getName() { @Override public String getDescription() { - return "Read file contents. Returns line-numbered content; for large files, read in segments via start_kb/end_kb. Returns a directory tree when reading a directory."; + return "Read file contents. Returns line-numbered content; for large files, read in segments via start_kb/end_kb (end_kb may exceed 50, up to the file size). Returns a directory tree when reading a directory."; } @Override @@ -64,7 +66,7 @@ public JSONObject getParameters() throws org.json.JSONException { .put("properties", new JSONObject() .put("file_path", new JSONObject().put("type", "string").put("description", "Absolute or relative file path")) .put("start_kb", new JSONObject().put("type", "number").put("description", "Start position in KB, default 0")) - .put("end_kb", new JSONObject().put("type", "number").put("description", "End position in KB, default 50, max 50"))) + .put("end_kb", new JSONObject().put("type", "number").put("description", "End position in KB, default 50; may exceed 50, clamped to the file size"))) .put("required", new org.json.JSONArray().put("file_path")); } @@ -85,7 +87,11 @@ public ToolResult execute(JSONObject input, ToolContext context) { } int startKb = Math.max(0, input.optInt("start_kb", 0)); - int endKb = Math.max(startKb + 1, Math.min(50, input.optInt("end_kb", 50))); + int endKb = Math.max(startKb + 1, input.optInt("end_kb", 50)); + // 限制单次读取跨度，防止超大的 end_kb 导致内存暴涨；大文件按页读取。 + if (endKb - startKb > MAX_KB_RANGE) { + endKb = startKb + MAX_KB_RANGE; + } boolean hasKbRange = input.has("start_kb") || input.has("end_kb"); long fileLen = file.length(); @@ -121,48 +127,38 @@ public ToolResult execute(JSONObject input, ToolContext context) { int startChar = 0; int endChar = content.length(); if (startByte > 0) { - int lineStart = content.lastIndexOf('\n', content.length() - 1); - if (lineStart >= 0) { - startChar = lineStart + 1; + // 若块起点落在行中间，跳过不完整的首行：向前找到块内第一个换行。 + boolean atLineStart; + try (RandomAccessFile raf = new RandomAccessFile(file, "r")) { + raf.seek(startByte - 1); + atLineStart = raf.read() == '\n'; + } + if (!atLineStart) { + int lineStart = content.indexOf('\n'); + if (lineStart >= 0) { + startChar = lineStart + 1; + } } } if (endByte < fileLen) { - int lineEnd = content.indexOf('\n', startChar); + // 块未到文件末尾时，结束位置对齐到块内最后一个换行，保留完整行。 + int lineEnd = content.lastIndexOf('\n'); if (lineEnd >= 0) { endChar = lineEnd + 1; } } // Count the absolute line number at the (snapped) start position. - long startLineNumber = 1; - try (RandomAccessFile raf = new RandomAccessFile(file, "r")) { - long scan = Math.max(0, startByte + startChar); - long pos = 0; - while (pos < scan) { - raf.seek(pos); - int b = raf.read(); - if (b < 0) break; - if (b == '\n') startLineNumber++; - pos++; - } - } + long scan = Math.max(0, startByte + startChar); + long startLineNumber = 1 + countNewlines(file, scan); String extracted = content.substring(startChar, endChar); StringBuilder result = new StringBuilder(); result.append(addLineNumbers(extracted, (int) startLineNumber)); // Add range info - long totalLines = 1; - try (RandomAccessFile raf = new RandomAccessFile(file, "r")) { - long pos = 0; - while (true) { - raf.seek(pos); - int b = raf.read(); - if (b < 0) break; - if (b == '\n') totalLines++; - pos++; - } - } + long totalLines = 1 + countNewlines(file, fileLen); + if (lastByteIsNewline(file)) totalLines--; result.append(context.getString(R.string.tool_file_read_range_info, totalLines, startKb, endKb, fileLen / 1024)); return ok(ToolResult.truncateContent(result.toString())); @@ -171,6 +167,41 @@ public ToolResult execute(JSONObject input, ToolContext context) { } } + /** 分块读取文件，统计字节位置 < upToByte 的 '\n' 个数，避免逐字节 seek/read 的 O(n) 系统调用。 */ + private static long countNewlines(File file, long upToByte) throws Exception { + long count = 0; + byte[] buffer = new byte[64 * 1024]; + long remaining = Math.max(0, upToByte); + try (RandomAccessFile raf = new RandomAccessFile(file, "r")) { + while (remaining > 0) { + int toRead = (int) Math.min(buffer.length, remaining); + int read = raf.read(buffer, 0, toRead); + if (read < 0) { + break; + } + for (int i = 0; i < read; i++) { + if (buffer[i] == '\n') { + count++; + } + } + remaining -= read; + } + } + return count; + } + + /** 判断文件最后一个字节是否为换行符（文件为空时返回 false）。 */ + private static boolean lastByteIsNewline(File file) throws Exception { + long len = file.length(); + if (len <= 0) { + return false; + } + try (RandomAccessFile raf = new RandomAccessFile(file, "r")) { + raf.seek(len - 1); + return raf.read() == '\n'; + } + } + /** 只读取文件的 [start, end) 字节区间，避免一次性加载整个文件。 */ private static byte[] readRange(File file, long start, long end) throws Exception { long len = end - start; @@ -193,11 +224,17 @@ private static byte[] readRange(File file, long start, long end) throws Exceptio } private String addLineNumbers(String content, int startLine) { + if (content.length() == 0) { + return ""; + } + // 内容以换行结尾时，split 会产生一个空的尾元素，需去掉，避免多出空行号。 + boolean endsWithNewline = content.endsWith("\n"); String[] lines = content.split("\n", -1); + int count = lines.length - (endsWithNewline ? 1 : 0); StringBuilder sb = new StringBuilder(); - for (int i = 0; i < lines.length; i++) { + for (int i = 0; i < count; i++) { sb.append(startLine + i).append('\t').append(lines[i]); - if (i + 1 < lines.length) { + if (i + 1 < count) { sb.append('\n'); } } diff --git a/feature-tool/src/main/java/cn/lineai/tool/builtin/ImageApiClient.java b/feature-tool/src/main/java/cn/lineai/tool/builtin/ImageApiClient.java index b1a00ef4..d5f83088 100644 --- a/feature-tool/src/main/java/cn/lineai/tool/builtin/ImageApiClient.java +++ b/feature-tool/src/main/java/cn/lineai/tool/builtin/ImageApiClient.java @@ -22,7 +22,7 @@ String postJson(String url, JSONObject body, Map headers) throws } DownloadedImage downloadImage(String url) throws Exception { - SimpleHttpClient.DownloadResult result = SimpleHttpClient.download(url, 20000, 120000); + SimpleHttpClient.DownloadResult result = SimpleHttpClient.download(url, 20000, 120000, MAX_DOWNLOAD_BYTES); return new DownloadedImage(result.mimeType, result.bytes); } diff --git a/feature-tool/src/main/java/cn/lineai/tool/builtin/ImageResponseParser.java b/feature-tool/src/main/java/cn/lineai/tool/builtin/ImageResponseParser.java index 48e9d2e6..f9645445 100644 --- a/feature-tool/src/main/java/cn/lineai/tool/builtin/ImageResponseParser.java +++ b/feature-tool/src/main/java/cn/lineai/tool/builtin/ImageResponseParser.java @@ -19,6 +19,7 @@ final class ImageResponseParser { } GeneratedImage parseImagesResponse(String raw, ToolContext context) throws Exception { + enforceResponseSizeLimit(raw); JSONObject response = new JSONObject(raw); JSONArray data = response.optJSONArray("data"); if (data == null || data.length() == 0 || data.optJSONObject(0) == null) { @@ -68,6 +69,7 @@ GeneratedImage parseImagesResponse(String raw, ToolContext context) throws Excep } GeneratedImage parseResponsesImage(String raw, ToolContext context) throws Exception { + enforceResponseSizeLimit(raw); JSONObject response = new JSONObject(raw); JSONObject error = response.optJSONObject("error"); if (error != null) { @@ -112,6 +114,16 @@ GeneratedImage parseResponsesImage(String raw, ToolContext context) throws Excep throw new Exception("Responses API did not return image_generation_call.result."); } + private void enforceResponseSizeLimit(String raw) throws Exception { + if (raw == null) { + return; + } + long sizeBytes = raw.getBytes(java.nio.charset.StandardCharsets.UTF_8).length; + if (sizeBytes > ImageApiClient.MAX_RESPONSE_BYTES) { + throw new Exception("Image API response too large, current limit is " + (ImageApiClient.MAX_RESPONSE_BYTES / 1024 / 1024) + " MB."); + } + } + private String mimeFromDataUrl(String dataUrl) { int colon = dataUrl.indexOf(':'); int semicolon = dataUrl.indexOf(';'); diff --git a/feature-tool/src/main/java/cn/lineai/tool/builtin/ShellExecuteTool.java b/feature-tool/src/main/java/cn/lineai/tool/builtin/ShellExecuteTool.java index 6577b8c0..3d55ac0c 100644 --- a/feature-tool/src/main/java/cn/lineai/tool/builtin/ShellExecuteTool.java +++ b/feature-tool/src/main/java/cn/lineai/tool/builtin/ShellExecuteTool.java @@ -140,7 +140,6 @@ private ToolResult executeViaTerminalProvider(String command, String cwd, long t @Override public void onOutput(String content) { synchronized (streamedOutput) { - streamedOutput.setLength(0); streamedOutput.append(content == null ? "" : content); } if (context != null) { @@ -190,7 +189,6 @@ private ToolResult executeViaSsh(String inputCommand, String cwd, long timeoutMs try { String output = sshService.executeCommand(command, (int) timeoutMs, null, streamed -> { synchronized (streamedOutput) { - streamedOutput.setLength(0); streamedOutput.append(streamed == null ? "" : streamed); } if (context != null) { diff --git a/feature-tool/src/test/java/cn/lineai/tool/ToolBuiltinsTest.java b/feature-tool/src/test/java/cn/lineai/tool/ToolBuiltinsTest.java index 63829fb7..0f18f336 100644 --- a/feature-tool/src/test/java/cn/lineai/tool/ToolBuiltinsTest.java +++ b/feature-tool/src/test/java/cn/lineai/tool/ToolBuiltinsTest.java @@ -36,9 +36,8 @@ public void fileReadReturnsNumberedLines() throws Exception { .put("file_path", "demo.txt"), context()); Assert.assertFalse(result.isError()); - Assert.assertTrue(result.getContent().contains("1\tone")); - Assert.assertTrue(result.getContent().contains("2\ttwo")); - Assert.assertTrue(result.getContent().contains("3\tthree")); + // No phantom empty trailing line after the final newline. + Assert.assertEquals("1\tone\n2\ttwo\n3\tthree", result.getContent()); } @Test @@ -53,6 +52,8 @@ public void fileReadWithKbRange() throws Exception { Assert.assertFalse(result.isError()); Assert.assertTrue(result.getContent().contains("1\tone")); + Assert.assertTrue(result.getContent().contains("2\ttwo")); + Assert.assertTrue(result.getContent().contains("3\tthree")); } @Test @@ -74,6 +75,55 @@ public void fileReadLargeFileWithKbRangeDoesNotError() throws Exception { Assert.assertFalse(result.isError()); Assert.assertTrue(result.getContent().contains("1\tline 0")); + // The whole 50KB page must be returned, not just the first line. + Assert.assertTrue(result.getContent().contains("2\tline 1")); + Assert.assertTrue(result.getContent().contains("100\tline 99")); + } + + @Test + public void fileReadKbMidFileChunkReturnsCompleteLines() throws Exception { + File file = folder.newFile("big.txt"); + StringBuilder sb = new StringBuilder(); + for (int i = 0; i < 200000; i++) { + sb.append("line ").append(i).append("\n"); + } + Files.write(file.toPath(), sb.toString().getBytes(StandardCharsets.UTF_8)); + + ToolResult result = new FileReadTool().execute(new JSONObject() + .put("file_path", "big.txt") + .put("start_kb", 10) + .put("end_kb", 11), context()); + + Assert.assertFalse(result.isError()); + String content = result.getContent(); + String[] lines = content.split("\n"); + Assert.assertTrue("expected several complete lines, got: " + content, lines.length >= 3); + int first = Integer.parseInt(lines[0].substring(0, lines[0].indexOf('\t'))); + int second = Integer.parseInt(lines[1].substring(0, lines[1].indexOf('\t'))); + Assert.assertEquals(first + 1, second); + Assert.assertTrue("mid-file chunk must not start at line 1", first > 1); + } + + @Test + public void fileReadEndKbBeyond50ReadsPastFirst50Kb() throws Exception { + File file = folder.newFile("big.txt"); + StringBuilder sb = new StringBuilder(); + for (int i = 0; i < 200000; i++) { + sb.append("line ").append(i).append("\n"); + } + Files.write(file.toPath(), sb.toString().getBytes(StandardCharsets.UTF_8)); + Assert.assertTrue(file.length() > 1024 * 1024); + + ToolResult result = new FileReadTool().execute(new JSONObject() + .put("file_path", "big.txt") + .put("start_kb", 0) + .put("end_kb", 100), context()); + + Assert.assertFalse(result.isError()); + Assert.assertTrue(result.getContent().contains("1\tline 0")); + // end_kb above 50 must be honored: the range info reports KB 0-100 + // instead of silently clamping to 50. + Assert.assertTrue(result.getContent().contains("showing KB 0-100")); } @Test diff --git a/markdown/src/main/java/cn/lineai/ui/markdown/MarkdownImageView.java b/markdown/src/main/java/cn/lineai/ui/markdown/MarkdownImageView.java index a05ed097..a8eb7f36 100644 --- a/markdown/src/main/java/cn/lineai/ui/markdown/MarkdownImageView.java +++ b/markdown/src/main/java/cn/lineai/ui/markdown/MarkdownImageView.java @@ -14,6 +14,9 @@ import java.util.Locale; public final class MarkdownImageView extends LinearLayout { + private static final long MAX_DATA_URI_BASE64_CHARS = 5L * 1024 * 1024; + private static final int MAX_DECODED_EDGE_PX = 2048; + public MarkdownImageView(Context context, String destination, String altText) { super(context); setOrientation(VERTICAL); @@ -23,7 +26,7 @@ public MarkdownImageView(Context context, String destination, String altText) { String imageLabel = context.getString(R.string.markdown_image_label); String fallbackText = altText == null || altText.trim().length() == 0 ? imageLabel - : imageLabel.substring(0, imageLabel.length() - 1) + ": " + altText.trim() + "]"; + : imageLabel + ": " + altText.trim(); TextView fallback = LineTheme.text(context, fallbackText, LineTheme.FONT_SM, @@ -54,8 +57,18 @@ private Bitmap decodeBitmap(String url) { if (comma < 0) { return null; } - byte[] bytes = decodeBase64(url.substring(comma + 1)); - return BitmapFactory.decodeByteArray(bytes, 0, bytes.length); + String payload = url.substring(comma + 1); + if (payload.length() > MAX_DATA_URI_BASE64_CHARS) { + // Reject oversized data-URI payloads before decoding to avoid OOM. + return null; + } + byte[] bytes = decodeBase64(payload); + BitmapFactory.Options bounds = new BitmapFactory.Options(); + bounds.inJustDecodeBounds = true; + BitmapFactory.decodeByteArray(bytes, 0, bytes.length, bounds); + BitmapFactory.Options options = new BitmapFactory.Options(); + options.inSampleSize = computeSampleSize(bounds.outWidth, bounds.outHeight); + return BitmapFactory.decodeByteArray(bytes, 0, bytes.length, options); } String path = url.startsWith("file://") ? Uri.parse(url).getPath() : url; if (path == null || !path.startsWith("/")) { @@ -71,6 +84,18 @@ private Bitmap decodeBitmap(String url) { } } + private static int computeSampleSize(int width, int height) { + if (width <= 0 || height <= 0) { + return 1; + } + int sampleSize = 1; + int longEdge = Math.max(width, height); + while (longEdge / sampleSize > MAX_DECODED_EDGE_PX && sampleSize < 128) { + sampleSize <<= 1; + } + return sampleSize; + } + private byte[] decodeBase64(String value) { try { return android.util.Base64.decode(value, android.util.Base64.DEFAULT); diff --git a/markdown/src/main/java/cn/lineai/ui/markdown/MarkdownInlineRenderer.java b/markdown/src/main/java/cn/lineai/ui/markdown/MarkdownInlineRenderer.java index 37f707f0..b1324ebf 100644 --- a/markdown/src/main/java/cn/lineai/ui/markdown/MarkdownInlineRenderer.java +++ b/markdown/src/main/java/cn/lineai/ui/markdown/MarkdownInlineRenderer.java @@ -103,11 +103,10 @@ private void appendNode(SpannableStringBuilder builder, Node node) { String alt = builder.subSequence(start, builder.length()).toString(); builder.delete(start, builder.length()); String imageLabel = context.getString(R.string.markdown_image_label); - builder.append(imageLabel, 0, imageLabel.length() - 1); + builder.append(imageLabel); if (alt.trim().length() > 0) { builder.append(": ").append(alt.trim()); } - builder.append(']'); return; } if (node instanceof HtmlInline) { diff --git a/terminal-provider/src/main/java/cn/lineai/terminalprovider/TerminalProviderService.java b/terminal-provider/src/main/java/cn/lineai/terminalprovider/TerminalProviderService.java index f04d311f..1434292a 100644 --- a/terminal-provider/src/main/java/cn/lineai/terminalprovider/TerminalProviderService.java +++ b/terminal-provider/src/main/java/cn/lineai/terminalprovider/TerminalProviderService.java @@ -2,6 +2,7 @@ import android.os.IBinder; import android.os.RemoteException; +import android.system.Os; import android.util.Log; import cn.lineai.ipc.service.AbstractIpcProviderService; import cn.lineai.ipc.service.IpcServerExecutors; @@ -10,17 +11,25 @@ import java.io.BufferedReader; import java.io.ByteArrayOutputStream; import java.io.File; +import java.io.FileInputStream; import java.io.FileOutputStream; import java.io.IOException; import java.io.InputStreamReader; import java.io.RandomAccessFile; +import java.lang.reflect.Field; import java.nio.charset.StandardCharsets; import java.util.ArrayList; +import java.util.HashMap; +import java.util.HashSet; import java.util.List; +import java.util.Map; +import java.util.Set; import java.util.concurrent.ExecutorService; import java.util.concurrent.Future; import java.util.concurrent.TimeUnit; import java.util.concurrent.TimeoutException; +import java.util.concurrent.atomic.AtomicBoolean; +import java.util.concurrent.atomic.AtomicReference; import org.json.JSONObject; public final class TerminalProviderService extends AbstractIpcProviderService { @@ -29,6 +38,8 @@ public final class TerminalProviderService extends AbstractIpcProviderService { // 单次分块传输的默认上限（1MB），避免 AIDL Binder 事务超限。 private static final int DEFAULT_CHUNK_SIZE = 1024 * 1024; private static final int MAX_CHUNK_SIZE = DEFAULT_CHUNK_SIZE; + // readFile 一次性读入内存的字节上限（32MB），避免超大文件耗尽堆内存。 + private static final long MAX_READ_FILE_BYTES = 32L * 1024L * 1024L; // 进程级共享线程池，由 :ipc 库统一管理。 private final ExecutorService executor = IpcServerExecutors.shared(); @@ -36,6 +47,7 @@ public final class TerminalProviderService extends AbstractIpcProviderService { @Override protected IBinder createBinder() { return new ITerminalProviderService.Stub() { + @Override public String getProviderType() { return "terminal"; @@ -86,6 +98,8 @@ public int executeShell(String command, String cwd, long timeoutMs, workingDir = getFilesDir(); } final File finalWorkingDir = workingDir; + final AtomicReference processRef = new AtomicReference<>(); + final AtomicBoolean finished = new AtomicBoolean(false); Future future = executor.submit(() -> { Process process = null; BufferedReader stdoutReader = null; @@ -95,6 +109,7 @@ public int executeShell(String command, String cwd, long timeoutMs, pb.directory(finalWorkingDir); pb.redirectErrorStream(false); process = pb.start(); + processRef.set(process); stdoutReader = new BufferedReader( new InputStreamReader(process.getInputStream(), StandardCharsets.UTF_8)); stderrReader = new BufferedReader( @@ -106,7 +121,7 @@ public int executeShell(String command, String cwd, long timeoutMs, String line; try { while ((line = finalStdout.readLine()) != null) { - if (callback != null) { + if (callback != null && !finished.get()) { callback.onOutput(line + "\n"); } } @@ -117,7 +132,7 @@ public int executeShell(String command, String cwd, long timeoutMs, String line; try { while ((line = finalStderr.readLine()) != null) { - if (callback != null) { + if (callback != null && !finished.get()) { callback.onOutput(line + "\n"); } } @@ -129,13 +144,13 @@ public int executeShell(String command, String cwd, long timeoutMs, int exitCode = finalProcess.waitFor(); stdoutThread.join(1000); stderrThread.join(1000); - if (callback != null) { + if (callback != null && !finished.get()) { callback.onComplete(exitCode); } return exitCode; } catch (Exception e) { Log.e(TAG, "executeShell failed", e); - if (callback != null) { + if (callback != null && !finished.get()) { try { callback.onError(e.getMessage() == null ? e.toString() : e.getMessage()); } catch (RemoteException ignored) { @@ -150,6 +165,8 @@ public int executeShell(String command, String cwd, long timeoutMs, try { stderrReader.close(); } catch (IOException ignored) {} } if (process != null) { + // 正常完成/异常时只销毁 shell 本身，保留用户启动的后台子进程 + // （如 nohup server &）；进程树击杀仅用于超时路径。 process.destroy(); } } @@ -159,10 +176,17 @@ public int executeShell(String command, String cwd, long timeoutMs, return future.get(effectiveTimeout, TimeUnit.MILLISECONDS); } catch (TimeoutException e) { future.cancel(true); - if (callback != null) { - try { - callback.onError("命令执行超时"); - } catch (RemoteException ignored) { + // 超时：终止整个进程树并把结果状态置为已结束，阻止后续任何回调。 + if (finished.compareAndSet(false, true)) { + Process shell = processRef.get(); + if (shell != null) { + killProcessTree(shell); + } + if (callback != null) { + try { + callback.onError("命令执行超时"); + } catch (RemoteException ignored) { + } } } return -2; @@ -191,6 +215,11 @@ public byte[] readFile(String path) { if (size <= 0L) { return new byte[0]; } + if (size > MAX_READ_FILE_BYTES) { + Log.e(TAG, "readFile: 文件超过读取上限 " + MAX_READ_FILE_BYTES + " 字节: " + path + + " (size=" + size + ")，请使用 readFileChunk 分块读取"); + return new byte[0]; + } if (size > Integer.MAX_VALUE) { Log.e(TAG, "readFile: 文件超过 int 范围: " + path); return new byte[0]; @@ -399,4 +428,127 @@ protected void onProviderDestroy() { // 保留钩子供未来按需扩展。 Log.i(TAG, "TerminalProviderService 销毁"); } + + /** + * 终止指定进程及其全部后代进程（扫描 /proc 下各进程的 stat 文件建立 ppid 链）。 + * 依赖父进程持有该进程树，先杀后代再杀 shell 自身，最后兜底 destroy()。 + */ + private static void killProcessTree(Process process) { + if (process == null) { + return; + } + int pid = processPid(process); + if (pid <= 0) { + process.destroy(); + return; + } + for (int descendant : collectDescendants(pid)) { + safeKill(descendant); + } + safeKill(pid); + process.destroy(); + } + + /** + * 获取 Process 的 pid。Android 的 {@link java.lang.Process} 未公开 pid() 方法， + * 通过反射读取内部 pid 字段（ProcessImpl）。 + */ + private static int processPid(Process process) { + if (process == null) { + return -1; + } + try { + Field field = process.getClass().getDeclaredField("pid"); + field.setAccessible(true); + Object value = field.get(process); + if (value instanceof Integer) { + return (Integer) value; + } + } catch (Throwable ignored) { + } + return -1; + } + + /** + * 扫描 /proc 下各进程的 stat 文件，收集以 rootPid 为根的全部后代进程 pid。 + */ + private static Set collectDescendants(int rootPid) { + Map ppidByPid = new HashMap<>(); + File[] dirs = new File("/proc").listFiles(); + if (dirs != null) { + for (File dir : dirs) { + int pid; + try { + pid = Integer.parseInt(dir.getName()); + } catch (NumberFormatException ignored) { + continue; + } + String stat = readStatLine(dir); + if (stat == null) { + continue; + } + int ppid = parsePpid(stat); + if (ppid > 0 && pid != rootPid) { + ppidByPid.put(pid, ppid); + } + } + } + Set descendants = new HashSet<>(); + boolean changed; + do { + changed = false; + for (Map.Entry entry : ppidByPid.entrySet()) { + int child = entry.getKey(); + int parent = entry.getValue(); + if ((parent == rootPid || descendants.contains(parent)) && descendants.add(child)) { + changed = true; + } + } + } while (changed); + return descendants; + } + + /** + * 从 /proc//stat 的单行内容解析 ppid。格式为：pid (comm) state ppid ... + */ + private static int parsePpid(String stat) { + int close = stat.lastIndexOf(')'); + if (close < 0) { + return -1; + } + String rest = stat.substring(close + 1).trim(); + if (rest.isEmpty()) { + return -1; + } + String[] fields = rest.split("\\s+"); + if (fields.length < 2) { + return -1; + } + try { + return Integer.parseInt(fields[1]); + } catch (NumberFormatException ignored) { + return -1; + } + } + + private static String readStatLine(File procDir) { + File stat = new File(procDir, "stat"); + try (BufferedReader reader = new BufferedReader( + new InputStreamReader(new FileInputStream(stat), StandardCharsets.UTF_8))) { + return reader.readLine(); + } catch (IOException ignored) { + return null; + } + } + + private static void safeKill(int pid) { + if (pid <= 0) { + return; + } + try { + Os.kill(pid, android.system.OsConstants.SIGKILL); + } catch (Throwable ignored) { + // 进程可能已退出或权限受限，忽略。 + } + } } diff --git a/update.md b/update.md index 0b4eea22..fb004fb8 100644 --- a/update.md +++ b/update.md @@ -1,6 +1,6 @@ # 更新日志 -## v1.2.6-rc.1 +## v1.2.6 ### 聊天滚动体验优化（嵌套滚动冲突修复） @@ -24,11 +24,65 @@ - 新增 `BoundedScrollViewTest`（`:ui-theme`，13 断言组）覆盖 `shouldHandleDrag` 全方向/边界组合、`canScrollContent` 边界、`shouldDisallowOnDown` 八种组合、DOWN→MOVE→反向→UP 完整手势序列决策 - 新增 `ToolCallBlockViewSignatureTest`（13 例）覆盖结构/内容签名拆分：仅内容变化签名相等、isError/reviewState/id/name/arguments/projectPath 变化签名不等 - 新增 `ToolCallAgentViewLayoutSignatureTest`（13 例）覆盖 agent 进度 JSON 的 status / tool_call_count / agent_id / output 存在性对布局签名的影响 +- 新增 `MemoryRankerRagInjectionTest` 证明 RAG 排序会选中记忆候选（`MemoryRanker` 相关性注入） + +### 功能与改进 + +- **MCP 协议升级（2025-03-26）** - 自定义 HTTP 工具（`CustomMcpHttpTool`）支持新版 MCP 协议：请求前初始化会话并携带 `Mcp-Session-Id` 会话管理、`Mcp-Protocol-Version` 协议版本头 +- **工具调用去重** - 原生与文本工具调用按签名去重，同一工具调用不会被执行两次 +- **工具调用上限提示** - 达到主流程总工具调用次数上限时给出明确的终止提示（多语言支持） +- **进度监听器透传** - `ToolContext` 新增进度监听器 accessor 并透传给工具执行器，供工具上报流式进度 +- **学习上下文改进** - 学习模式关闭时改用手动记忆构建学习上下文（不再误用自动抽取） +- **响应头暴露** - `SimpleHttpClient` 捕获并暴露响应头，供调用方读取（如 MCP 会话头） +- **Kimi 模型兼容层** - 新增 Kimi/Moonshot 推理策略兼容层（温度下限 1），`OpenAiCompatibleProtocol` 按模型适配 +- **`memory_update` 可见性** - `memory_update` 工具在学习模式开启后可见（`ToolSettingsRepository` 按模式过滤） +- **死代码清理** - 删除从未被调用的 `MemoryExtractionService`（587 行）及其专属 prompt 模板（memory/skill-extraction）与相关字符串，`CLAUDE.md`/`README` 引用同步更新 + +### 全量代码审计修复 + +对全项目（core / data / feature-model / feature-tool / feature-ssh / feature-share / markdown / terminal-provider / app 共 11 个范围）进行了系统性代码审计并逐项核实，修复**高危 2 项、中危 30 项**： + +**文件读取（`file_read`）** +- 修复 KB 分页读取**只返回第一行**：行边界对齐的 `indexOf`/`lastIndexOf` 搜索方向写反，非末页读取被截断到第一个换行 +- `end_kb` 不再硬性限制 50KB，可按文件大小分页读取整文件（单次跨度上限 1MB 防内存暴涨），参数说明同步更新 +- 总行数 / 起始行号统计改为 64KB 分块缓冲读取，替换逐字节 `seek+read`（大文件不再卡死工具线程） +- 消除内容末尾换行的幻影空行号、总行数多计 1 行的问题 + +**Shell 执行（Terminal Provider）** +- 修复执行命令输出**只返回最后一行**：输出回调逐块覆盖缓冲改为累积（SSH 路径同步修正） +- 命令超时后击杀**整个进程树**（扫描 /proc ppid 链，含子进程），并抑制超时后的过期回调；正常完成只销毁 shell 本体，不误杀 `nohup ... &` 后台任务 +- `readFile` 单次读入内存上限 32MB（超大文件走 `readFileChunk` 分块），防 OOM +- Terminal Provider 定位为**开放插件**：手动开启 + 工具层切换执行模式后才被使用，不设签名/包名等调用方校验，兼容 GPLv3 第三方渠道重签名分发 + +**安全加固** +- `UrlPolicy` 私网判定由主机名**前缀匹配**改为字面 IPv4 校验（`10.evil.com`、`192.168.attacker.example` 等公网域名不再被放行明文 HTTP；`10.0.2.2` 模拟器别名不受影响） +- `SimpleHttpClient` 响应体增加 32MB 上限；3 参 `download` 不再传 `Integer.MAX_VALUE`；字节计数 `int` 溢出改 `long` +- 图片生成工具下载上限（12MB）/ 响应上限（24MB）此前定义了但从未生效，现已真正执行 +- Markdown `data:image` base64 解码增加 5MB 上限 + `inSampleSize` 降采样（防恶意/超大内嵌图 OOM） +- 错误日志 `summary` 字段补上脱敏（此前 details/堆栈已脱敏）；`.linecode` 导入的 `fileName` 消毒 + 规范路径校验（防 `../` 穿越读取任意文件） +- 中文图片占位标签 `[图片]` 不再被截成 `[图`（内联渲染与回退文本两处） + +**上下文压缩** +- 修复 Codex/Responses 压缩路径不重置 `TokenUsageTracker`，压缩后基线残留旧 usage 导致反复过度压缩 + +**并发与稳定性** +- `SshConnectionPool` 并发首次借用竞态：`put` 覆盖丢条目、锁永不释放、`release` 对非持有锁 `unlock` 抛 `IllegalMonitorStateException` → `putIfAbsent` + 锁守卫 +- `ToolRegistry` 增加读写锁（后台 `reloadExtensions` 与主线程读取竞争，`ConcurrentModificationException` 风险） +- 会话持久化移出主线程热路径：单线程后台执行器 + latest-wins 合并，切换会话前等待落库（防 ANR 与过期覆盖） +- 工具调用预算只统计**实际执行**的工具（暂停确认/拒绝/跳过不再空耗额度）；Agent 内部工具调用并入全局上限并按轮重置 +- Skill 创建/安装（含 GitHub 下载）、聊天导出（PDF/图片）移出主线程；MCP 请求头改为主线程快照（消除工作线程读 EditText 竞态） + +**UI 与细节** +- `MainChatView` 屏幕缓存上限 12 条并淘汰最旧（动态 screen id 每次导航泄漏 View/Context 的内存问题） +- 存储页刷新按钮误绑定"返回"、统计查询移出主线程；压缩模型"查询列表"按钮实时读取凭据（新增模型不再永远禁用） +- Agent 编辑页分区标题 `%2$s` 误传按钮文案 → 新增 `screen_agent_tools_selected`（三语言） +- PDF 导出宽度换行 off-by-one 丢字符；无障碍点击只点击第一个匹配节点；SKILL.md 读取有界化 +- 新增/加强回归测试：`ToolBuiltinsTest` 23 例（KB 分页多行、中间页连续行号、end_kb>50） ### 版本 -- 版本号升级到 `1.2.6-rc.1` -- `versionCode` 升级到 `29` +- 版本号升级到 `1.2.6`（正式版） +- `versionCode` 升级到 `31` ---