diff --git a/AGENTS.md b/AGENTS.md index ea86087..f9fe87a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -39,15 +39,15 @@ 3. **Link Validity**: 确保所有引用的 URL 可访问且具备明确的上下文价值； 4. **Testing**: 统一在 tests/ 下维护测试用例，区分单元测试（unit）和集成测试（integration），所有测试的本地运行总时间控制在 3 min 以内； 5. **Pre-commit Hooks**: 首次克隆仓库使用 `uv run pre-commit install` 激活本地 Git hooks，使 Ruff lint（含 auto-fix）、Ruff format 及通用代码卫生检查在每次 commit 前自动运行。若 hooks 自动修复了问题，提交会被中断，执行 `git add -p` 审阅修复内容后重新提交即可； - 6. **Issue**: 在 [issue.md](docs/agents/issue.md) 中维护你处理过的 Issue 摘要（问题描述、表因根因、处理方式、后续防范、同类问题影响与处理注意事项等），便于同类问题的跨上下文处理；注意识别相同 Issue，不要同 Issue 多处维护； + 6. **Issue**: 在 [issue.md](docs/.agents/issue.md) 中维护你处理过的 Issue 摘要（问题描述、表因根因、处理方式、后续防范、同类问题影响与处理注意事项等），便于同类问题的跨上下文处理；注意识别相同 Issue，不要同 Issue 多处维护； - **Package Management Standardization (包管理规范)**: 1. **Python**: 严禁使用 pip/poetry，**必须**统一使用 `uv` 进行包管理与脚本执行（如 `uv run`）； 2. **JavaScript/TypeScript**: 严禁使用 npm/yarn，**必须**统一使用 `pnpm` 进行包管理与脚本执行； - **Database Management**: 谨慎操作，数据迁移、测试等操作严禁将现有数据删除，谨慎操作数据迁移的回滚，防止数据被清理。 - **In-depth and close to the facts**：系统且全面地进行问题的分析，深入贴近事实，如有疑问，需先发问，不要乱做决定。 -- **Browser Validation Protocol (浏览器验证准则)**：Agent 不得自行完成、绕过或模拟任何 OAuth / SSO 认证流程，所有登录态均来源于用户已认证的 Chrome 主 profile（真实用户登录态）。完整协议（连通性自检、凭证管理、E2E 集成、实机回归等）详见 [浏览器验证协议](./docs/agents/browser-validation.md)； +- **Browser Validation Protocol (浏览器验证准则)**：Agent 不得自行完成、绕过或模拟任何 OAuth / SSO 认证流程，所有登录态均来源于用户已认证的 Chrome 主 profile（真实用户登录态）。完整协议（连通性自检、凭证管理、E2E 集成、实机回归等）详见 [浏览器验证协议](./docs/.agents/browser-validation.md)； 1. **安全红线**：禁止在 Sandbox 浏览器中跳转 Google 同意屏；禁止以模拟用户或第三方账号替代真实登录态；禁止要求用户在 chat 中粘贴密码、Cookie 或验证码； -- **Knowledge Map (知识索引)**：项目所有文档索引统一维护在 [知识索引](./docs/agents/knowledge-map.md)，并在文档目录变更时即时同步跟新； +- **Knowledge Map (知识索引)**：项目所有文档索引统一维护在 [知识索引](./docs/.agents/knowledge-map.md)，并在文档目录变更时即时同步跟新； - **Documentation Standards (文档规范)**： 1. **Visual Documentation (图文并茂)**: 对于复杂逻辑，优先 **Mermaid Visualization Norms (Mermaid 可视化规范)**，构建”图文并茂”的直观文档； - **色彩语义与兼容性**：为图表节点配置具备语义辨识度的色彩，并确保在深色模式（Dark Mode）下具有极高的对比度与清晰度； @@ -55,4 +55,4 @@ 2. **语言叙事**：用语精准，叙事完备，行文专业，聚焦核心，篇幅精炼，形象具体，体现真实作用与用户吸引性，字数恰当； 3. **Direct Hyperlinking (直接跳转)**: 在文档中提及 Repo 内其他资源（文档/代码）时，**必须**构建可跳转的相对路径链接（如 `[Doc Name](./path.md)`），严禁使用”死文本”引用，以降低信息检索熵； 4. **实操截图**：文档需要引入必要的浏览器实操截图时，需自行通过默认浏览器打开相关页面，通过实操现场截图并保留到文档路径进行文档引用； -- **Reference Specifications (IEEE)**：为保障工程决策的可追溯性与学术严谨性，核心引用需遵循 [reference-specifications.md](docs/agents/reference-specifications.md)IEEE 标准引用格式； +- **Reference Specifications (IEEE)**：为保障工程决策的可追溯性与学术严谨性，核心引用需遵循 [reference-specifications.md](docs/.agents/reference-specifications.md)IEEE 标准引用格式； diff --git a/CHANGELOG.md b/CHANGELOG.md index f745ec1..8947065 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,48 @@ ## [Unreleased] +## [v0.5.2a7](https://github.com/ThreeFish-AI/coding-proxy/releases/tag/v0.5.2a7) - 2026-07-04 + +- style(dashboard): Overview 页 6 张 KPI 卡片（今日请求数 / Token 总量 / 输出 Token / 费用估算 / 故障转移 / 平均延迟）固定单行不折行——`.kpi-grid` 由 `repeat(auto-fit, minmax(200px, 1fr))`（视口 < ~1289px 即减列换行）改为 `repeat(6, minmax(0, 1fr))`，`minmax(0,…)` 覆盖 grid 默认 `min-width:auto` 杜绝内容撑列引发的横向溢出，桌面/笔记本/平板横屏（≥1024px）恒定 6 列单行；`gap` 由 off-grid 的 `5px` 归一为 `--gap-section`（12px）；`.kpi-value` 采用 `clamp(20px, 2.2vw, 32px)` 平滑缩放使中宽度不裁剪数值，费用卡 `#kpi-cost-today` 允许在 `" + "` 处换行使双币种始终完整可见；新增 ≤1023px→3 列、≤480px→2 列 优雅降级，`header` 补 `flex-wrap` 修复窄屏横向溢出（Playwright 8 档宽度实测：1440/1280/1024px 均 6 列单行、无横向滚动、无数值裁剪）(#276)； +- a11y(dashboard): 为 `.range-btn` / `.btn-refresh` / `.page-btn` / `.copy-btn` 补 `:focus-visible` 键盘焦点环，统一交互元素键盘可达性；新增 `@media (prefers-reduced-motion: reduce)` 降级 `fadeInUp` 动画与 hover 过渡，照顾前庭敏感用户 (#276)； +- fix(dashboard): 补定义 `--bg-secondary` / `--text-muted` 两个此前**未定义**的 CSS 变量（消除 `.mc-model-row` / `.mc-empty` 背景与文字色回退瑕疵）；Google Fonts 补载 JetBrains Mono `700` 字重，修正 `.kpi-value` 合成粗体渲染 (#276)； +- docs(readme): 页脚补充作者主页（ThreeFish-AI）与 Apache License 2.0 开源协议超链接； + +## [v0.5.2a6](https://github.com/ThreeFish-AI/coding-proxy/releases/tag/v0.5.2a6) - 2026-07-04 + +- style(branding): 品牌图标（logo/favicon）迭代至「通透青绿圆环 + Tabler prompt `>_` 笔划」（方案 d05）：全透明底 + 青绿渐变（`#22d3ee` → `#2dd4bf`）细圆环（stroke 0.5，仅原 1/3、通透若隐）+ 饱和青绿渐变 `>_`（stroke 2.5），极细环与饱和笔划形成权重对比、`>_` 为视觉主角，青绿承载终端/代码语义；基础笔划改用 Tabler prompt（chevron `M5 7l5 5l-5 5` + 下划线 `M13 17l6 0`，下划线长 6 单位、不会小尺寸点化）；渐变改 `gradientUnits="userSpaceOnUse"` 绝对坐标，修复纯水平下划线（包围盒高度 0）在默认 objectBoundingBox 渐变下不着色而「消失」的问题；`_LOGO_DEFS` / `_PROMPT_PATHS` 单一事实源统一供 SVG favicon、ICO 回退（省略 0.5px 细环、保留饱和笔划）与页面 logo 消费；`.logo` 容器改全透明、SVG 内嵌圆环 + 青绿 drop-shadow 辉光（实机验证）(#274)； +- style(branding): 品牌图标醒目度前置优化（同主题演进过程，已并入本版）：容器由「淡色磨砂卡片」改为「玻璃雾卡」（半透明白 + hairline 描边环）与深色毛玻璃 header 语言统一、消除「亮白贴片」压制感，`>_` 笔划加粗 stroke-width 2→2.5 并撑开间距避免粘连、防止下划线小尺寸退化为圆点 (#274)； + +## [v0.5.2a5](https://github.com/ThreeFish-AI/coding-proxy/releases/tag/v0.5.2a5) - 2026-07-04 + +- fix(dashboard): 供应商队列拖拽由原生 HTML5 DnD 改为 Pointer Events 实现，修复 v0.5.2a4 引入的拖拽排序功能在浏览器中无法拖动的问题；引入 4px 拖拽阈值、乐观实时重排与指针捕获机制，同步支持触屏设备；拖拽完成后顺序立即影响 cc 运行时供应商优先级调度（实机验证）；为 `/dashboard` 响应增加 `Cache-Control: no-cache`，防止浏览器缓存旧内联脚本；补齐前端守卫测试断言 (#271)； +- style(branding): 重设计 logo/favicon 为 `#f5f6fb` 淡色磨砂卡片 + 品牌渐变（`#667eea` → `#764ba2`）`>_` 笔划 + 半透明细描边环，轮廓成为视觉主角、小尺寸下更清晰（Linear/Vercel 风）；抽取 `_LOGO_DEFS` 作为品牌渐变 `` 单一事实源，统一供 SVG favicon、ICO 回退与页面 logo 消费 (#272)； + +## [v0.5.2a4](https://github.com/ThreeFish-AI/coding-proxy/releases/tag/v0.5.2a4) - 2026-07-03 + +- feat(dashboard): 供应商状态模块支持拖拽调整供应商优先级，新增纯重排序端点 `PUT /api/tier-order`（复用 `router.reorder_tiers`，不重置熔断器/配额守卫/rate limit），拖拽后配额与用量统计保持当日值不变；与 `reset -v` 同源、运行时内存生效，失败自动回滚至服务端真实顺序 (#269)； +- feat(session): 扩充标题豁免前缀，新增 `[Session]` 过滤注入式 Prompt 候选； +- chore(pricing): 更新智谱 GLM-5.2 模型定价配置； + +## [v0.5.2a3](https://github.com/ThreeFish-AI/coding-proxy/releases/tag/v0.5.2a3) - 2026-07-02 + +- fix(session): 标题豁免前缀扩展至全层级兜底标题，修复 `[Session] ` 等 Level 4 合成标题无法豁免（原 `title_exempt_prefixes` 仅 Level 1 生效）；改为在 `_extract_session_title` 编排层对 L1/L2/L3/L4 候选统一拦截，命中回退、全豁免返回空串待后续真实输入回填，默认行为零影响 (#267)； + +## [v0.5.2a2](https://github.com/ThreeFish-AI/coding-proxy/releases/tag/v0.5.2a2) - 2026-07-02 + +- feat(session): 新增 Session 标题豁免前缀配置 `title_exempt_prefixes`，过滤注入式 Prompt（典型如 "Write the title in the language the user wrote in..."）被误用为 Session 标题；首条 user 输入经噪声剥离后命中豁免前缀则跳过、继续向后查找 title 候选，Level 2/3/4 回退不受影响 (#265)； + +## [v0.5.2a1](https://github.com/ThreeFish-AI/coding-proxy/releases/tag/v0.5.2a1) - 2026-06-30 + +- fix(zhipu): 将 429/529 兜底退避抖动从 Full Jitter 改为 Equal Jitter（`[0, ceiling]` → `[ceiling/2, ceiling]`），修复 529 过载重试延迟非单调（实测 418.8→1857.7→961.6→3769.7ms）问题，重试延迟呈单调非递减指数形态；429 与 529 共用退避路径同步受益，server `retry-after` 优先级不变 (#263)； +- feat(dashboard): Model Calling 实时监控扩展至全 vendor / 全 model（仅 CC 场景），其他 vendor 在 monitor 模式下仅计数不限流，Zhipu 保留 limited 模式 + FIFO 排队； +- feat(concurrency): 新增 `peak_pending_recent` 最近 10s 排队峰值追踪，瞬时排队释放后前端仍可见"曾排队 N" 余晖徽章； +- perf(dashboard): Model Calling 轮询间隔由 5000ms 缩短至 1500ms，提升瞬时排队可观测性； +- refactor(vendors): `ModelConcurrencyLimiter` 重构为 `ModelConcurrencyController`，统一 monitor / limited 双模式抽象（保留旧名别名）；并发控制由 vendor 内部迁移至 executor 层 `track_in_flight` 包裹，行为对所有 vendor 一致； + +> [!NOTE] +> 本版另补记 v0.5.1 Release Note 遗漏的 4 项并发与监控改动（feat/perf/refactor，代码已随 v0.5.1 上线，文档此前未记录）。 + ## [v0.5.0](https://github.com/ThreeFish-AI/coding-proxy/releases/tag/v0.5.0) - 2026-05-27 > [!IMPORTANT] diff --git a/README.md b/README.md index 6cb7211..0bfb995 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@
-# ⚡ coding-proxy +# coding-proxy coding-proxy **A High-Availability, Transparent, and Smart Multi-Vendor Proxy for Claude Code** @@ -125,7 +125,7 @@ graph RL Client["💻
Client (Claude Code)"]:::client - subgraph CodingProxy["⚡ coding-proxy"] + subgraph CodingProxy["coding-proxy"] direction RL Router["RequestRouter
routing/router.py"]:::router @@ -189,5 +189,5 @@ During our chaotic yet rewarding exploration of engineering practices, we were h ---
- Built with 🧠, ❤️, and an absurd amount of coffee by ThreeFish-AI + Built with 🧠, ❤️, and an absurd amount of coffee by ThreeFish-AI · Released under the Apache License 2.0.
diff --git a/assets/logo.svg b/assets/logo.svg new file mode 100644 index 0000000..1a7b64e --- /dev/null +++ b/assets/logo.svg @@ -0,0 +1 @@ + diff --git a/docs/agents/browser-validation.md b/docs/.agents/browser-validation.md similarity index 100% rename from docs/agents/browser-validation.md rename to docs/.agents/browser-validation.md diff --git a/docs/agents/issue.md b/docs/.agents/issue.md similarity index 74% rename from docs/agents/issue.md rename to docs/.agents/issue.md index c202b8a..82c76f1 100644 --- a/docs/agents/issue.md +++ b/docs/.agents/issue.md @@ -4,6 +4,80 @@ --- +## Session 标题豁免前缀对 `[Session]` 兜底标题不生效（豁免仅 Level 1 生效） + +**问题描述** + +用户在 `session_policies.title_exempt_prefixes` 配置中加入 `"[Session]"`，期望过滤掉 `[Session] claude-opus-4-8` 这类不预期标题（此前已用同机制成功豁免 `Write the title in the language ...` 注入式 Prompt）。但配置后无效——`[Session] claude-opus-4-8` 仍被写入 Session 标题。 + +**表因** + +`[Session] claude-opus-4-8` **不是用户输入**，而是标题提取 Level 4 元数据兜底（`_extract_title_from_metadata`，`f"[Session] {request.model}"`）的产物。用户误以为它来自 user 文本，故尝试用 `title_exempt_prefixes` 豁免。 + +**根因** + +`_extract_session_title`（`routing/executor.py`）采用 4 层回退，但 `exempt_prefixes` **仅在 Level 1**（`_extract_title_from_user_text`）消费——这是原始设计：v0.5.2a1 #265 引入该配置时仅覆盖 L1，docstring 明写「Level 2/3/4 不参与豁免」。Level 2/3/4 的合成标题（`[Tool output]` / `[N Image]` / `[Tool call]` / `[Session]`）完全不查询豁免名单。故用户配的 `"[Session]"` 只进到 L1，对 L4 兜底标题毫无作用——这是「配了却不生效」的根本原因，**非配置错误**。 + +**处理方式** + +在 `_extract_session_title` 编排层引入 `_is_exempt` 闭包，对 L1/L2/L3/L4 每层候选统一做大小写敏感 `startswith` 拦截，命中则向下一层回退、全豁免返回空串。返回空串触发调用方 `if title:` 跳过写库，标题保持空，待后续请求带真实 user 文本时经 `update_empty_session_title` 回填。L1 内部「逐条消息跳过」语义不变，外层闭包作一致性兜底；并补入参空串前缀防御。**默认行为零影响**（不添加前缀则不豁免）；**不**把合成前缀塞进默认 config，避免误杀正常 `[Tool output]` 标题。新增 8 个回归测试（含 mock recorder 验证写库跳过）。 + +**后续防范** + +- **新增合成兜底标题须纳入豁免链路**：未来若在 L2/L3/L4 增加新的 `f"[Xxx] ..."` 合成标题，编排层的 `_is_exempt` 自动覆盖（无需改子函数），但应在 `config.default.yaml` 注释里补充示例前缀，提示用户可豁免。 +- **空串前缀恒真陷阱**：`"".startswith` 恒真会豁免一切。配置层 field_validator、构造器、模块函数入参已三重过滤，任何新增的 startswith 豁免/绑定逻辑都须保留空串防御。 + +**同类问题影响与处理注意事项** + +- **`[Session] ` 触发条件**：请求无可提取的 user TEXT / TOOL_RESULT / IMAGE / tool_names，只剩 model 字段（典型如首条消息全是 `` 噪声、或仅 assistant 消息的续接请求）。排查此类标题应优先确认请求体内容，而非怀疑配置。 +- **豁免语义边界**：豁免 =「跳过该候选、继续向下一层回退」，非「替换为别的标题」。L4 是最后一层，被豁免只能落到空串；依赖标题触发 `_apply_title_based_policy` 供应商自动绑定的场景，空标题不会触发绑定（预期副作用，语义自洽）。 +- **不要把合成前缀写进默认配置**：`[Tool output]` 等对用户有信息量，默认豁免会造成正常工具结果摘要标题消失的回归。 + +--- + +## Zhipu 529 过载重试退避非单调（对齐 429 指数退避语义） + +**问题描述** + +cc 调用 zhipu 返回 529 过载触发重试时，延迟序列呈非单调形态（实测 `418.8ms → 1857.7ms → 961.6ms → 3769.7ms`，第 3 次反而短于第 2 次），不像 429 那样呈现干净的指数退避。 + +**表因** + +`src/coding/proxy/routing/retry.py::calculate_delay` 的兜底抖动为 **Full Jitter**：`delay = random.uniform(0, ceiling)`，每次延迟是 `[0, ceiling]` 区间均匀随机值，本质非单调。实测值逐项精确匹配 `attempt 0/1/2/3` 的 `uniform(0,1000)/(0,2000)/(0,4000)/(0,8000)`。 + +**根因** + +关键认知修正：429 与 529 在代码层面**早已共用同一退避路径** `ZhipuVendor._compute_retry_delay_from_headers`（`vendors/zhipu.py:230-247`），并非"两套规则"。感知差异来自两个因素叠加： + +1. **服务端响应头不对称**：429（限流）响应通常携带 `Retry-After` 头 → 走确定性 server-guided 路径（`retry_after * 1.1`），看起来"干净递增"；529（过载）响应通常**不携带**该头 → 落入 Full Jitter 兜底分支。 +2. **Full Jitter 本身非单调**：即使 429 也一样，只是 429 多数情况有 `Retry-After` 遮蔽了该问题。 + +故真正修复对象是共享的抖动策略，而非给 529 单独加逻辑。 + +**处理方式** + +将 `calculate_delay` 从 Full Jitter 改为 **Equal Jitter**（AWS M. Brooker, "Exponential Backoff And Jitter," 2015）： + +``` +temp = min(initial * backoff^attempt, max) +delay = temp/2 + random.uniform(0, temp/2) # [temp/2, temp] +``` + +Zhipu 配置下区间变为 `[500,1000] → [1000,2000] → [2000,4000] → [4000,8000]`，相邻区间仅边界相切，延迟几乎必然单调非递减；保留抖动以防惊群；429/529 同步受益；`retry-after` 优先级链不动。同步更新 `retry.py` / `zhipu.py` 共 4 处 "Full Jitter" docstring，新增 `tests/test_retry.py` 独立单元测试与 `test_zhipu.py::test_529_equal_jitter_delay_in_expected_band`。 + +**后续防范** + +- **单调性是配置相关的弱保证**：依赖 `backoff_multiplier >= 2.0` 且未触及 `max_delay_ms` 封顶。封顶后（`initial*2^attempt >= max`）各 attempt 区间退化为同一 `[max/2, max]`，单调性丧失。当前 Zhipu `max_retries=4` 触及不到，但未来调参须注意。函数 docstring 已写明此契约边界。 +- **诚实权衡（Brooker 反方观点）**：Brooker 2015 分布式吞吐基准显示 Full Jitter 优于 Equal Jitter。但本场景是单代理（CC）低并发过载恢复，诉求是可解释性（日志递增可预测）而非分布式吞吐最优，CC 并发量级远未达"海量"，差异在噪声内。若未来观测到并发过载下的惊群回归，应优先要求 server 提供 `retry-after` 或评估 Decorrelated Jitter，而非回退 Full Jitter。 + +**同类问题影响与处理注意事项** + +- **双 `RetryConfig` 死代码（重要遗留债）**：仓库存在两个 `RetryConfig`——`routing/retry.py:25`（活跃 dataclass，仅 Zhipu 用）与 `config/resiliency.py:15`（Pydantic 死代码，`config/routing.py:285` 注册为 `retry` 字段但 `VendorTier.retry_config` 从未赋值，全仓库 `grep "retry_config="` 为空）。`docs/arch/routing.md:22` 与 `config-reference.md:95` 引用了不同的 `RetryConfig`，构成认知陷阱。本次修复**未触碰**死代码（超出范围），后续应单独清理。这也是本次"为何选方案 A（直改 `calculate_delay`）而非方案 B（加可配置 jitter_strategy 字段）"的决定性理由——方案 B 会扩大 schema 与运行时的裂缝。 +- **`calculate_delay` 唯一消费者确证**：经全仓库 grep，仅 `vendors/zhipu.py:247` 调用（`routing/__init__.py` 仅 re-export，`tier.retry_config` 字段未激活）。修改其抖动策略爆炸半径仅限 Zhipu，安全。 +- **测试缺口已补**：修复前 `calculate_delay` 无任何独立单元测试（仅通过 zhipu 集成测试间接覆盖，且那些测试要么禁用 jitter、要么走 retry-after 路径），现已新建 `tests/test_retry.py`。 + +--- + ## streaming usage parse failed: 'NoneType' object has no attribute 'get' **问题描述** diff --git a/docs/agents/knowledge-map.md b/docs/.agents/knowledge-map.md similarity index 100% rename from docs/agents/knowledge-map.md rename to docs/.agents/knowledge-map.md diff --git a/docs/agents/reference-specifications.md b/docs/.agents/reference-specifications.md similarity index 100% rename from docs/agents/reference-specifications.md rename to docs/.agents/reference-specifications.md diff --git a/docs/arch/design-patterns.md b/docs/arch/design-patterns.md index 0b8c31b..25b7c20 100644 --- a/docs/arch/design-patterns.md +++ b/docs/arch/design-patterns.md @@ -541,6 +541,8 @@ $$ > **参数默认值**：参见 [配置参考 -- RetryConfig](./config-reference.md#elastic-params) +**Vendor 级应用（Zhipu 429/529 内部退避重试）**：[`vendors/zhipu.py`](../../src/coding/proxy/vendors/zhipu.py) 复用 `RetryConfig` / `calculate_delay()`，在 `send_message` / `send_message_stream` 内对 **429 Rate Limit（限流）** 与 **529 Overloaded（并发过载）** 两类服务端瞬态过载信号做就地指数退避重试（共用 `_BACKOFF_RETRY_STATUS = {429, 529}` 作单一事实源，max=5、1s→2s→4s→8s、Full Jitter，优先尊重 server `retry-after`）。耗尽重试后将状态码原样返回，交由上层 `should_trigger_failover`（§3.6 VendorTier）与 [CircuitBreaker](#circuit-breaker) 处理，从而降低 failover 频率。 + --- ## 3.13 Rate Limit Deadline Tracking（速率限制截止追踪） diff --git a/docs/zh-CN/README.md b/docs/zh-CN/README.md index 4b32986..c8fda53 100644 --- a/docs/zh-CN/README.md +++ b/docs/zh-CN/README.md @@ -2,7 +2,7 @@
-# ⚡ coding-proxy +# coding-proxy coding-proxy **面向 Claude Code 的多供应商高可用透明智能代理** @@ -125,7 +125,7 @@ graph RL Client["💻
Client (Claude Code)"]:::client - subgraph CodingProxy["⚡ coding-proxy"] + subgraph CodingProxy["coding-proxy"] direction RL Router["RequestRouter
routing/router.py"]:::router diff --git a/pyproject.toml b/pyproject.toml index 14dcba1..2d15fd1 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "coding-proxy" -version = "0.5.0" +version = "0.5.2a7" description = "A High-Availability, Transparent, and Smart Multi-Vendor Proxy for Claude Code. Support Claude Plans, GitHub Copilot, Google Antigravity, ZAI/GLM, MiniMax, Qwen, Xiaomi, Kimi, Doubao..." readme = "README.md" requires-python = ">=3.12" diff --git a/src/coding/proxy/config/config.default.yaml b/src/coding/proxy/config/config.default.yaml index d945125..e1bd596 100644 --- a/src/coding/proxy/config/config.default.yaml +++ b/src/coding/proxy/config/config.default.yaml @@ -228,51 +228,51 @@ failover: # === 模型映射 === model_mapping: # GitHub Copilot 可用模型（通过 /api/copilot/models 诊断获取） - - pattern: "claude-sonnet-.*" - vendors: ["copilot"] - target: "claude-sonnet-4.6" - is_regex: true - pattern: "claude-opus-.*" vendors: ["copilot"] target: "claude-opus-4.6" is_regex: true + - pattern: "claude-sonnet-.*" + vendors: ["copilot"] + target: "claude-sonnet-4.6" + is_regex: true - pattern: "claude-haiku-.*" vendors: ["copilot"] target: "claude-haiku-4.5" is_regex: true # Google Antigravity 模型映射 - - pattern: "claude-sonnet-.*" - vendors: ["antigravity"] - target: "Claude Sonnet 4.6" - is_regex: true - pattern: "claude-opus-.*" vendors: ["antigravity"] target: "Claude Opus 4.6 Thinking" is_regex: true + - pattern: "claude-sonnet-.*" + vendors: ["antigravity"] + target: "Claude Sonnet 4.6" + is_regex: true - pattern: "claude-haiku-.*" vendors: ["antigravity"] target: "Gemini 3 Flash" is_regex: true # 智谱 GLM 映射（官方原生 Anthropic 兼容端点） # glm-5v-turbo 完整承载 Claude Code 满血能力（tool calling / thinking / 多模态） - - pattern: "claude-sonnet-.*" + - pattern: "claude-opus-.*" vendors: ["zhipu"] - target: "glm-5v-turbo" + target: "glm-5.2" is_regex: true - - pattern: "claude-opus-.*" + - pattern: "claude-sonnet-.*" vendors: ["zhipu"] target: "glm-5.1" is_regex: true - pattern: "claude-haiku-.*" vendors: ["zhipu"] - target: "glm-4.5-air" + target: "glm-5.1" is_regex: true # MiniMax 模型映射（原生 Anthropic 兼容端点） - - pattern: "claude-sonnet-.*" + - pattern: "claude-opus-.*" vendors: ["minimax"] target: "minimax-m2.7" is_regex: true - - pattern: "claude-opus-.*" + - pattern: "claude-sonnet-.*" vendors: ["minimax"] target: "minimax-m2.7" is_regex: true @@ -281,11 +281,11 @@ model_mapping: target: "minimax-m2.7" is_regex: true # 阿里 Qwen 模型映射（原生 Anthropic 兼容端点） - - pattern: "claude-sonnet-.*" + - pattern: "claude-opus-.*" vendors: ["alibaba"] target: "qwen3.6-plus" is_regex: true - - pattern: "claude-opus-.*" + - pattern: "claude-sonnet-.*" vendors: ["alibaba"] target: "qwen3.6-plus" is_regex: true @@ -294,11 +294,11 @@ model_mapping: target: "qwen3.6-plus" is_regex: true # 小米 MiMo 模型映射（原生 Anthropic 兼容端点） - - pattern: "claude-sonnet-.*" + - pattern: "claude-opus-.*" vendors: ["xiaomi"] target: "mimo-v2-pro" is_regex: true - - pattern: "claude-opus-.*" + - pattern: "claude-sonnet-.*" vendors: ["xiaomi"] target: "mimo-v2-pro" is_regex: true @@ -307,11 +307,11 @@ model_mapping: target: "mimo-v2-pro" is_regex: true # Kimi 模型映射（原生 Anthropic 兼容端点） - - pattern: "claude-sonnet-.*" + - pattern: "claude-opus-.*" vendors: ["kimi"] target: "kimi-k2.5" is_regex: true - - pattern: "claude-opus-.*" + - pattern: "claude-sonnet-.*" vendors: ["kimi"] target: "kimi-k2.5" is_regex: true @@ -339,6 +339,18 @@ model_mapping: # 未配置定价的模型在统计中 Cost 列显示 "-" pricing: # ── Anthropic Claude ── + - vendor: anthropic + model: claude-fable-5 + input_cost_per_mtok: $10.0 + output_cost_per_mtok: $50.0 + cache_write_cost_per_mtok: $12.50 + cache_read_cost_per_mtok: $1.00 + - vendor: anthropic + model: claude-opus-4-8 + input_cost_per_mtok: $5.0 + output_cost_per_mtok: $25.0 + cache_write_cost_per_mtok: $6.25 + cache_read_cost_per_mtok: $0.50 - vendor: anthropic model: claude-opus-4-7 input_cost_per_mtok: $5.0 @@ -521,20 +533,10 @@ pricing: cache_read_cost_per_mtok: $0.05 # ── 智谱 GLM ── - vendor: zhipu - model: glm-4.5-air # 待区分长短上下文定价 - input_cost_per_mtok: ¥0.80 - output_cost_per_mtok: ¥2.00 - cache_read_cost_per_mtok: ¥0.16 - - vendor: zhipu - model: glm-4.7 # 待区分长短上下文定价 - input_cost_per_mtok: ¥2.00 - output_cost_per_mtok: ¥8.00 - cache_read_cost_per_mtok: ¥0.40 - - vendor: zhipu - model: glm-5v-turbo # 待区分长短上下文定价 - input_cost_per_mtok: ¥5.00 - output_cost_per_mtok: ¥22.00 - cache_read_cost_per_mtok: ¥1.20 + model: glm-5.2 # 待区分长短上下文定价 + input_cost_per_mtok: ¥8.00 + output_cost_per_mtok: ¥28.00 + cache_read_cost_per_mtok: ¥2.00 - vendor: zhipu model: glm-5.1 # 待区分长短上下文定价 input_cost_per_mtok: ¥6.00 @@ -550,6 +552,21 @@ pricing: input_cost_per_mtok: ¥4.00 output_cost_per_mtok: ¥18.00 cache_read_cost_per_mtok: ¥1.00 + - vendor: zhipu + model: glm-5v-turbo # 待区分长短上下文定价 + input_cost_per_mtok: ¥5.00 + output_cost_per_mtok: ¥22.00 + cache_read_cost_per_mtok: ¥1.20 + - vendor: zhipu + model: glm-4.7 # 待区分长短上下文定价 + input_cost_per_mtok: ¥2.00 + output_cost_per_mtok: ¥8.00 + cache_read_cost_per_mtok: ¥0.40 + - vendor: zhipu + model: glm-4.5-air # 待区分长短上下文定价 + input_cost_per_mtok: ¥0.80 + output_cost_per_mtok: ¥2.00 + cache_read_cost_per_mtok: ¥0.16 # ── MiniMax ── - vendor: minimax model: minimax-m2.7 @@ -679,4 +696,21 @@ native_api: # tiers: ["copilot", "anthropic", "zhipu"] # # 未配置时（默认），所有 Session 使用全局 tiers 顺序。 -session_policies: [] +session_policies: + policies: [] + # 标题前缀 → 供应商自动绑定。 + # 当 Session 标题以指定前缀开头时，自动将该 Session 绑定到对应供应商。 + # 匹配规则按列表顺序求值，首次匹配生效。 + title_vendor_bindings: + - prefix: "# 目标" + vendor: "zhipu" + # Session 标题豁免前缀：标题提取每一层（user 文本 / 工具结果 / 图片 / 元数据兜底） + # 产出的候选以任一前缀开头则跳过，继续向下一层回退。既用于过滤注入式 Prompt + # （如示例语言指令），也可豁免无信息量的合成兜底标题。大小写敏感 startswith。 + # 例：添加 "[Session]" 可让 [Session] 这类兜底标题不再写入， + # session 标题保持空，待后续真实输入经延迟补写回填。 + # ⚠️ 自定义 yaml 中该字段为整体替换（loader 的 list 合并约定，非追加合并）； + # 写空列表即关闭豁免。加载时会自动 strip + 去空 + 去重。 + title_exempt_prefixes: + - "Write the title in the language" + - "[Session]" diff --git a/src/coding/proxy/config/schema.py b/src/coding/proxy/config/schema.py index 40e5428..ec8ac5f 100644 --- a/src/coding/proxy/config/schema.py +++ b/src/coding/proxy/config/schema.py @@ -44,7 +44,7 @@ # ── 子模块 re-export ──────────────────────────────────────────── from .server import DatabaseConfig, LoggingConfig, ServerConfig # noqa: F401 -from .session_policy import SessionPoliciesConfig # noqa: F401 +from .session_policy import SessionPoliciesConfig, TitleVendorBinding # noqa: F401 from .vendors import ( # noqa: F401 AlibabaConfig, AnthropicConfig, @@ -350,4 +350,5 @@ def compat_state_path(self) -> Path: "NativeApiConfig", # session policy "SessionPoliciesConfig", + "TitleVendorBinding", ] diff --git a/src/coding/proxy/config/session_policy.py b/src/coding/proxy/config/session_policy.py index cb2c512..82aca1a 100644 --- a/src/coding/proxy/config/session_policy.py +++ b/src/coding/proxy/config/session_policy.py @@ -2,7 +2,7 @@ from __future__ import annotations -from pydantic import BaseModel, Field +from pydantic import BaseModel, Field, field_validator class SessionPolicyMatch(BaseModel): @@ -50,6 +50,22 @@ class SessionPolicy(BaseModel): ) +class TitleVendorBinding(BaseModel): + """标题前缀 → 供应商自动绑定规则.""" + + prefix: str = Field( + min_length=1, + description=( + "标题前缀匹配模式（大小写敏感的 startswith 匹配）。" + "禁止空字符串——空前缀会匹配所有标题,导致全量误绑定。" + ), + ) + vendor: str = Field( + min_length=1, + description="匹配后绑定的目标供应商名称", + ) + + class SessionPoliciesConfig(BaseModel): """顶层 Session 策略配置容器.""" @@ -57,3 +73,45 @@ class SessionPoliciesConfig(BaseModel): default_factory=list, description="Session 路由策略列表，按定义顺序求值，首次匹配生效", ) + title_vendor_bindings: list[TitleVendorBinding] = Field( + default_factory=list, + description=( + "标题前缀 → 供应商自动绑定规则。" + "当 Session 标题以指定前缀开头时，自动绑定到对应供应商。" + "匹配规则按列表顺序求值，首次匹配生效。" + ), + ) + title_exempt_prefixes: list[str] = Field( + default_factory=list, + description=( + "Session 标题豁免前缀名单。标题提取的每一层（user TEXT 噪声剥离、" + "TOOL_RESULT 摘要、IMAGE 计数、元数据兜底）产出的候选，若以任一前缀" + "开头，则视为豁免、继续向下一层回退；全部层级均被豁免时返回空串，" + "调用方据此跳过写库，session 标题保持空待后续真实输入回填。" + "既用于过滤注入式 Prompt（如示例语言指令），也可豁免无信息量的合成" + "兜底标题（如 [Session]、[Tool call]、[Tool output]）。" + "大小写敏感的 startswith 匹配，与 title_vendor_bindings 语义一致。" + "加载时会自动 strip + 去空 + 去重；空字符串前缀会被丢弃" + "（防空串 startswith 恒真导致全量误豁免）。" + ), + ) + + @field_validator("title_exempt_prefixes", mode="after") + @classmethod + def _normalize_exempt_prefixes(cls, value: list[str]) -> list[str]: + """归一化豁免前缀：strip + 去空 + 去重保序. + + 空字符串前缀必须丢弃——``"".startswith`` 恒真会豁免一切用户输入， + 导致 Level 1 标题提取永久失效。 + """ + seen: set[str] = set() + result: list[str] = [] + for item in value: + if not isinstance(item, str): + continue + cleaned = item.strip() + if not cleaned or cleaned in seen: + continue + seen.add(cleaned) + result.append(cleaned) + return result diff --git a/src/coding/proxy/convert/vendor_channels.py b/src/coding/proxy/convert/vendor_channels.py index 456a9b3..030f7a3 100644 --- a/src/coding/proxy/convert/vendor_channels.py +++ b/src/coding/proxy/convert/vendor_channels.py @@ -52,6 +52,117 @@ def get_transition_channel( # ── 共享辅助函数 ────────────────────────────────────────────── +def _dump_message_digest( + messages: list[dict[str, Any]], + *, + max_messages: int = 10, + label: str = "", +) -> None: + """输出前 N 条消息的结构摘要（role + content_type_counts），用于过渡管线诊断. + + 仅在 DEBUG 级别输出，且仅在消息数 > 0 时才输出，避免噪声。 + """ + if not messages or not logger.isEnabledFor(logging.DEBUG): + return + parts: list[str] = [f"[{label}]" if label else ""] + limit = min(len(messages), max_messages) + for idx in range(limit): + msg = messages[idx] + role = msg.get("role", "?") if isinstance(msg, dict) else "?" + content = msg.get("content") if isinstance(msg, dict) else None + if isinstance(content, list): + type_counts: dict[str, int] = {} + for b in content: + if isinstance(b, dict): + t = b.get("type", "?") + type_counts[t] = type_counts.get(t, 0) + 1 + else: + type_counts["raw"] = type_counts.get("raw", 0) + 1 + counts_str = ",".join(f"{t}:{c}" for t, c in sorted(type_counts.items())) + elif isinstance(content, str): + counts_str = f"str({len(content)})" + else: + counts_str = "empty" + parts.append(f"{idx}:{role}[{counts_str}]") + if len(messages) > max_messages: + parts.append(f"...+{len(messages) - max_messages}more") + logger.debug("Transition digest %s", " ".join(parts)) + + +def _validate_anthropic_pairing( + messages: list[dict[str, Any]], + *, + context: str = "", +) -> list[str]: + """独立的 Anthropic tool_use/tool_result 配对自检（过渡管线末端执行）. + + 与 ``_enforce_pairing_sanity_pass`` 不同，此函数： + - 不修改消息列表（纯检测） + - 针对每个 assistant + tool_use，精确记录下一条 user 消息中匹配/缺失的 ID + - 发现不一致时输出 WARNING 级别日志含 message index 与具体 ID + + Returns: + 检测到的问题描述列表（空列表表示全部通过）。 + """ + issues: list[str] = [] + for i, msg in enumerate(messages): + if not isinstance(msg, dict) or msg.get("role") != "assistant": + continue + content = msg.get("content") + if not isinstance(content, list): + continue + tool_use_ids = [ + b["id"] + for b in content + if isinstance(b, dict) and b.get("type") == "tool_use" and b.get("id") + ] + if not tool_use_ids: + continue + + next_idx = i + 1 + if next_idx >= len(messages): + issues.append(f"messages[{i}]: assistant with tool_uses at end of list") + continue + + next_msg = messages[next_idx] + if not isinstance(next_msg, dict) or next_msg.get("role") != "user": + issues.append( + f"messages[{i}]: next messages[{next_idx}] is not user " + f"(role={next_msg.get('role') if isinstance(next_msg, dict) else '?'})" + ) + continue + + user_content = next_msg.get("content") + if not isinstance(user_content, list): + user_content = [] + + result_ids = { + b["tool_use_id"] + for b in user_content + if isinstance(b, dict) + and b.get("type") == "tool_result" + and isinstance(b.get("tool_use_id"), str) + } + + missing = [uid for uid in tool_use_ids if uid not in result_ids] + if missing: + issue = ( + f"messages[{i}]: {len(missing)}/{len(tool_use_ids)} tool_use(s) " + f"without tool_result in messages[{next_idx}]: {missing[:5]}" + ) + issues.append(issue) + + if issues: + prefix = f"[{context}] " if context else "" + logger.warning( + "Anthropic pairing validation: %s%d issue(s) found: %s", + prefix, + len(issues), + "; ".join(issues), + ) + return issues + + def strip_thinking_blocks(body: dict[str, Any]) -> int: """从 assistant 消息中移除 thinking/redacted_thinking 块（就地）. @@ -678,6 +789,7 @@ def prepare_zhipu_to_anthropic( 2. 改写 ``srvtoolu_*`` ID 与 ``server_tool_use`` 类型为标准 Anthropic 形式 3. 强制 tool_use/tool_result 配对（单遍正向扫描） 4. 剥离 thinking blocks（signature 无效） + 5. 独立的 Anthropic 兼容性自检（纯检测，不修改，定位 enforce/sanity 未覆盖的边界 case） 所有变换均为幂等操作，安全地在已清理的请求体上重复执行。 @@ -686,6 +798,10 @@ def prepare_zhipu_to_anthropic( """ prepared = copy.deepcopy(body) adaptations: list[str] = [] + msgs = prepared.get("messages", []) + + # ── 过渡管线诊断：变换前快照 ── + _dump_message_digest(msgs, label="zhipu→anthropic.before") # Step 1: 剥离 zhipu 私有 content block 类型（如 server_tool_use_delta） removed_vendor_blocks = _remove_vendor_blocks(prepared, _ZHIPU_VENDOR_BLOCK_TYPES) @@ -696,16 +812,24 @@ def prepare_zhipu_to_anthropic( rewritten, _ = _rewrite_srvtoolu_ids(prepared) if rewritten: adaptations.append(f"rewritten_{rewritten}_srvtoolu_ids") + _dump_message_digest(msgs, label="zhipu→anthropic.after_rewrite") # Step 3: 强制 tool_use/tool_result 配对 - pairing_fixes = enforce_anthropic_tool_pairing(prepared.get("messages", [])) + pairing_fixes = enforce_anthropic_tool_pairing(msgs) if pairing_fixes: adaptations.extend(pairing_fixes) + _dump_message_digest(msgs, label="zhipu→anthropic.after_enforce") # Step 4: 剥离 thinking blocks（zhipu signature 无效） stripped = strip_thinking_blocks(prepared) if stripped: adaptations.append(f"stripped_{stripped}_thinking_blocks") + _dump_message_digest(msgs, label="zhipu→anthropic.after_strip") + + # Step 5: 独立的 Anthropic 兼容性自检（纯检测，不修改） + validation_issues = _validate_anthropic_pairing(msgs, context="zhipu→anthropic") + if validation_issues: + adaptations.append("anthropic_pairing_validation_issues") return prepared, adaptations diff --git a/src/coding/proxy/logging/db.py b/src/coding/proxy/logging/db.py index 8470966..79cd9e2 100644 --- a/src/coding/proxy/logging/db.py +++ b/src/coding/proxy/logging/db.py @@ -335,6 +335,20 @@ async def set_session_title(self, session_key: str, title: str) -> None: ) await self._db.commit() + async def update_empty_session_title(self, session_key: str, title: str) -> None: + """为标题为空的 session 补写标题（幂等,仅覆盖空标题行）. + + 使用 ``AND title = ''`` 条件确保不覆盖已有标题, + 即使行已存在但标题为空也会被更新。 + """ + if not self._db or not title or not session_key: + return + await self._db.execute( + "UPDATE session_meta SET title = ? WHERE session_key = ? AND title = ''", + (title, session_key), + ) + await self._db.commit() + async def get_session_titles(self, session_keys: list[str]) -> dict[str, str]: """批量查询 session 标题.""" if not self._db or not session_keys: diff --git a/src/coding/proxy/routing/executor.py b/src/coding/proxy/routing/executor.py index 4c37f02..ea652ed 100644 --- a/src/coding/proxy/routing/executor.py +++ b/src/coding/proxy/routing/executor.py @@ -11,10 +11,13 @@ import re import time from collections.abc import AsyncIterator -from typing import Any +from typing import TYPE_CHECKING, Any import httpx +if TYPE_CHECKING: + from ..config.session_policy import TitleVendorBinding + from ..vendors.base import ( NoCompatibleVendorError, RequestCapabilities, @@ -50,11 +53,13 @@ CompatibilityStatus, build_canonical_request, ) -from ..model.compat import CanonicalRequest +from ..model.compat import CanonicalMessagePart, CanonicalRequest logger = logging.getLogger(__name__) -_SESSION_TITLE_MAX_LEN = 30 +_SESSION_TITLE_MAX_LEN = 600 +# 回退标题截取长度 — 工具结果等非用户直接输入的摘要上限。 +_FALLBACK_TITLE_MAX_LEN = 80 # Claude Code 注入的"噪声"标签 — 系统级上下文,不应进入 Session 标题。 # 这些标签由 CC harness 在首个 user 消息 content 中拼接,高度同质, @@ -63,11 +68,17 @@ r"<(?Psystem-reminder|user-preferences|" r"local-command-stdout|local-command-stderr|" r"bash-input|bash-stdout|bash-stderr|" - r"ide_selection|stdin|system_instruction)\b[^>]*>" + r"ide_selection|stdin|system_instruction|session|" + r"artifactMetadata|thinking)\b[^>]*>" r".*?", flags=re.DOTALL | re.IGNORECASE, ) +# 标签需要特殊处理:当用户文本在 标签内部时, +# 完整块剥离会连同用户文本一起删除。此模式仅去除外壳标签(保留内容), +# 用于首轮完整剥离结果为空时的二次回退提取。 +_SESSION_TAG_WRAPPER = re.compile(r"]*>", flags=re.IGNORECASE) + # Slash command 子标签:用于识别 /commit、/review 等命令式调用, # 合成"命令 + 参数"式标题。 _CMD_NAME_PATTERN = re.compile(r"(.*?)", flags=re.DOTALL) @@ -77,6 +88,9 @@ r".*?", flags=re.DOTALL ) +# 空白折叠 +_WHITESPACE_PATTERN = re.compile(r"\s+") + def _sanitize_user_text(raw: str) -> str: """剔除 Claude Code 注入的系统级 XML 块,还原真实用户输入。 @@ -85,8 +99,10 @@ def _sanitize_user_text(raw: str) -> str: 1. Slash command 优先识别 — 若检测到 ,合成"命令 + 参数" 式标题(因为残留文本通常为空,直接取标签内容更有意义)。 2. 通用噪声剥离 — 移除已知白名单内的 system-reminder 等标签。 - 3. 残留 command-* 包裹清除 — 兜底去除 command-message 等次要标签。 - 4. 前后空白归一化 — 折叠连续空白为单空格,便于 30 字截断。 + 3. 二次回退 — 若首轮剥离后为空,说明用户文本可能在 + 标签内部;此时仅去除外壳标签,保留内部文本再做噪声剥离。 + 4. 残留 command-* 包裹清除 — 兜底去除 command-message 等次要标签。 + 5. 前后空白归一化 — 折叠连续空白为单空格。 """ if not raw: return "" @@ -104,23 +120,140 @@ def _sanitize_user_text(raw: str) -> str: # 阶段二: 通用噪声剥离 cleaned = _NOISE_TAG_PATTERN.sub("", raw) cleaned = _CMD_WRAPPER_PATTERN.sub("", cleaned) + cleaned = _WHITESPACE_PATTERN.sub(" ", cleaned).strip() + if cleaned: + return cleaned + + # 阶段三: 二次回退 + # 当首轮全部剥离为空时,用户文本很可能被 标签完整包裹。 + # 此时不去除 块,而是仅剥掉外壳标签,保留内部文本后重新剥离。 + if "" +# Level 3: user IMAGE → 计数 → "[1 Image]" / "[N Images]" +# Level 4: 请求元数据 → tool_names / model → "[Tool call] Bash, Read" +# / "[Session] claude-opus-4-8" +# ───────────────────────────────────────────────────────────────── -def _extract_session_title(request: CanonicalRequest) -> str: - """从规范化请求中提取首个用户消息文本作为 session 标题。 - 跳过 Claude Code 注入的系统级 XML 块(system-reminder、user-preferences 等), - 确保标题反映用户真实输入而非高同质化的系统模板。 +def _extract_title_from_user_text( + messages: list[CanonicalMessagePart], + exempt_prefixes: list[str] | None = None, +) -> str: + """Level 1: 从 user TEXT 部分提取经噪声剥离后的首条非空文本. + + ``exempt_prefixes`` 非空时，清洗后文本以任一前缀开头则跳过该输入、继续向后 + 查找——用于过滤注入式 Prompt，避免其被误用作 Session 标题。匹配作用于 + ``_sanitize_user_text`` 清洗后的纯文本（而非带 XML 标签的原始 raw）。 """ - for part in request.messages: + for part in messages: if part.role != "user" or part.type != CanonicalPartType.TEXT: continue cleaned = _sanitize_user_text(part.text) + if not cleaned: + continue + if exempt_prefixes and any(cleaned.startswith(p) for p in exempt_prefixes): + logger.debug( + "Title candidate skipped by exempt prefix %r (snippet=%r)", + next(p for p in exempt_prefixes if cleaned.startswith(p)), + cleaned[:60], + ) + continue + return cleaned[:_SESSION_TITLE_MAX_LEN] + return "" + + +def _extract_title_from_tool_results(messages: list[CanonicalMessagePart]) -> str: + """Level 2: 从 user TOOL_RESULT 部分截取文本摘要.""" + for part in messages: + if part.role != "user" or part.type != CanonicalPartType.TOOL_RESULT: + continue + if not part.text: + continue + cleaned = _sanitize_user_text(part.text) if cleaned: - return cleaned[:_SESSION_TITLE_MAX_LEN] + snippet = cleaned[:_FALLBACK_TITLE_MAX_LEN] + return f"[Tool output] {snippet}" + return "" + + +def _extract_title_from_images(messages: list[CanonicalMessagePart]) -> str: + """Level 3: 统计 user IMAGE 部分数量,生成图片描述标题.""" + count = sum( + 1 for p in messages if p.role == "user" and p.type == CanonicalPartType.IMAGE + ) + if count == 0: + return "" + return f"[{count} Image{'s' if count > 1 else ''}]" + + +def _extract_title_from_metadata(request: CanonicalRequest) -> str: + """Level 4: 从请求元数据 (tool_names / model) 合成兜底标题.""" + if request.tool_names: + names = ", ".join(request.tool_names[:3]) + return f"[Tool call] {names}" + if request.model: + return f"[Session] {request.model}" + return "" + + +def _extract_session_title( + request: CanonicalRequest, + exempt_prefixes: list[str] | None = None, +) -> str: + """从规范化请求中提取 session 标题 — 多层级回退策略。 + + 依次尝试: user TEXT 噪声剥离 → TOOL_RESULT 摘要 → IMAGE 计数 → 元数据兜底。 + 任一层级产出候选且未被豁免即返回,确保 Dashboard 尽可能展示有辨识度的标题。 + + ``exempt_prefixes`` 对所有层级统一生效：任一层产出的候选标题若以任一前缀 + 开头，则视为豁免、继续向下一层回退。Level 1 内部仍保留「逐条消息跳过」 + 语义（清洗后文本以豁免前缀开头则跳过该条、继续查找下一条 user TEXT）； + 本函数在编排层对每层最终候选再做一次 ``_is_exempt`` 拦截，使 Level 2/3/4 + 的合成标题（如 ``[Tool output]``、``[N Image(s)]``、``[Tool call]``、 + ``[Session]``）也能被用户显式豁免。全部层级均被豁免时返回空串，调用方 + ``if title:`` 据此跳过写库，session 标题保持空，待后续请求经 + ``update_empty_session_title`` 回填有意义标题。默认 ``None`` 时行为与历史一致。 + """ + # 防御性过滤空串前缀："" startswith 恒真会误豁免一切。入参虽经配置层与 + # 构造器两层归一化，模块级直接调用（如测试）仍可能传入 [""]，此处兜底。 + prefixes = [p for p in (exempt_prefixes or []) if p] + + def _is_exempt(candidate: str) -> bool: + """候选标题是否命中任一豁免前缀（大小写敏感 startswith）。""" + return bool(prefixes) and any(candidate.startswith(p) for p in prefixes) + + messages = request.messages + # Level 1: 内部已对每条 user TEXT 做豁免跳过；外层 _is_exempt 对其最终候选 + # 再做一次拦截（正常路径下冗余，作为 _sanitize_user_text 一致性兜底）。 + title = _extract_title_from_user_text(messages, exempt_prefixes=prefixes) + if title and not _is_exempt(title): + return title[:_SESSION_TITLE_MAX_LEN] + # Level 2 / Level 3: 候选为合成标题，统一经 _is_exempt 拦截后回退。 + for extractor in ( + _extract_title_from_tool_results, + _extract_title_from_images, + ): + title = extractor(messages) + if title and not _is_exempt(title): + return title[:_SESSION_TITLE_MAX_LEN] + # Level 4: 签名不同（依赖 request 元数据）。被豁免则落到末尾返回空串， + # 使调用方 if title: 跳过写库、保留空标题待后续回填。 + title = _extract_title_from_metadata(request) + if title and not _is_exempt(title): + return title[:_SESSION_TITLE_MAX_LEN] return "" @@ -525,6 +658,8 @@ def __init__( session_manager: RouteSessionManager, reauth_coordinator: Any | None = None, session_policy_resolver: SessionPolicyResolver | None = None, + title_vendor_bindings: list[TitleVendorBinding] | None = None, + title_exempt_prefixes: list[str] | None = None, ) -> None: self._router = router self._tiers = tiers @@ -532,6 +667,15 @@ def __init__( self._session_mgr = session_manager self._reauth_coordinator = reauth_coordinator self._policy_resolver = session_policy_resolver or SessionPolicyResolver() + self._title_vendor_bindings = title_vendor_bindings or [] + # 豁免前缀名单：构造期二次归一化（strip + 去空），双重防御配置层 field_validator。 + # 即便绕过配置层直接构造 executor，也能避免空串 startswith 恒真与纯空白失配。 + self._title_exempt_prefixes: list[str] = [ + stripped + for p in (title_exempt_prefixes or []) + if isinstance(p, str) and (stripped := p.strip()) + ] + self._validate_title_vendor_bindings() # Tier 名称 → OAuth provider 名称的映射 self._tier_provider_map: dict[str, str] = { @@ -539,6 +683,26 @@ def __init__( "antigravity": "google", } + def _validate_title_vendor_bindings(self) -> None: + """启动期校验标题绑定引用的 vendor 均存在,缺失则告警. + + 与手动绑定 API（拒绝未知 vendor）的语义对齐：此处不硬失败， + 仅记录警告——避免单条误配置阻断整个代理启动；运行时 + `_resolve_effective_tiers` 会静默跳过未知 vendor 回退默认顺序。 + """ + if not self._title_vendor_bindings: + return + valid = {t.name for t in self._tiers} + for binding in self._title_vendor_bindings: + if binding.vendor not in valid: + logger.warning( + "title_vendor_bindings 引用了未知 vendor %r（前缀 %r）；" + "可用 vendor: %s。该绑定将在运行时被静默跳过。", + binding.vendor, + binding.prefix, + sorted(valid), + ) + # ── 公开执行入口 ────────────────────────────────────── def _resolve_effective_tiers(self, session_key: str) -> list[VendorTier]: @@ -565,6 +729,38 @@ def _resolve_effective_tiers(self, session_key: str) -> list[VendorTier]: seen.add(tier.name) return ordered + def _extract_session_title(self, request: CanonicalRequest) -> str: + """封装模块级标题提取，注入实例持有的豁免前缀名单. + + 将 ``self._title_exempt_prefixes`` 透传给模块级 ``_extract_session_title``， + 使 Level 1 能跳过以豁免前缀开头的注入式 Prompt。4 处标题提取调用点统一 + 经此方法，避免重复透传（零透传污染）。 + """ + return _extract_session_title( + request, exempt_prefixes=self._title_exempt_prefixes + ) + + def _apply_title_based_policy(self, session_key: str, title: str) -> None: + """根据 Session 标题前缀自动绑定供应商. + + 当标题以预配置的前缀开头时，通过 SessionPolicyResolver.upsert() + 将该 Session 绑定到指定供应商，后续请求无需再走默认路由。 + + 仅在新 Session 首次提取标题时调用，避免覆盖手动绑定的策略。 + """ + if not title or not self._title_vendor_bindings: + return + for binding in self._title_vendor_bindings: + if title.startswith(binding.prefix): + self._policy_resolver.upsert(session_key, [binding.vendor]) + logger.info( + "Session title prefix %r matched → auto-bind to %s (session=%s)", + binding.prefix, + binding.vendor, + session_key[:12], + ) + return + def _prepare_body_for_tier( self, body: dict[str, Any], @@ -658,11 +854,19 @@ async def execute_stream( canonical_request.trace_id, ) if is_new_session: - title = _extract_session_title(canonical_request) + title = self._extract_session_title(canonical_request) if title: await self._recorder.set_session_title( canonical_request.session_key, title ) + self._apply_title_based_policy(canonical_request.session_key, title) + else: + # 延迟标题补写: 若 session 尚无标题,尝试从当前请求中提取并回写。 + title = self._extract_session_title(canonical_request) + if title: + await self._recorder.update_empty_session_title( + canonical_request.session_key, title + ) incompatible_reasons: list[str] = [] effective_tiers = self._resolve_effective_tiers(canonical_request.session_key) last_idx = len(effective_tiers) - 1 @@ -689,15 +893,17 @@ async def execute_stream( tier.name, failed_tier_name, session_record, body ) body_for_tier = self._prepare_body_for_tier(body, tier, source_vendor) - async for chunk in tier.vendor.send_message_stream( - body_for_tier, headers - ): - parse_usage_from_chunk( - chunk, - usage, - vendor_label=_VENDOR_PROTOCOL_LABEL_MAP.get(tier.name), - ) - yield chunk, tier.name + _mapped_model = tier.vendor.map_model(body.get("model", "")) + async with tier.vendor.track_in_flight(_mapped_model): + async for chunk in tier.vendor.send_message_stream( + body_for_tier, headers + ): + parse_usage_from_chunk( + chunk, + usage, + vendor_label=_VENDOR_PROTOCOL_LABEL_MAP.get(tier.name), + ) + yield chunk, tier.name info = self._recorder.build_usage_info(usage) if has_missing_input_usage_signals(info): @@ -835,11 +1041,19 @@ async def execute_message( canonical_request.trace_id, ) if is_new_session: - title = _extract_session_title(canonical_request) + title = self._extract_session_title(canonical_request) if title: await self._recorder.set_session_title( canonical_request.session_key, title ) + self._apply_title_based_policy(canonical_request.session_key, title) + else: + # 延迟标题补写: 若 session 尚无标题,尝试从当前请求中提取并回写。 + title = self._extract_session_title(canonical_request) + if title: + await self._recorder.update_empty_session_title( + canonical_request.session_key, title + ) incompatible_reasons: list[str] = [] effective_tiers = self._resolve_effective_tiers(canonical_request.session_key) last_idx = len(effective_tiers) - 1 @@ -863,7 +1077,9 @@ async def execute_message( tier.name, failed_tier_name, session_record, body ) body_for_tier = self._prepare_body_for_tier(body, tier, source_vendor) - resp = await tier.vendor.send_message(body_for_tier, headers) + _mapped_model = tier.vendor.map_model(body.get("model", "")) + async with tier.vendor.track_in_flight(_mapped_model): + resp = await tier.vendor.send_message(body_for_tier, headers) if resp.status_code < 400: duration = int((time.monotonic() - start) * 1000) diff --git a/src/coding/proxy/routing/retry.py b/src/coding/proxy/routing/retry.py index c407cc1..af4971c 100644 --- a/src/coding/proxy/routing/retry.py +++ b/src/coding/proxy/routing/retry.py @@ -1,4 +1,4 @@ -"""传输层重试策略 — 指数退避与 Full Jitter. +"""传输层重试策略 — 指数退避与 Equal Jitter. 与 Circuit Breaker 正交互： - Retry 处理瞬态网络抖动（秒级恢复） @@ -68,15 +68,24 @@ def is_retryable_status(status_code: int) -> bool: def calculate_delay(attempt: int, cfg: RetryConfig) -> float: - """计算第 N 次重试的延迟（毫秒），含指数退避和 Full Jitter. + """计算第 N 次重试的延迟（毫秒），含指数退避和 Equal Jitter. + + Equal Jitter 策略: temp = min(initial * backoff^attempt, max); + delay = temp/2 + random(0, temp/2)，落在 [temp/2, temp]。 + 相较 Full Jitter (random(0, temp))，Equal Jitter 保留一半固定基线， + 使相邻重试的延迟区间仅边界相切，呈现单调非递减的指数形态， + 同时保留抖动以防惊群。 + + 契约边界：单调非递减依赖 ``backoff_multiplier >= 2.0`` 且未触及 + ``max_delay_ms`` 封顶；封顶后各 attempt 区间退化为同一 [max/2, max] + （当前 Zhipu ``max_retries=4`` 触及不到该边界）。 - Full Jitter 策略: delay = random(0, min(initial * backoff^attempt, max)) 参考: AWS "Exponential Backoff And Jitter" (Marc Brooker, 2015) """ delay = cfg.initial_delay_ms * (cfg.backoff_multiplier**attempt) delay = min(delay, cfg.max_delay_ms) if cfg.jitter: - delay = random.uniform(0, delay) + delay = delay / 2 + random.uniform(0, delay / 2) return delay diff --git a/src/coding/proxy/routing/router.py b/src/coding/proxy/routing/router.py index 32757a8..3ea162c 100644 --- a/src/coding/proxy/routing/router.py +++ b/src/coding/proxy/routing/router.py @@ -14,6 +14,7 @@ from typing import TYPE_CHECKING, Any if TYPE_CHECKING: + from ..config.session_policy import TitleVendorBinding from ..pricing import PricingTable from .executor import _RouteExecutor @@ -38,6 +39,8 @@ def __init__( reauth_coordinator: Any | None = None, compat_session_store: CompatSessionStore | None = None, session_policy_resolver: SessionPolicyResolver | None = None, + title_vendor_bindings: list[TitleVendorBinding] | None = None, + title_exempt_prefixes: list[str] | None = None, ) -> None: if not tiers: raise ValueError("至少需要一个供应商层级") @@ -56,6 +59,8 @@ def __init__( session_manager=self._session_mgr, reauth_coordinator=reauth_coordinator, session_policy_resolver=session_policy_resolver, + title_vendor_bindings=title_vendor_bindings, + title_exempt_prefixes=title_exempt_prefixes, ) def set_pricing_table(self, table: PricingTable) -> None: diff --git a/src/coding/proxy/routing/usage_recorder.py b/src/coding/proxy/routing/usage_recorder.py index 8887c09..e83a8fe 100644 --- a/src/coding/proxy/routing/usage_recorder.py +++ b/src/coding/proxy/routing/usage_recorder.py @@ -33,6 +33,11 @@ async def set_session_title(self, session_key: str, title: str) -> None: if self._token_logger: await self._token_logger.set_session_title(session_key, title) + async def update_empty_session_title(self, session_key: str, title: str) -> None: + """为标题为空的 session 补写标题（委托给 TokenLogger）.""" + if self._token_logger: + await self._token_logger.update_empty_session_title(session_key, title) + # ── 用量信息构建 ────────────────────────────────────── @staticmethod diff --git a/src/coding/proxy/server/app.py b/src/coding/proxy/server/app.py index 5ce8011..cf8aad4 100644 --- a/src/coding/proxy/server/app.py +++ b/src/coding/proxy/server/app.py @@ -161,6 +161,8 @@ def create_app(config: ProxyConfig | None = None) -> FastAPI: reauth_coordinator, compat_session_store, session_policy_resolver=SessionPolicyResolver(config.session_policies.policies), + title_vendor_bindings=config.session_policies.title_vendor_bindings, + title_exempt_prefixes=config.session_policies.title_exempt_prefixes, ) app = FastAPI(title="coding-proxy", version=__version__, lifespan=lifespan) diff --git a/src/coding/proxy/server/dashboard.py b/src/coding/proxy/server/dashboard.py index 75dd812..2694440 100644 --- a/src/coding/proxy/server/dashboard.py +++ b/src/coding/proxy/server/dashboard.py @@ -11,28 +11,111 @@ from ..logging.db import TimePeriod +# ── 品牌图标：Tabler prompt 核心笔划（> 与 _）+ 通透青绿圆环 ────────────── +# 设计（方案 d05）：全透明底 + 青绿渐变细圆环（stroke 0.5，仅原 1/3，通透若隐）+ +# 饱和青绿渐变 >_ 笔划（stroke 2.5）。极细环与饱和笔划形成权重对比，>_ 为绝对主角； +# 青绿（#22d3ee -> #2dd4bf）承载终端/代码语义，通透明快。基础笔划取自 Tabler prompt +# （chevron M5 7l5 5l-5 5 + 下划线 M13 17l6 0，下划线长 6 单位、不会小尺寸点化）。 +# 渐变用 gradientUnits="userSpaceOnUse" + 绝对坐标：纯水平下划线包围盒高度为 0，若用 +# 默认 objectBoundingBox 渐变会映射失效而不着色（下划线消失），userSpaceOnUse 规避。 +# 单一事实源：_PROMPT_PATHS 与 _LOGO_DEFS 同时供 favicon SVG（f-string）与页面 +# logo（{{PROMPT}} / {{LOGO_DEFS}} 模板替换）消费。 +_BRAND_FROM, _BRAND_TO = "#22d3ee", "#2dd4bf" +_LOGO_DEFS = ( + '' + f'' + f'' + "" +) +_PROMPT_PATHS = '' + + +# ── Favicon (SVG, 现代浏览器主选) ────────────────────────────────────────── +def _build_favicon_svg() -> str: + """生成 24×24 SVG favicon：通透青绿圆环 + 品牌渐变 prompt 笔划（方案 d05）. + + 全透明底 + 青绿渐变细圆环（``r=11``、``stroke-width=0.5``、``stroke-opacity=.55``， + 通透若隐）+ 饱和青绿渐变 ``>`` 与 ``_`` 笔划（``stroke-width=2.5``）。极细环与饱和 + 笔划形成权重对比，``>_`` 为视觉主角。独立 SVG 文档无 CSS 上下文，``currentColor`` + 不可靠，故渐变复用 ``_LOGO_DEFS``（userSpaceOnUse，规避水平下划线不着色）单一事实源。 + """ + return ( + '' + f"{_LOGO_DEFS}" + '' + f'{_PROMPT_PATHS}' + "" + ) + -# ── Favicon (16×16, 蓝紫渐变) ──────────────────────────────────────────── +# ── Favicon (ICO, 32×32 光栅回退) ───────────────────────────────────────── def _build_favicon() -> bytes: - """程序化生成 16×16 ICO，蓝紫渐变与 Dashboard Logo 一致.""" + """程序化生成 32×32 ICO：全透明底 + 青绿渐变 prompt 笔划（旧浏览器/Safari 回退）. + + 项目无 Pillow/cairosvg 等图像库，故以纯 Python 按像素光栅化：全透明底 + 青绿渐变 + （#22d3ee -> #2dd4bf）``>`` 折线与 ``_`` 下划线。沿用「全零 AND-mask + per-pixel + alpha」透明方案（现代渲染器按 alpha 通道处理透明）。方案 d05 的细圆环（0.5px）在 + 32px 下不足 1 像素、无法渲染，故 ICO 省略圆环、仅保留饱和 ``>_`` 笔划（符合「图标 + 为主角、环极弱」的设计意图），完整外观由 SVG 主版承载。 + """ + import math import struct - width, height = 16, 16 + width = height = 32 + scale = width / 24.0 # SVG 24 单位 -> 像素 + + # 图标笔划青绿渐变：#22d3ee -> #2dd4bf（与 _LOGO_DEFS 同源） + r0, g0, b0 = 0x22, 0xD3, 0xEE + r1, g1, b1 = 0x2D, 0xD4, 0xBF + + # 像素缓冲：[B, G, R, A]，默认全透明（全透底，无容器填充） + buf: list[list[list[int]]] = [ + [[0, 0, 0, 0] for _ in range(width)] for _ in range(height) + ] + + def brand_at(px: float, py: float) -> tuple[int, int, int]: + """按对角参数 t 返回 (px,py) 处的青绿渐变 RGB（与 SVG url(#cpBrand) 同向）.""" + t = (px + py) / (width + height - 2) + return ( + int(r0 + (r1 - r0) * t), + int(g0 + (g1 - g0) * t), + int(b0 + (b1 - b0) * t), + ) + + def stamp(px: float, py: float) -> None: + """以 (px, py) 为中心盖 ~3px 青绿渐变方块（笔划加粗，保证小尺寸可辨识）.""" + ix, iy = int(round(px)), int(round(py)) + cr, cg, cb = brand_at(px, py) + for oy in (-1, 0, 1): + for ox in (-1, 0, 1): + nx, ny = ix + ox, iy + oy + if 0 <= nx < width and 0 <= ny < height: + buf[ny][nx] = [cb, cg, cr, 255] + + def line(x0: float, y0: float, x1: float, y1: float) -> None: + """稠密采样 + stamp 形成青绿渐变粗线段.""" + dist = math.hypot(x1 - x0, y1 - y0) + steps = max(1, int(dist * 3)) + for i in range(steps + 1): + s = i / steps + stamp(x0 + (x1 - x0) * s, y0 + (y1 - y0) * s) + + # prompt 笔划（SVG 24 单位 -> 像素） + # > 折线 (5,7)->(10,12)->(5,17)；_ 下划线 (13,17)->(19,17) + line(5 * scale, 7 * scale, 10 * scale, 12 * scale) + line(10 * scale, 12 * scale, 5 * scale, 17 * scale) + line(13 * scale, 17 * scale, 19 * scale, 17 * scale) + + # 3) 打包 ICO（BMP bottom-up；全零 AND-mask，信任 per-pixel alpha 实现圆角透明） pixel_rows: list[bytes] = [] - cx, cy = width / 2.0, height / 2.0 for y in range(height - 1, -1, -1): # BMP bottom-up row = bytearray() for x in range(width): - dx = x - cx + 0.5 - dy = y - cy + 0.5 - if dx * dx + dy * dy > (width / 2.0) ** 2: - row.extend([0, 0, 0, 0]) # 圆外透明 - else: - t = (x + (height - 1 - y)) / (width + height - 2) - r = int(88 + (188 - 88) * t) - g = int(166 + (140 - 166) * t) - b = 255 - row.extend([b, g, r, 255]) # BGRA + b, g, r, a = buf[y][x] + row.extend([b, g, r, a]) # BGRA pixel_rows.append(bytes(row)) bmp_hdr = struct.pack( @@ -50,6 +133,7 @@ def _build_favicon() -> bytes: _FAVICON_ICO: bytes = _build_favicon() +_FAVICON_SVG: str = _build_favicon_svg() logger = logging.getLogger(__name__) @@ -61,21 +145,24 @@ def _build_favicon() -> bytes: Coding Proxy Dashboard + - +
- +

Coding Proxy Dashboard

v-.-.-
@@ -787,7 +934,7 @@ def _build_favicon() -> bytes:
-
+
近 7 天 Token 用量（按 Vendor / 模型）
@@ -1229,12 +1376,15 @@ def _build_favicon() -> bytes: } function updateVendorStatus(status) { + // 拖拽进行中 / 重排序 POST 未决时跳过重渲染，避免抢占正在操作的 DOM + if (_tierDrag.active || _tierDrag.inFlight) return; const tiers = status.tiers || []; const list = document.getElementById('vendor-list'); if (!tiers.length) { list.innerHTML = '
🔌
无供应商数据
'; return; } + const reorderable = tiers.length >= 2; list.innerHTML = tiers.map(tier => { const cb = tier.circuit_breaker || {}; const cbClass = cbStateClass(cb.state); @@ -1248,8 +1398,13 @@ def _build_favicon() -> bytes: const rlInfo = tier.rate_limit || {}; const rlHtml = rlInfo.limited ? `限速中` : ''; - return `
+ // Pointer Events 重排：仅需 data-vendor 定位；手柄作为拖拽发起点（无原生 draggable） + const dragAttrs = ` data-vendor="${tier.name}"`; + const handle = reorderable ? `
⠿
` : ''; + + return `
+ ${handle}
${initial}
${tier.name}
@@ -1262,6 +1417,145 @@ def _build_favicon() -> bytes: }).join(''); } +// ── 供应商状态：拖拽调整优先级（Pointer Events 重排，运行时生效，不重置配额） ──── +// 采用 Pointer Events 而非原生 HTML5 DnD：后者触屏不支持、跨浏览器易「拿不起来」， +// 是业界公认脆弱的重排序方案（故 SortableJS / dnd-kit 等均改用指针事件）。 +const _tierDrag = { + active: false, // 已越过阈值、进入拖拽 + srcName: null, // 被拖拽 vendor 名 + srcEl: null, // 被拖拽行 + pointerId: null, // 捕获的指针 ID + startY: 0, // pointerdown 起点 Y + origOrder: null, // 拖拽开始时的顺序快照（无变化则跳过 PUT） + inFlight: false, // PUT 未决 +}; +const _TIER_DRAG_THRESHOLD = 4; // px，越过才判定为拖拽（否则视作点击） + +function _tierCurrentOrder(listEl) { + return Array.from(listEl.querySelectorAll('.vendor-item')) + .map(function(el) { return el.dataset.vendor; }) + .filter(Boolean); +} + +// 依据指针 Y 与各兄弟行中点比较，即时(乐观)重排 DOM +function _tierDragMoveTo(listEl, clientY) { + const dragged = _tierDrag.srcEl; + if (!dragged) return; + const siblings = Array.from(listEl.querySelectorAll('.vendor-item')) + .filter(function(el) { return el !== dragged; }); + for (var i = 0; i < siblings.length; i++) { + const rect = siblings[i].getBoundingClientRect(); + if (clientY < rect.top + rect.height / 2) { + if (dragged !== siblings[i] && dragged.nextElementSibling !== siblings[i]) { + listEl.insertBefore(dragged, siblings[i]); + } + return; + } + } + if (listEl.lastElementChild !== dragged) listEl.appendChild(dragged); // 落到末尾 +} + +function _tierDragReset() { + if (_tierDrag.srcEl) { + if (_tierDrag.pointerId != null) { + try { _tierDrag.srcEl.releasePointerCapture(_tierDrag.pointerId); } catch (_) {} + } + _tierDrag.srcEl.classList.remove('dragging'); + } + _tierDrag.active = false; + _tierDrag.srcName = null; + _tierDrag.srcEl = null; + _tierDrag.pointerId = null; + _tierDrag.startY = 0; + _tierDrag.origOrder = null; +} + +function initTierDrag() { + // 指针事件委托绑定在静态容器 #vendor-list 上一次；子节点重渲染后仍生效 + const list = document.getElementById('vendor-list'); + if (!list || list.dataset.dndBound === '1') return; + list.dataset.dndBound = '1'; + + list.addEventListener('pointerdown', function(e) { + if (e.button != null && e.button !== 0) return; // 仅主指针/左键 + const handle = e.target && e.target.closest && e.target.closest('.drag-handle'); + if (!handle) return; // 仅从手柄发起，避免误触与文本选择 + const item = handle.closest('.vendor-item'); + if (!item || !item.dataset.vendor) return; + if (list.querySelectorAll('.vendor-item').length < 2) return; + _tierDrag.srcEl = item; + _tierDrag.srcName = item.dataset.vendor; + _tierDrag.pointerId = e.pointerId; + _tierDrag.startY = e.clientY; + try { item.setPointerCapture(e.pointerId); } catch (_) {} // 捕获后 DOM 重排不丢事件 + e.preventDefault(); // 阻止文本选择 + }); + + list.addEventListener('pointermove', function(e) { + if (!_tierDrag.srcEl || e.pointerId !== _tierDrag.pointerId) return; + if (!_tierDrag.active) { + if (Math.abs(e.clientY - _tierDrag.startY) < _TIER_DRAG_THRESHOLD) return; + _tierDrag.active = true; // 越阈值 → 正式进入拖拽 + _tierDrag.origOrder = _tierCurrentOrder(list); + _tierDrag.srcEl.classList.add('dragging'); + } + e.preventDefault(); + _tierDragMoveTo(list, e.clientY); + }); + + function _tierDragFinish(commit) { + if (!_tierDrag.srcEl) return; + const wasActive = _tierDrag.active; + const order = _tierCurrentOrder(list); + const orig = _tierDrag.origOrder; + _tierDragReset(); + if (!wasActive) return; // 仅点击手柄未拖动 → 空操作 + if (!commit) { _tierRevertToList(); return; } // 取消 → 回滚服务端真实顺序 + if (orig && order.join(',') === orig.join(',')) return; // 顺序未变 → 跳过 PUT + persistTierOrder(order); // 提交新顺序（PUT + 失败回滚） + } + + list.addEventListener('pointerup', function(e) { + if (!_tierDrag.srcEl || e.pointerId !== _tierDrag.pointerId) return; + _tierDragFinish(true); + }); + list.addEventListener('pointercancel', function(e) { + if (!_tierDrag.srcEl || e.pointerId !== _tierDrag.pointerId) return; + _tierDragFinish(false); + }); +} + +function persistTierOrder(names) { + if (_tierDrag.inFlight) return; // 防并发：上一次未决则忽略 + if (!Array.isArray(names) || names.length < 2) return; + _tierDrag.inFlight = true; + fetch('/api/tier-order', { + method: 'PUT', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ vendors: names }) + }).then(function(res) { + _tierDrag.inFlight = false; // 先清守卫，允许随后的 refreshOverview 重渲染 + if (res.ok) { + // 服务端已更新顺序；刷新以同步请求趋势图等的排序 + if (!_tierDrag.active) refreshOverview().catch(function() {}); + } else { + console.error('tier-order rejected:', res.status); + _tierRevertToList(); + } + }).catch(function(e) { + _tierDrag.inFlight = false; + console.error('tier-order failed:', e); + _tierRevertToList(); + }); +} + +function _tierRevertToList() { + // 回滚到服务端真实顺序（PUT 失败 → 服务端顺序未变） + fetchJSON('/api/status').then(function(status) { + updateVendorStatus(status); + }).catch(function() {}); +} + // ── Model Calling 实时状态 ──────────────────────────────── function updateModelCalling(status) { var wrap = document.getElementById('model-calling-wrap'); @@ -1282,10 +1576,12 @@ def _build_favicon() -> bytes: models.push({ vendor: tier.name, model: model, - limit: d.limit || 0, + mode: d.mode || 'limited', + limit: d.limit, in_use: d.in_use || 0, - available: d.available || 0, + available: d.available, pending: d.pending || 0, + peak_pending_recent: d.peak_pending_recent || 0, }); } } @@ -1298,18 +1594,33 @@ def _build_favicon() -> bytes: var html = '
'; for (var k = 0; k < models.length; k++) { var m = models[k]; - var pct = m.limit > 0 ? Math.round((m.in_use / m.limit) * 100) : 0; - var barClass = pct <= 50 ? 'mc-low' : (pct <= 80 ? 'mc-mid' : 'mc-high'); - - html += '
' - + '' + escapeHtml(m.vendor + '/' + m.model) + '' - + '
' - + '
' - + '' + m.in_use - + '/' + m.limit + '' - + (m.pending > 0 ? '⏳ ' + m.pending + '' : '') - + '
' - + '
'; + + if (m.mode === 'monitor') { + // monitor 模式：纯计数徽章，无 limit/进度条 + html += '
' + + '' + escapeHtml(m.vendor + '/' + m.model) + '' + + '
' + + '
' + + '' + m.in_use + '' + + '
' + + '
'; + } else { + // limited 模式：保留现有渲染（进度条 + limit 编辑） + var limit = m.limit || 0; + var pct = limit > 0 ? Math.round((m.in_use / limit) * 100) : 0; + var barClass = pct <= 50 ? 'mc-low' : (pct <= 80 ? 'mc-mid' : 'mc-high'); + + html += '
' + + '' + escapeHtml(m.vendor + '/' + m.model) + '' + + '
' + + '
' + + '' + m.in_use + + '/' + limit + '' + + (m.pending > 0 ? '⏳ ' + m.pending + '' : '') + + (m.pending === 0 && m.peak_pending_recent > 0 ? '🕘 曾排队 ' + m.peak_pending_recent + '' : '') + + '
' + + '
'; + } } html += '
'; wrap.innerHTML = html; @@ -1325,7 +1636,7 @@ def _build_favicon() -> bytes: }).catch(function() {}); } tick(); - _mcTimer = setInterval(tick, 5000); + _mcTimer = setInterval(tick, 10000); } function stopModelCallingPoll() { if (_mcTimer) { clearInterval(_mcTimer); _mcTimer = null; } @@ -1878,9 +2189,11 @@ def _build_favicon() -> bytes: '' + formatCategories(s.client_categories) + '' + '' + '
' + + '
' + + '
Title
' + (sessionTitle ? escapeHtml(sessionTitle) : '–') + '
' + + '
' + '
' + '
Session ID
' + escapeHtml(parsed.session_id || s.session_key) + '
' + - '
Title
' + (sessionTitle ? escapeHtml(sessionTitle) : '–') + '
' + '
Device
' + (parsed.device_id ? escapeHtml(parsed.device_id) : '–') + '
' + '
Account
' + (parsed.account_uuid ? escapeHtml(parsed.account_uuid) : '–') + '
' + '
' + @@ -2078,6 +2391,7 @@ def _build_favicon() -> bytes: fetchJSON('/api/dashboard/summary?days=7').then(function(s) { if (s && s.version) document.getElementById('version-badge').textContent = 'v' + s.version; }).catch(function(){}); + initTierDrag(); // 绑定供应商列表拖拽（事件委托，幂等） refresh(); // 仅加载初始页签的数据 setInterval(refresh, 600000); // 每 10 分钟刷新当前页签 if (initial === 'overview') startModelCallingPoll(); @@ -2085,7 +2399,7 @@ def _build_favicon() -> bytes: -""" +""".replace("{{PROMPT}}", _PROMPT_PATHS).replace("{{LOGO_DEFS}}", _LOGO_DEFS) # ── 数据计算工具 ────────────────────────────────────────────────────────── @@ -2162,10 +2476,20 @@ async def favicon() -> Response: """返回内嵌 favicon.""" return Response(content=_FAVICON_ICO, media_type="image/x-icon") + @app.get("/favicon.svg", include_in_schema=False) + async def favicon_svg() -> Response: + """返回 SVG favicon（现代浏览器优先）.""" + return Response(content=_FAVICON_SVG, media_type="image/svg+xml") + @app.get("/dashboard", response_class=HTMLResponse, include_in_schema=False) async def dashboard() -> HTMLResponse: - """返回 Dashboard HTML 页面.""" - return HTMLResponse(content=_DASHBOARD_HTML) + """返回 Dashboard HTML 页面. + + 内联 JS/CSS 随版本变化，禁用缓存以免浏览器留存旧内联脚本掩盖前端修复。 + """ + return HTMLResponse( + content=_DASHBOARD_HTML, headers={"Cache-Control": "no-cache"} + ) @app.get("/api/dashboard/summary") async def dashboard_summary(request: Request, days: int = 7) -> Response: diff --git a/src/coding/proxy/server/routes.py b/src/coding/proxy/server/routes.py index 7c13d2f..6b91651 100644 --- a/src/coding/proxy/server/routes.py +++ b/src/coding/proxy/server/routes.py @@ -8,7 +8,7 @@ import httpx from fastapi import Request, Response -from fastapi.responses import StreamingResponse +from fastapi.responses import RedirectResponse, StreamingResponse from ..vendors.base import NoCompatibleVendorError @@ -197,11 +197,15 @@ async def health() -> dict: return {"status": "ok"} @app.head("/") - @app.get("/") - async def root() -> Response: + async def root_head() -> Response: """根路径连通性探测 — Claude Code 在建连前发送 HEAD / 作为 health probe.""" return Response(status_code=200) + @app.get("/") + async def root_get() -> RedirectResponse: + """GET / 重定向到 Dashboard.""" + return RedirectResponse(url="/dashboard", status_code=307) + def register_status_route(app: Any, router: Any) -> None: """注册状态查询路由.""" @@ -254,16 +258,15 @@ async def update_concurrency(request: Request) -> Response: for tier in router.tiers: if tier.name == tier_name: vendor = tier.vendor - update_fn = getattr(vendor, "update_concurrency", None) - if update_fn is None: + try: + vendor.update_concurrency(model, limit) + except ValueError as exc: return json_error_response( - 400, + 422, error_type="invalid_request_error", - message=f"vendor '{tier_name}' does not support concurrency", + message=str(exc), ) - try: - update_fn(model, limit) - except (ValueError, AttributeError) as exc: + except AttributeError as exc: return json_error_response( 400, error_type="invalid_request_error", message=str(exc) ) @@ -383,6 +386,53 @@ async def reset_circuit(request: Request) -> Response: ) +def register_tier_order_route(app: Any, router: Any) -> None: + """注册运行时 N-tier 链路顺序调整路由（纯重排序，不重置熔断器/配额守卫）.""" + + @app.put("/api/tier-order") + async def update_tier_order(request: Request) -> Response: + """替换整个 N-tier 链路顺序. + + JSON body: ``{"vendors": ["v1", "v2", ...]}``（需覆盖所有当前启用的 vendor）。 + 与 ``/api/reset`` 的重排序语义一致，但**不会**重置熔断器/配额守卫/rate limit， + 以保留当日用量统计（供 Web 拖拽调整优先级使用）。 + """ + try: + body = await request.json() + except Exception: + return json_error_response( + 400, error_type="invalid_request_error", message="body must be JSON" + ) + if not isinstance(body, dict): + return json_error_response( + 400, + error_type="invalid_request_error", + message="body must be a JSON object", + ) + raw = body.get("vendors") + if not isinstance(raw, list) or not raw: + return json_error_response( + 400, + error_type="invalid_request_error", + message="requires non-empty 'vendors' list", + ) + vendor_names = [str(v) for v in raw] + try: + router.reorder_tiers(vendor_names) + except ValueError as exc: + return json_error_response( + 400, error_type="invalid_request_error", message=str(exc) + ) + return Response( + content=json.dumps( + {"ok": True, "tier_order": router.get_vendor_names()}, + ensure_ascii=False, + ).encode(), + status_code=200, + media_type="application/json", + ) + + def register_session_vendor_routes(app: Any, router: Any) -> None: """注册 Session-Vendor 运行时绑定路由.""" @@ -515,6 +565,7 @@ def register_all_routes( register_concurrency_route(app, router) register_copilot_routes(app, router) register_admin_routes(app, router) + register_tier_order_route(app, router) register_session_vendor_routes(app, router) if reauth_coordinator: register_reauth_routes(app, reauth_coordinator) diff --git a/src/coding/proxy/vendors/base.py b/src/coding/proxy/vendors/base.py index 0c08248..d1434bc 100644 --- a/src/coding/proxy/vendors/base.py +++ b/src/coding/proxy/vendors/base.py @@ -44,6 +44,7 @@ ) from ..compat.session_store import CompatSessionRecord from ..config.schema import FailoverConfig +from .concurrency import ModelConcurrencyController logger = logging.getLogger(__name__) @@ -63,6 +64,8 @@ def __init__( self._client: httpx.AsyncClient | None = None self._compat_trace: CompatibilityTrace | None = None self._compat_session_record: CompatSessionRecord | None = None + # 默认 monitor 模式（仅计数不限流）；子类可覆盖为 limited 模式 + self._concurrency_controller = ModelConcurrencyController(None) def _get_client(self) -> httpx.AsyncClient: if self._client is None or self._client.is_closed: @@ -246,8 +249,30 @@ def get_diagnostics(self) -> dict[str, Any]: diagnostics: dict[str, Any] = {} if self._compat_trace is not None: diagnostics["compat"] = self._compat_trace.to_dict() + concurrency = self._concurrency_controller.get_diagnostics() + if concurrency: + diagnostics["concurrency"] = concurrency return diagnostics + def track_in_flight(self, mapped_model: str): + """返回用于追踪在途请求的异步上下文管理器. + + 空 model name 时返回 no-op context（防御性处理）。 + """ + if not mapped_model: + from contextlib import nullcontext + + return nullcontext() + return self._concurrency_controller.track(mapped_model) + + def update_concurrency(self, model: str, limit: int) -> None: + """运行时更新指定模型的并发限制. + + 默认实现委托给 ``_concurrency_controller.set_limit``。 + monitor 模式下抛 ``ValueError``。 + """ + self._concurrency_controller.set_limit(model, limit) + def should_trigger_failover( self, status_code: int, body: dict[str, Any] | None ) -> bool: diff --git a/src/coding/proxy/vendors/concurrency.py b/src/coding/proxy/vendors/concurrency.py index 7944bdd..a5e5763 100644 --- a/src/coding/proxy/vendors/concurrency.py +++ b/src/coding/proxy/vendors/concurrency.py @@ -1,78 +1,122 @@ -"""每模型并发限制器 — 支持运行时动态调整的公平排队. +"""统一并发控制器 — 支持监控 (monitor) 与限流 (limited) 双模式. -为每个映射后的模型（如 ``glm-5v-turbo``）独立维护一个 ``_ConcurrencySlot`， -确保同一时间点该模型的并行请求数不超过配置的上限。当所有槽位被占满时， -新请求按 FIFO 顺序排队等待，直到有槽位释放。 +为每个映射后的模型（如 ``glm-5v-turbo``）独立维护一个 ``_ConcurrencySlot``， +根据模式提供不同语义： + + **monitor 模式** (config=None) + - 仅计数 ``in_use``，不做排队与限流 + - ``pending`` 恒为 0，``available`` / ``limit`` 为 None + - 所有 vendor 默认使用此模式 + + **limited 模式** (config 非 None) + - ``in_use`` 不超过 ``limit`` 时立即获取，超限时 FIFO 排队 + - ``pending`` 反映当前排队数，``peak_pending_recent`` 记录最近 10s 峰值 + - 由 ZhipuVendor 等需限流的 vendor 启用 设计要点： - **惰性创建**：仅在首次请求到达时才为该模型创建 Slot，避免冷启动开销 - - **FIFO 公平**：``asyncio.Event`` + while 循环天然满足 FIFO 排队语义 + - **FIFO 公平**：``asyncio.Event`` + while 循环天然满足 FIFO 排队语义（limited 模式） - **动态调整**：支持运行时修改 per-model limit，无需重启进程 - **按映射后模型名键控**：与上游真实承载能力对齐，而非按客户端请求名 + - **峰值余晖**：记录 ``peak_pending_recent`` 使瞬时排队可观测 """ from __future__ import annotations import asyncio import logging +import time +from collections import deque +from contextlib import asynccontextmanager +from typing import Any, Literal from ..config.vendors import ZhipuConcurrencyConfig logger = logging.getLogger(__name__) +# peak_pending_recent 滑窗宽度（秒） +_PEAK_WINDOW_SECONDS = 10.0 + class _ConcurrencySlot: - """支持动态 limit 的并发槽位. + """支持双模式的并发槽位. - 使用 ``asyncio.Event`` 作为等待/通知原语，在 ``acquire`` 中 await 等待， - 在 ``release`` / ``set_limit`` 中唤醒。``set_limit`` 修改上限后立即唤醒 - 所有等待者，由它们重新判断是否可获得槽位。 + ``limit=None`` (monitor) 时 acquire 走 fast path，仅计数。 + ``limit>0`` (limited) 时在满槽位后 FIFO 排队等待。 """ - def __init__(self, limit: int) -> None: + def __init__(self, limit: int | None) -> None: self._limit = limit self._in_use: int = 0 self._pending: int = 0 self._wake = asyncio.Event() self._wake.set() + # peak_pending_recent 追踪：存储 (timestamp, pending_value) 元组 + self._peak_samples: deque[tuple[float, int]] = deque() - async def acquire(self) -> _ConcurrencySlot: - """获取一个并发槽位，必要时阻塞排队. + async def acquire(self) -> None: + """获取一个并发槽位. - 返回 ``self``，调用方在请求完成后调用 ``release()``。 + monitor 模式 (limit=None)：仅 in_use++，永不排队。 + limited 模式 (limit>0)：满槽时阻塞等待。 """ - # Fast path + # monitor 模式：仅计数 + if self._limit is None: + self._in_use += 1 + return + + # limited — fast path if self._in_use < self._limit: self._in_use += 1 - return self - # Slow path — 等待槽位释放 + return + + # limited — slow path: FIFO 排队 self._pending += 1 + self._observe_peak() try: while True: self._wake.clear() await self._wake.wait() if self._in_use < self._limit: self._in_use += 1 - return self + return finally: self._pending -= 1 def release(self) -> None: """释放一个并发槽位.""" self._in_use = max(0, self._in_use - 1) - self._wake.set() + if self._limit is not None: + self._wake.set() def set_limit(self, new_limit: int) -> None: """动态调整并发上限. - 增大 limit 时立即唤醒等待者；缩小时已持有的槽位不受影响， - 新 limit 在后续 acquire 中自然生效。 + 仅 limited 模式有效；monitor 模式调用抛 ValueError。 """ + if self._limit is None: + msg = "Cannot set limit on monitor-only slot" + raise ValueError(msg) self._limit = new_limit self._wake.set() + def _observe_peak(self) -> None: + """记录当前 pending 值作为峰值采样点.""" + now = time.monotonic() + self._peak_samples.append((now, self._pending)) + + def _get_peak_pending_recent(self) -> int: + """获取最近窗口内的 peak pending 值.""" + cutoff = time.monotonic() - _PEAK_WINDOW_SECONDS + # 剔除过期采样 + while self._peak_samples and self._peak_samples[0][0] < cutoff: + self._peak_samples.popleft() + if not self._peak_samples: + return 0 + return max(v for _, v in self._peak_samples) + @property - def limit(self) -> int: + def limit(self) -> int | None: return self._limit @property @@ -80,59 +124,93 @@ def in_use(self) -> int: return self._in_use @property - def available(self) -> int: + def available(self) -> int | None: + if self._limit is None: + return None return max(0, self._limit - self._in_use) @property def pending(self) -> int: return self._pending + @property + def peak_pending_recent(self) -> int: + return self._get_peak_pending_recent() -class ModelConcurrencyLimiter: - """按模型名提供独立并发槽位的限制器. + +class ModelConcurrencyController: + """按模型名提供独立并发槽位的控制器. 用法:: - limiter = ModelConcurrencyLimiter(config) - slot = await limiter.acquire("glm-5v-turbo") - try: + # monitor 模式（默认） + ctrl = ModelConcurrencyController(None) + async with ctrl.track("model-a"): ... # 执行请求 - finally: - slot.release() + + # limited 模式（Zhipu 等） + ctrl = ModelConcurrencyController(config) + async with ctrl.track("glm-5v-turbo"): + ... # 满槽时排队等待 """ - def __init__(self, config: ZhipuConcurrencyConfig) -> None: + def __init__(self, config: ZhipuConcurrencyConfig | None) -> None: self._config = config self._slots: dict[str, _ConcurrencySlot] = {} + @property + def mode(self) -> Literal["monitor", "limited"]: + """当前控制器模式.""" + return "limited" if self._config is not None else "monitor" + def _get_or_create_slot(self, model: str) -> _ConcurrencySlot: """获取（或惰性创建）指定模型的并发槽位.""" slot = self._slots.get(model) if slot is None: - limit = self._config.get_limit(model) + if self._config is not None: + limit = self._config.get_limit(model) + else: + limit = None slot = _ConcurrencySlot(limit) self._slots[model] = slot - logger.debug( - "ModelConcurrencyLimiter: created slot model=%s limit=%d", - model, - limit, - ) + if self._config is not None: + logger.debug( + "ModelConcurrencyController: created slot mode=limited " + "model=%s limit=%d", + model, + limit, + ) + else: + logger.debug( + "ModelConcurrencyController: created slot mode=monitor model=%s", + model, + ) return slot - async def acquire(self, model: str) -> _ConcurrencySlot: - """获取指定模型的并发槽位，必要时阻塞排队. + @asynccontextmanager + async def track(self, model: str): + """异步上下文管理器：获取 → 执行 → 释放. + + 用法:: - 返回已获取的 Slot 实例，调用方负责在请求完成后调用 ``release()``。 + async with controller.track("glm-5v-turbo"): + await vendor.send_message(...) """ slot = self._get_or_create_slot(model) await slot.acquire() - return slot + try: + yield + finally: + slot.release() def set_limit(self, model: str, new_limit: int) -> None: """运行时修改指定模型的并发上限. - 同时更新 config.models 以确保后续惰性创建使用新值。 + 仅 limited 模式支持；monitor 模式抛 ValueError。 """ + if self._config is None: + msg = f"vendor is monitor-only; cannot update limit for model '{model}'" + raise ValueError(msg) slot = self._slots.get(model) if slot is None: slot = _ConcurrencySlot(new_limit) @@ -141,22 +219,33 @@ def set_limit(self, model: str, new_limit: int) -> None: slot.set_limit(new_limit) self._config.models[model] = new_limit logger.info( - "ModelConcurrencyLimiter: updated limit model=%s new_limit=%d", + "ModelConcurrencyController: updated limit model=%s new_limit=%d", model, new_limit, ) - def get_diagnostics(self) -> dict[str, dict[str, int]]: + def get_diagnostics(self) -> dict[str, dict[str, Any]]: """返回每个模型的并发状态快照（用于可观测性）.""" - snapshot: dict[str, dict[str, int]] = {} + snapshot: dict[str, dict[str, Any]] = {} + mode = self.mode for model, slot in self._slots.items(): - snapshot[model] = { - "limit": slot.limit, + entry: dict[str, Any] = { + "mode": mode, "in_use": slot.in_use, - "available": slot.available, "pending": slot.pending, + "peak_pending_recent": slot.peak_pending_recent, } + if mode == "limited": + entry["limit"] = slot.limit + entry["available"] = slot.available + else: + entry["limit"] = None + entry["available"] = None + snapshot[model] = entry return snapshot -__all__ = ["ModelConcurrencyLimiter"] +# 向后兼容别名 +ModelConcurrencyLimiter = ModelConcurrencyController + +__all__ = ["ModelConcurrencyController", "ModelConcurrencyLimiter"] diff --git a/src/coding/proxy/vendors/zhipu.py b/src/coding/proxy/vendors/zhipu.py index 64407ba..ca6270a 100644 --- a/src/coding/proxy/vendors/zhipu.py +++ b/src/coding/proxy/vendors/zhipu.py @@ -1,4 +1,4 @@ -"""智谱 GLM 供应商 — 原生 Anthropic 兼容端点代理（兼容转换 + 429 重试）. +"""智谱 GLM 供应商 — 原生 Anthropic 兼容端点代理（兼容转换 + 429/529 重试）. 官方端点 (https://open.bigmodel.cn/api/anthropic) 支持大部分 Anthropic Messages API 协议，本模块做以下适配： @@ -13,10 +13,16 @@ - reasoning_effort 参数：静默忽略 - metadata 字段：暂不处理（待进一步诊断确认兼容性） -额外提供 429 Rate Limit 专用重试挽回机制： +额外提供 429/529 专用重试挽回机制： + - 429 Rate Limit（限流）与 529 Overloaded（并发过载）共用同一退避策略 - max_attempt = 5（1 初始 + 4 重试） - - 指数退避 + Full Jitter（1s → 2s → 4s → 8s） + - 指数退避 + Equal Jitter（区间 0.5–1s → 1–2s → 2–4s → 4–8s） - 优先尊重 server retry-after header + +并发限流由 BaseVendor._concurrency_controller 统一管控 +（limited 模式），在 executor 层通过 ``track_in_flight`` 触发， +slot 跨 429 重试自然持有（executor 的 async with 包裹整个 +send_message/send_message_stream 调用链）。 """ from __future__ import annotations @@ -37,7 +43,7 @@ ) from ..routing.retry import RetryConfig, calculate_delay from .base import VendorResponse -from .concurrency import ModelConcurrencyLimiter +from .concurrency import ModelConcurrencyController from .native_anthropic import NativeAnthropicVendor logger = logging.getLogger(__name__) @@ -51,14 +57,21 @@ jitter=True, ) +# 触发指数退避重试的瞬态状态码（单一事实源） +# 429 Rate Limit（限流）；529 Overloaded（并发过载） +# 二者均为服务端瞬态过载信号，适用同一退避挽回策略。 +_BACKOFF_RETRY_STATUS = frozenset({429, 529}) + class ZhipuVendor(NativeAnthropicVendor): - """智谱 GLM 原生 Anthropic 兼容端点供应商（薄透传 + 429 重试挽回）. + """智谱 GLM 原生 Anthropic 兼容端点供应商（薄透传 + 429/529 重试挽回）. 通过官方 /api/anthropic 端点转发请求， 仅替换模型名和认证头，其余原样透传。 - 429 Rate Limit 时自动重试（指数退避），降低 failover 频率。 + 429 Rate Limit（限流）/ 529 Overloaded（并发过载）时自动重试 + （指数退避），降低 failover 频率。 + 并发限流由 BaseVendor._concurrency_controller 统一管控。 """ _vendor_name = "zhipu" @@ -72,12 +85,8 @@ def __init__( ) -> None: super().__init__(config, model_mapper, failover_config) self._rl_retry = _RATE_LIMIT_RETRY - # 每模型并发限制器（config.concurrency 为 None 时禁用） - self._concurrency_limiter: ModelConcurrencyLimiter | None = ( - ModelConcurrencyLimiter(config.concurrency) - if config.concurrency is not None - else None - ) + # 覆盖 BaseVendor 默认的 monitor 模式为 limited 模式 + self._concurrency_controller = ModelConcurrencyController(config.concurrency) # ── 首选 tier 参数兼容转换 ──────────────────────────────── @@ -122,41 +131,25 @@ async def _prepare_request( return body, new_headers - # ── 非流式：429 重试 ──────────────────────────────────── + # ── 非流式：429/529 重试 ──────────────────────────────── async def send_message( self, request_body: dict[str, Any], headers: dict[str, str], ) -> VendorResponse: - """非流式请求，429 时自动重试. - - 在 429 重试循环外层套上每模型并发槽位获取，确保同一时间点同一模型的 - 在途请求数不超过配置上限；超过时新请求 FIFO 排队等待。 - """ - sem = await self._maybe_acquire_concurrency_slot(request_body) - try: - return await self._send_message_with_retry(request_body, headers) - finally: - if sem is not None: - sem.release() - - async def _send_message_with_retry( - self, - request_body: dict[str, Any], - headers: dict[str, str], - ) -> VendorResponse: - """原 send_message 主体逻辑（不含并发控制）.""" + """非流式请求，429/529 时自动重试.""" max_attempts = self._rl_retry.max_attempts for attempt in range(max_attempts): resp = await super().send_message(request_body, headers) - if resp.status_code != 429: + if resp.status_code not in _BACKOFF_RETRY_STATUS: return resp if attempt == max_attempts - 1: logger.warning( - "Zhipu 429 rate limit exhausted after %d attempts", + "Zhipu %d overload exhausted after %d attempts", + resp.status_code, max_attempts, ) return resp @@ -165,7 +158,8 @@ async def _send_message_with_retry( resp.response_headers, attempt ) logger.info( - "Zhipu 429 rate limit, retry %d/%d in %.1fms", + "Zhipu %d overload, retry %d/%d in %.1fms", + resp.status_code, attempt + 1, max_attempts - 1, delay, @@ -174,99 +168,62 @@ async def _send_message_with_retry( return resp # pragma: no cover - # ── 流式：429 重试 ────────────────────────────────────── + # ── 流式：429/529 重试 ────────────────────────────────── async def send_message_stream( self, request_body: dict[str, Any], headers: dict[str, str], ) -> AsyncIterator[bytes]: - """流式请求，429 时自动重试. + """流式请求，429/529 时自动重试. - 安全性：429 在 BaseVendor.send_message_stream 中于 + 安全性：429/529 在 BaseVendor.send_message_stream 中于 status code 检查阶段即 raise（在任何 chunk yield 之前）， 因此重试不会导致已发出数据不一致。 - - 在 429 重试循环外层套上每模型并发槽位获取，确保流式请求与非流式请求 - 共用同一信号量，统一限制同一模型的总在途并发数。 """ - sem = await self._maybe_acquire_concurrency_slot(request_body) max_attempts = self._rl_retry.max_attempts - try: - for attempt in range(max_attempts): - try: - # 429 在 status code 检查阶段即 raise（在任何 chunk 之前）， - # 因此 __anext__ 安全：要么拿到首个 chunk，要么抛异常。 - ait = super().send_message_stream(request_body, headers) - head = await ait.__anext__() - except StopAsyncIteration: - return - except httpx.HTTPStatusError as exc: - if exc.response is None or exc.response.status_code != 429: - raise - if attempt == max_attempts - 1: - logger.warning( - "Zhipu 429 stream rate limit exhausted after %d attempts", - max_attempts, - ) - raise - - delay = self._compute_retry_delay_from_response( - exc.response, attempt - ) - logger.info( - "Zhipu 429 stream rate limit, retry %d/%d in %.1fms", - attempt + 1, - max_attempts - 1, - delay, - ) - await asyncio.sleep(delay / 1000.0) - continue - - # yield 在 try/except 之外，避免捕获外部 athrow 的异常 - yield head - async for chunk in ait: - yield chunk + for attempt in range(max_attempts): + try: + # 429/529 在 status code 检查阶段即 raise（在任何 chunk 之前）， + # 因此 __anext__ 安全：要么拿到首个 chunk，要么抛异常。 + ait = super().send_message_stream(request_body, headers) + head = await ait.__anext__() + except StopAsyncIteration: return - finally: - if sem is not None: - sem.release() - - # ── 并发控制 ──────────────────────────────────────────── + except httpx.HTTPStatusError as exc: + if ( + exc.response is None + or exc.response.status_code not in _BACKOFF_RETRY_STATUS + ): + raise + status_code = exc.response.status_code + if attempt == max_attempts - 1: + logger.warning( + "Zhipu %d stream overload exhausted after %d attempts", + status_code, + max_attempts, + ) + raise - async def _maybe_acquire_concurrency_slot( - self, - request_body: dict[str, Any], - ) -> asyncio.Semaphore | None: - """按映射后模型名获取并发槽位；未配置 concurrency 时返回 None. + delay = self._compute_retry_delay_from_headers( + exc.response.headers, attempt + ) + logger.info( + "Zhipu %d stream overload, retry %d/%d in %.1fms", + status_code, + attempt + 1, + max_attempts - 1, + delay, + ) + await asyncio.sleep(delay / 1000.0) + continue - ``map_model()`` 是纯同步字典查找，在 Semaphore 等待前调用是安全的， - 且能确保排队键与上游真实承载模型对齐。 - """ - if self._concurrency_limiter is None: - return None - raw_model = request_body.get("model", "") if request_body else "" - mapped_model = self.map_model(raw_model) if raw_model else "" - if not mapped_model: - return None - return await self._concurrency_limiter.acquire(mapped_model) - - # ── 诊断信息 ───────────────────────────────────────────── - - def get_diagnostics(self) -> dict[str, Any]: - """返回供应商运行时诊断信息，包含每模型并发状态.""" - diagnostics = super().get_diagnostics() - if self._concurrency_limiter is not None: - diagnostics["concurrency"] = self._concurrency_limiter.get_diagnostics() - return diagnostics - - def update_concurrency(self, model: str, limit: int) -> None: - """运行时更新指定模型的并发限制.""" - if self._concurrency_limiter is None: - msg = "Concurrency limiter is not enabled for this vendor" - raise ValueError(msg) - self._concurrency_limiter.set_limit(model, limit) + # yield 在 try/except 之外，避免捕获外部 athrow 的异常 + yield head + async for chunk in ait: + yield chunk + return # ── 延迟计算 ──────────────────────────────────────────── @@ -275,24 +232,15 @@ def _compute_retry_delay_from_headers( headers: dict[str, str] | None, attempt: int, ) -> float: - """计算重试延迟（毫秒），优先使用 server retry-after.""" - rl_info = parse_rate_limit_headers(headers, 429, None) - server_delay_s = compute_effective_retry_seconds(rl_info) - if server_delay_s is not None: - return min(server_delay_s * 1000, self._rl_retry.max_delay_ms) - return calculate_delay(attempt, self._rl_retry) + """计算重试延迟（毫秒），优先使用 server retry-after. - def _compute_retry_delay_from_response( - self, - response: httpx.Response, - attempt: int, - ) -> float: - """计算重试延迟（毫秒），从 httpx.Response 提取 header.""" - rl_info = parse_rate_limit_headers( - response.headers, - response.status_code, - response.text[:500] if response.text else None, - ) + 非流式与流式 429/529 重试共用此方法（单一延迟逻辑）。 + 以"限流退避"语义（429）解析 header：``parse_rate_limit_headers`` + 仅对 429/403 解析 retry-after，故此处固定传 429， + 使 529 也能尊重 server retry-after，与 429 行为一致。 + 无 server 信号时回退到指数退避 + Equal Jitter。 + """ + rl_info = parse_rate_limit_headers(headers, 429, None) server_delay_s = compute_effective_retry_seconds(rl_info) if server_delay_s is not None: return min(server_delay_s * 1000, self._rl_retry.max_delay_ms) diff --git a/tests/e2e/test_e2e_http.py b/tests/e2e/test_e2e_http.py index fe84db5..442577d 100644 --- a/tests/e2e/test_e2e_http.py +++ b/tests/e2e/test_e2e_http.py @@ -188,13 +188,16 @@ async def test_http_health_probe(e2e_client: object) -> None: ) get_resp = await e2e_client.get("/") - assert get_resp.status_code == 200, f"GET / 预期 200，实际 {get_resp.status_code}" + assert get_resp.status_code == 307, f"GET / 预期 307，实际 {get_resp.status_code}" + assert get_resp.headers["location"] == "/dashboard" health_resp = await e2e_client.get("/health") assert health_resp.status_code == 200 assert health_resp.json() == {"status": "ok"} - print("\n[E2E] HTTP health probe 成功: HEAD /=200, GET /=200, /health=ok") + print( + "\n[E2E] HTTP health probe 成功: HEAD /=200, GET /=307→/dashboard, /health=ok" + ) @pytest.mark.e2e diff --git a/tests/test_app_routes.py b/tests/test_app_routes.py index 4c460e3..5c0aa53 100644 --- a/tests/test_app_routes.py +++ b/tests/test_app_routes.py @@ -35,11 +35,12 @@ def test_head_root_returns_200(): assert resp.status_code == 200 -def test_get_root_returns_200(): - """GET / 返回 200.""" +def test_get_root_redirects_to_dashboard(): + """GET / 重定向到 /dashboard.""" with _make_app() as client: - resp = client.get("/") - assert resp.status_code == 200 + resp = client.get("/", follow_redirects=False) + assert resp.status_code == 307 + assert resp.headers["location"] == "/dashboard" # ── count_tokens 透传 ──────────────────────────────────────── @@ -1090,3 +1091,147 @@ def test_reorder_tiers_shared_reference(): # Executor 的列表也改变了（因为是同一个对象） assert [t.name for t in executor_tiers] == ["c", "a", "b"] assert router.get_vendor_names() == ["c", "a", "b"] + + +# ── /api/tier-order 纯重排序测试（不重置配额/熔断器/rate limit）────────────── + + +def test_tier_order_reorder_success(): + """PUT /api/tier-order → 精确替换链路顺序，返回最新 tier_order.""" + app = _make_reorder_app() + with TestClient(app) as client: + resp = client.put( + "/api/tier-order", json={"vendors": ["copilot", "anthropic", "zhipu"]} + ) + assert resp.status_code == 200 + data = resp.json() + assert data["ok"] is True + assert data["tier_order"] == ["copilot", "anthropic", "zhipu"] + assert app.state.router.get_vendor_names() == ["copilot", "anthropic", "zhipu"] + + +def test_tier_order_unknown_vendor_returns_400(): + """未知 vendor 名称 → 400.""" + app = _make_reorder_app() + with TestClient(app) as client: + resp = client.put( + "/api/tier-order", + json={"vendors": ["nonexist", "anthropic", "zhipu", "copilot"]}, + ) + assert resp.status_code == 400 + assert "未知 vendor" in resp.json()["error"]["message"] + + +def test_tier_order_duplicate_returns_400(): + """重复 vendor 名称 → 400.""" + app = _make_reorder_app() + with TestClient(app) as client: + resp = client.put( + "/api/tier-order", + json={"vendors": ["anthropic", "anthropic", "zhipu", "copilot"]}, + ) + assert resp.status_code == 400 + assert "重复" in resp.json()["error"]["message"] + + +def test_tier_order_incomplete_returns_400(): + """不完整的 vendor 列表（缺少现有 tier）→ 400.""" + app = _make_reorder_app() + with TestClient(app) as client: + resp = client.put("/api/tier-order", json={"vendors": ["anthropic", "zhipu"]}) + assert resp.status_code == 400 + assert "缺少 vendor" in resp.json()["error"]["message"] + + +def test_tier_order_invalid_body_returns_400(): + """非法 body（非 JSON / 缺 vendors / 非列表 / 空列表）→ 400.""" + app = _make_reorder_app() + with TestClient(app) as client: + # 非 JSON + resp = client.put( + "/api/tier-order", + content=b"not-json", + headers={"content-type": "application/json"}, + ) + assert resp.status_code == 400 + # body 非 dict + resp = client.put("/api/tier-order", json=["anthropic", "zhipu", "copilot"]) + assert resp.status_code == 400 + # 缺 vendors + resp = client.put("/api/tier-order", json={"foo": "bar"}) + assert resp.status_code == 400 + # vendors 非列表 + resp = client.put("/api/tier-order", json={"vendors": "zhipu"}) + assert resp.status_code == 400 + # vendors 空列表 + resp = client.put("/api/tier-order", json={"vendors": []}) + assert resp.status_code == 400 + + +def test_tier_order_preserves_quota_guard(): + """核心：重排序不重置配额守卫的滑动窗口用量（保留当日统计）.""" + app = _make_reorder_app() + router = app.state.router + # 选用首个 tier，手动启用其配额守卫并写入用量 + tier = router.tiers[0] + qg = tier.quota_guard + assert qg is not None + qg._enabled = True + qg._budget = 100_000 + qg.record_usage(40_000) + before = qg.get_info() + assert before["window_usage_tokens"] == 40_000 + assert before["usage_percent"] == 40.0 + + with TestClient(app) as client: + resp = client.put( + "/api/tier-order", json={"vendors": ["zhipu", "copilot", "anthropic"]} + ) + assert resp.status_code == 200 + + # reorder_tiers 仅重排引用、不重建对象 → 同一 QuotaGuard 实例用量保持 + after = qg.get_info() + assert after["window_usage_tokens"] == 40_000 + assert after["usage_percent"] == 40.0 + + +def test_tier_order_preserves_circuit_breaker_and_rate_limit(): + """重排序不重置熔断器与 rate limit（与 /api/reset 行为相反）.""" + app = _make_reorder_app() + router = app.state.router + target = router.tiers[0] + target.record_failure(retry_after_seconds=300) + target._rate_limit_deadline = 999_999.0 + assert not target.can_execute() + + with TestClient(app) as client: + resp = client.put( + "/api/tier-order", json={"vendors": ["zhipu", "anthropic", "copilot"]} + ) + assert resp.status_code == 200 + + same = next(t for t in router.tiers if t.name == target.name) + assert not same.can_execute() # 仍熔断 + assert same.is_rate_limited # 仍限速 + + +def test_dashboard_serves_pointer_drag_reorder_and_no_cache(): + """Dashboard 供应商拖拽已由脆弱的原生 HTML5 DnD 改为 Pointer Events，并禁用页面缓存. + + #269 仅覆盖后端 /api/tier-order，前端拖拽机制无任何断言（拖拽从未被验证）。 + 此处补一条轻量守卫：确保重排序 UI 机制（指针事件 + PUT /api/tier-order）随页面交付、 + 不回退到原生 DnD，且响应带 no-cache 以免浏览器留存旧内联脚本掩盖前端修复。 + """ + with _make_app() as client: + resp = client.get("/dashboard") + assert resp.status_code == 200 + assert resp.headers.get("cache-control") == "no-cache" + html = resp.text + # Pointer Events 重排机制在位 + assert "initTierDrag" in html + assert "pointerdown" in html + assert "setPointerCapture" in html + assert "/api/tier-order" in html + # 已移除脆弱的原生 HTML5 DnD（不再渲染 draggable 属性 / 监听 dragstart） + assert 'draggable="true"' not in html + assert "addEventListener('dragstart'" not in html diff --git a/tests/test_concurrency_monitor.py b/tests/test_concurrency_monitor.py new file mode 100644 index 0000000..4fce021 --- /dev/null +++ b/tests/test_concurrency_monitor.py @@ -0,0 +1,158 @@ +"""ModelConcurrencyController monitor 模式专项测试. + +验证 ``config=None`` 时的纯计数行为： + - acquire 不阻塞，无 limit / available + - pending 永远为 0 + - set_limit 抛 ValueError + - 100 并发 in_use 峰值正确 + - get_diagnostics 输出 mode="monitor" + limit/available=None +""" + +from __future__ import annotations + +import asyncio + +import pytest + +from coding.proxy.vendors.concurrency import ModelConcurrencyController + + +class TestMonitorMode: + """monitor 模式（config=None）基础行为.""" + + def test_mode_property(self) -> None: + ctrl = ModelConcurrencyController(None) + assert ctrl.mode == "monitor" + + @pytest.mark.asyncio + async def test_acquire_never_blocks(self) -> None: + """monitor 模式下任意数量并发都立即获取槽位.""" + ctrl = ModelConcurrencyController(None) + slot = ctrl._get_or_create_slot("model-x") + # 即使触发多次 acquire 也不阻塞 + for _ in range(10): + await slot.acquire() + assert slot.in_use == 10 + assert slot.pending == 0 + + @pytest.mark.asyncio + async def test_100_concurrent_acquires_no_queue(self) -> None: + """100 并发请求全部立即拿到槽位，无排队.""" + ctrl = ModelConcurrencyController(None) + + gate = asyncio.Event() + max_in_use = 0 + + async def hold(model: str) -> None: + nonlocal max_in_use + async with ctrl.track(model): + slot = ctrl._get_or_create_slot(model) + max_in_use = max(max_in_use, slot.in_use) + await gate.wait() + + tasks = [asyncio.create_task(hold("test-model")) for _ in range(100)] + # 等所有任务进入 track + await asyncio.sleep(0.1) + slot = ctrl._get_or_create_slot("test-model") + assert slot.in_use == 100, "monitor 模式应允许全部并行" + assert slot.pending == 0, "monitor 模式 pending 应恒为 0" + gate.set() + await asyncio.gather(*tasks) + # 释放后归零 + assert slot.in_use == 0 + + @pytest.mark.asyncio + async def test_release_after_track(self) -> None: + """track 上下文退出后 in_use 归零.""" + ctrl = ModelConcurrencyController(None) + slot = ctrl._get_or_create_slot("m") + async with ctrl.track("m"): + assert slot.in_use == 1 + assert slot.in_use == 0 + + def test_set_limit_raises_in_monitor(self) -> None: + """monitor 模式下 set_limit 抛 ValueError.""" + ctrl = ModelConcurrencyController(None) + with pytest.raises(ValueError, match="monitor-only"): + ctrl.set_limit("m", 5) + + def test_diagnostics_monitor_shape(self) -> None: + """monitor 模式 get_diagnostics 输出 mode + limit/available=None.""" + ctrl = ModelConcurrencyController(None) + ctrl._get_or_create_slot("m") + snap = ctrl.get_diagnostics() + assert snap["m"]["mode"] == "monitor" + assert snap["m"]["limit"] is None + assert snap["m"]["available"] is None + assert snap["m"]["in_use"] == 0 + assert snap["m"]["pending"] == 0 + assert snap["m"]["peak_pending_recent"] == 0 + + @pytest.mark.asyncio + async def test_pending_never_increases_in_monitor(self) -> None: + """monitor 模式即使触发大量并发，pending 永远不增.""" + ctrl = ModelConcurrencyController(None) + gate = asyncio.Event() + + async def hold() -> None: + async with ctrl.track("m"): + await gate.wait() + + tasks = [asyncio.create_task(hold()) for _ in range(20)] + await asyncio.sleep(0.05) + slot = ctrl._get_or_create_slot("m") + assert slot.pending == 0 + assert slot.in_use == 20 + gate.set() + await asyncio.gather(*tasks) + + +class TestBaseVendorTrackInFlight: + """BaseVendor.track_in_flight 默认 monitor 行为.""" + + def test_empty_model_returns_noop(self) -> None: + """空 model name 返回 no-op 上下文，不影响 controller 状态.""" + from coding.proxy.config.schema import AnthropicConfig, FailoverConfig + from coding.proxy.vendors.anthropic import AnthropicVendor + + vendor = AnthropicVendor(AnthropicConfig(), FailoverConfig()) + ctx = vendor.track_in_flight("") + # nullcontext 是同步上下文管理器，应有 __enter__/__exit__ + # 我们不进入它，只验证不抛错 + assert ctx is not None + + @pytest.mark.asyncio + async def test_track_in_flight_increments_in_use(self) -> None: + """非空 model name → controller.track 进入，in_use 自增.""" + from coding.proxy.config.schema import AnthropicConfig, FailoverConfig + from coding.proxy.vendors.anthropic import AnthropicVendor + + vendor = AnthropicVendor(AnthropicConfig(), FailoverConfig()) + async with vendor.track_in_flight("claude-test"): + slot = vendor._concurrency_controller._get_or_create_slot("claude-test") + assert slot.in_use == 1 + # 退出后归零 + slot = vendor._concurrency_controller._get_or_create_slot("claude-test") + assert slot.in_use == 0 + + def test_update_concurrency_default_is_monitor_only(self) -> None: + """BaseVendor 默认 monitor → update_concurrency 抛 ValueError.""" + from coding.proxy.config.schema import AnthropicConfig, FailoverConfig + from coding.proxy.vendors.anthropic import AnthropicVendor + + vendor = AnthropicVendor(AnthropicConfig(), FailoverConfig()) + with pytest.raises(ValueError, match="monitor-only"): + vendor.update_concurrency("m", 5) + + def test_get_diagnostics_includes_concurrency_after_use(self) -> None: + """track_in_flight 用过后 get_diagnostics 输出 concurrency 字段.""" + from coding.proxy.config.schema import AnthropicConfig, FailoverConfig + from coding.proxy.vendors.anthropic import AnthropicVendor + + vendor = AnthropicVendor(AnthropicConfig(), FailoverConfig()) + # 触发 slot 创建 + vendor._concurrency_controller._get_or_create_slot("claude-test") + diag = vendor.get_diagnostics() + assert "concurrency" in diag + assert "claude-test" in diag["concurrency"] + assert diag["concurrency"]["claude-test"]["mode"] == "monitor" diff --git a/tests/test_executor_in_flight_tracking.py b/tests/test_executor_in_flight_tracking.py new file mode 100644 index 0000000..9b57e39 --- /dev/null +++ b/tests/test_executor_in_flight_tracking.py @@ -0,0 +1,233 @@ +"""Executor 层 track_in_flight 包裹行为验证. + +验证 ``_RouteExecutor`` 在调用 ``vendor.send_message[_stream]`` 前 +进入 ``vendor.track_in_flight(mapped_model)`` 上下文，在调用结束（包括异常） +后正确退出（释放槽位）。 +""" + +from __future__ import annotations + +from contextlib import asynccontextmanager +from unittest.mock import AsyncMock, MagicMock + +import httpx +import pytest + +from coding.proxy.compat.canonical import ( + CompatibilityDecision, + CompatibilityStatus, +) +from coding.proxy.routing.executor import _RouteExecutor +from coding.proxy.routing.session_manager import RouteSessionManager +from coding.proxy.routing.tier import VendorTier +from coding.proxy.routing.usage_recorder import UsageRecorder +from coding.proxy.vendors.base import ( + BaseVendor, + RequestCapabilities, + UsageInfo, + VendorCapabilities, + VendorResponse, +) + + +class _TrackingProbe: + """共享状态：记录 track_in_flight enter/exit 时序与 send 调用顺序.""" + + def __init__(self) -> None: + self.events: list[str] = [] + self.in_flight: int = 0 + self.peak_in_flight: int = 0 + + def track_factory(self, vendor_name: str): + @asynccontextmanager + async def _track(mapped_model: str): + self.events.append(f"enter:{vendor_name}:{mapped_model}") + self.in_flight += 1 + self.peak_in_flight = max(self.peak_in_flight, self.in_flight) + try: + yield + finally: + self.in_flight -= 1 + self.events.append(f"exit:{vendor_name}:{mapped_model}") + + def _factory(mapped_model: str): + return _track(mapped_model) + + return _factory + + +def _mock_vendor_with_probe( + probe: _TrackingProbe, name: str = "test", **caps_kwargs +) -> BaseVendor: + """创建带 track_in_flight 探针的 mock vendor.""" + vendor = MagicMock(spec=BaseVendor) + vendor.get_name.return_value = name + vendor.map_model.return_value = f"{name}-mapped" + caps = VendorCapabilities(**caps_kwargs) + vendor.get_capabilities.return_value = caps + vendor.get_compatibility_profile.return_value = MagicMock() + vendor.make_compatibility_decision.return_value = CompatibilityDecision( + status=CompatibilityStatus.NATIVE, + ) + vendor.get_compat_trace.return_value = None + + def _supports_request(_caps: RequestCapabilities): + return True, [] + + vendor.supports_request.side_effect = _supports_request + vendor.check_health = AsyncMock(return_value=True) + vendor.close = AsyncMock() + vendor.set_compat_context = MagicMock() + + # 关键：track_in_flight 委托给 probe + vendor.track_in_flight = MagicMock(side_effect=probe.track_factory(name)) + + # send_message 默认返回成功 + async def _send_message(_body, _headers): + probe.events.append(f"send:{name}") + return VendorResponse( + status_code=200, + raw_body=b"{}", + usage=UsageInfo(input_tokens=1, output_tokens=1), + ) + + vendor.send_message = AsyncMock(side_effect=_send_message) + + # send_message_stream 默认产出空流 + async def _send_stream(_body, _headers): + probe.events.append(f"stream_start:{name}") + yield b'data: {"type":"message_start"}\n\n' + probe.events.append(f"stream_end:{name}") + + vendor.send_message_stream = MagicMock(side_effect=_send_stream) + return vendor + + +def _make_executor(vendor: BaseVendor) -> _RouteExecutor: + tier = VendorTier(vendor=vendor) + return _RouteExecutor( + router=MagicMock(), + tiers=[tier], + usage_recorder=UsageRecorder(), + session_manager=RouteSessionManager(), + ) + + +class TestExecuteMessageInFlightTracking: + """非流式调用的 track_in_flight 包裹行为.""" + + @pytest.mark.asyncio + async def test_track_enter_before_send_exit_after(self): + """success path: enter → send → exit 顺序.""" + probe = _TrackingProbe() + vendor = _mock_vendor_with_probe(probe, name="kimi") + exec_inst = _make_executor(vendor) + + resp = await exec_inst.execute_message({"model": "claude-test"}, {}) + assert resp.status_code == 200 + + assert probe.events == [ + "enter:kimi:kimi-mapped", + "send:kimi", + "exit:kimi:kimi-mapped", + ] + vendor.track_in_flight.assert_called_once_with("kimi-mapped") + + @pytest.mark.asyncio + async def test_track_exits_on_http_error(self): + """异常路径：track_in_flight 仍执行 exit.""" + probe = _TrackingProbe() + vendor = _mock_vendor_with_probe(probe, name="kimi") + vendor.send_message = AsyncMock(side_effect=httpx.ConnectError("down")) + exec_inst = _make_executor(vendor) + + with pytest.raises(httpx.ConnectError): + await exec_inst.execute_message({"model": "claude-test"}, {}) + + # enter 与 exit 都应被记录（finally 触发） + assert "enter:kimi:kimi-mapped" in probe.events + assert "exit:kimi:kimi-mapped" in probe.events + # exit 应在 enter 之后 + assert probe.events.index("exit:kimi:kimi-mapped") > probe.events.index( + "enter:kimi:kimi-mapped" + ) + + @pytest.mark.asyncio + async def test_concurrent_message_calls_track_each(self): + """多并发请求每个都触发 enter/exit；in_flight 峰值正确.""" + import asyncio + + probe = _TrackingProbe() + vendor = _mock_vendor_with_probe(probe, name="copilot") + + async def slow_send(_body, _headers): + probe.events.append("send:copilot") + await asyncio.sleep(0.05) + return VendorResponse( + status_code=200, + raw_body=b"{}", + usage=UsageInfo(input_tokens=1, output_tokens=1), + ) + + vendor.send_message = AsyncMock(side_effect=slow_send) + exec_inst = _make_executor(vendor) + + tasks = [ + asyncio.create_task(exec_inst.execute_message({"model": "claude-test"}, {})) + for _ in range(5) + ] + results = await asyncio.gather(*tasks) + assert all(r.status_code == 200 for r in results) + # 5 个 enter + 5 个 send + 5 个 exit + assert sum(1 for e in probe.events if e.startswith("enter:")) == 5 + assert sum(1 for e in probe.events if e.startswith("exit:")) == 5 + # 并发期间 in_flight 峰值应达到 5（monitor 模式不限流） + assert probe.peak_in_flight == 5 + + +class TestExecuteStreamInFlightTracking: + """流式调用的 track_in_flight 包裹行为.""" + + @pytest.mark.asyncio + async def test_stream_track_enter_exit(self): + """成功流式：enter → stream → exit 顺序.""" + probe = _TrackingProbe() + vendor = _mock_vendor_with_probe(probe, name="doubao") + exec_inst = _make_executor(vendor) + + chunks = [] + async for chunk, name in exec_inst.execute_stream({"model": "claude-test"}, {}): + chunks.append((chunk, name)) + + assert len(chunks) >= 1 + # 应包含 enter:doubao:doubao-mapped 与 exit:doubao:doubao-mapped + assert "enter:doubao:doubao-mapped" in probe.events + assert "exit:doubao:doubao-mapped" in probe.events + # exit 在 stream_end 之后 + assert probe.events.index("exit:doubao:doubao-mapped") > probe.events.index( + "stream_end:doubao" + ) + + @pytest.mark.asyncio + async def test_stream_track_exits_on_error(self): + """流式异常退出时 track exit 仍触发.""" + probe = _TrackingProbe() + vendor = _mock_vendor_with_probe(probe, name="minimax") + + async def error_stream(_body, _headers): + yield b'data: {"type":"message_start"}\n\n' + raise httpx.HTTPStatusError( + "500", + request=httpx.Request("POST", "https://example.com"), + response=httpx.Response(500), + ) + + vendor.send_message_stream = MagicMock(side_effect=error_stream) + exec_inst = _make_executor(vendor) + + with pytest.raises(httpx.HTTPStatusError): + async for _ in exec_inst.execute_stream({"model": "claude-test"}, {}): + pass + + assert "enter:minimax:minimax-mapped" in probe.events + assert "exit:minimax:minimax-mapped" in probe.events diff --git a/tests/test_retry.py b/tests/test_retry.py new file mode 100644 index 0000000..dc38c06 --- /dev/null +++ b/tests/test_retry.py @@ -0,0 +1,98 @@ +"""routing.retry 模块单元测试 — calculate_delay（指数退避 + Equal Jitter）.""" + +import random + +from coding.proxy.routing.retry import RetryConfig, calculate_delay + +# Equal Jitter 蒙特卡洛采样次数（纯计算，开销可忽略） +_SAMPLES = 500 + + +def _cfg(**overrides) -> RetryConfig: + """构造测试用 RetryConfig（Zhipu 实际配置为默认基线）.""" + defaults = dict( + max_retries=4, + initial_delay_ms=1000, + max_delay_ms=30000, + backoff_multiplier=2.0, + jitter=True, + ) + defaults.update(overrides) + return RetryConfig(**defaults) + + +# --- 无抖动：精确指数（回归基线）--- + + +def test_calculate_delay_no_jitter_exact_exponential(): + """jitter=False 时延迟为纯指数 1000 → 2000 → 4000 → 8000 ms.""" + cfg = _cfg(jitter=False) + assert calculate_delay(0, cfg) == 1000.0 + assert calculate_delay(1, cfg) == 2000.0 + assert calculate_delay(2, cfg) == 4000.0 + assert calculate_delay(3, cfg) == 8000.0 + + +# --- Equal Jitter：区间边界 --- + + +def test_calculate_delay_equal_jitter_bounds(): + """Equal Jitter 落在 [temp/2, temp]：[500,1000]/[1000,2000]/[2000,4000]/[4000,8000].""" + cfg = _cfg(jitter=True) + bounds = [(500.0, 1000.0), (1000.0, 2000.0), (2000.0, 4000.0), (4000.0, 8000.0)] + for attempt, (low, high) in enumerate(bounds): + for _ in range(_SAMPLES): + delay = calculate_delay(attempt, cfg) + assert low <= delay <= high, ( + f"attempt={attempt} delay={delay} 越界 [{low}, {high}]" + ) + + +def test_calculate_delay_capped_at_max(): + """触及 max_delay_ms 封顶后，区间退化为 [max/2, max] = [15000, 30000].""" + cfg = _cfg(jitter=True) + # attempt=5: 1000 * 2^5 = 32000 > max=30000 → temp=30000 + for _ in range(_SAMPLES): + delay = calculate_delay(5, cfg) + assert 15000.0 <= delay <= 30000.0 + + +# --- 单调非递减（multiplier=2.0 下数学保证）--- + + +def test_calculate_delay_monotonic_non_decreasing(): + """相邻重试延迟单调非递减（multiplier>=2.0 且未封顶时成立）. + + 数学保证：delay[i] ∈ [temp_i/2, temp_i]，delay[i+1] ∈ [temp_i, 2·temp_i]， + 故 delay[i] <= temp_i <= delay[i+1] 恒成立（用 <= 而非 <，体现边界相切）。 + """ + cfg = _cfg(jitter=True) + for _ in range(_SAMPLES): + delays = [calculate_delay(a, cfg) for a in range(4)] + for i in range(len(delays) - 1): + assert delays[i] <= delays[i + 1], ( + f"非单调：delays={delays}（index {i} > {i + 1}）" + ) + + +# --- 健壮性：极小 initial 不报错 --- + + +def test_calculate_delay_tiny_initial_no_error(): + """initial_delay_ms 极小值时不抛异常，仍落在合法区间.""" + cfg = _cfg(initial_delay_ms=1, jitter=True) + delay = calculate_delay(0, cfg) # temp=1 → [0.5, 1.0] + assert 0.5 <= delay <= 1.0 + + +# --- 可复现性：固定 random seed --- + + +def test_calculate_delay_reproducible_with_seed(): + """固定 random seed 后延迟序列可复现（验证 jitter 可观测、可调试）.""" + cfg = _cfg(jitter=True) + random.seed(42) + first = [calculate_delay(a, cfg) for a in range(4)] + random.seed(42) + second = [calculate_delay(a, cfg) for a in range(4)] + assert first == second diff --git a/tests/test_router_executor.py b/tests/test_router_executor.py index 9506e67..837edcc 100644 --- a/tests/test_router_executor.py +++ b/tests/test_router_executor.py @@ -9,7 +9,7 @@ from __future__ import annotations -from unittest.mock import AsyncMock, MagicMock +from unittest.mock import AsyncMock, MagicMock, patch import httpx import pytest @@ -19,11 +19,17 @@ CompatibilityStatus, build_canonical_request, ) +from coding.proxy.config.session_policy import TitleVendorBinding from coding.proxy.routing.executor import ( + _FALLBACK_TITLE_MAX_LEN, _SESSION_TITLE_MAX_LEN, _VENDOR_PROTOCOL_LABEL_MAP, _build_semantic_rejection_diagnostic, _extract_session_title, + _extract_title_from_images, + _extract_title_from_metadata, + _extract_title_from_tool_results, + _extract_title_from_user_text, _has_tool_results, _is_likely_request_format_error, _log_vendor_response_error, @@ -31,6 +37,7 @@ _sanitize_user_text, ) from coding.proxy.routing.session_manager import RouteSessionManager +from coding.proxy.routing.session_policy import SessionPolicyResolver from coding.proxy.routing.tier import VendorTier from coding.proxy.routing.usage_recorder import UsageRecorder from coding.proxy.vendors.base import ( @@ -90,6 +97,11 @@ def _supports_request(request_caps: RequestCapabilities): vendor.check_health = AsyncMock(return_value=True) vendor.close = AsyncMock() vendor.set_compat_context = MagicMock() + + # track_in_flight 返回 nullcontext（不影响执行流，仅满足 async with 协议） + from contextlib import nullcontext + + vendor.track_in_flight = MagicMock(return_value=nullcontext()) return vendor @@ -123,6 +135,19 @@ def _executor(tiers: list[VendorTier] | None = None, **kwargs) -> _RouteExecutor ) +def _stub_session_manager(is_new: bool = True) -> MagicMock: + """构造返回指定 is_new 的 session manager stub. + + 默认 RouteSessionManager(无 store) 的 get_or_create_record 恒返回 + is_new=False；测试新 session 路径需显式 stub 返回 is_new=True。 + """ + mgr = MagicMock(spec=RouteSessionManager) + mgr.get_or_create_record = AsyncMock(return_value=(None, is_new)) + mgr.apply_compat_context = MagicMock() + mgr.persist_session = AsyncMock() + return mgr + + # ── _VENDOR_PROTOCOL_LABEL_MAP ─────────────────────────── @@ -2244,6 +2269,63 @@ def test_strips_local_command_output(self): raw = "build ok构建后的下一步问题" assert _sanitize_user_text(raw) == "构建后的下一步问题" + def test_strips_session_tag(self): + """```` 标签应被完整剥离,不残留在标题中.""" + raw = "session metadata用户真实输入文本" + assert _sanitize_user_text(raw) == "用户真实输入文本" + + def test_strips_session_tag_multiline(self): + raw = "\nline1\nline2\n真实标题" + assert _sanitize_user_text(raw) == "真实标题" + + def test_strips_artifact_metadata_tag(self): + """```` 标签应被完整剥离.""" + raw = "artifact context用户文本" + assert _sanitize_user_text(raw) == "用户文本" + + def test_strips_thinking_tag(self): + """```` 标签应被完整剥离.""" + raw = "内部推理过程用户实际提问" + assert _sanitize_user_text(raw) == "用户实际提问" + + def test_strips_thinking_tag_multiline(self): + raw = "\nline1\nline2\n清理后文本" + assert _sanitize_user_text(raw) == "清理后文本" + + # ── 标签包裹用户文本的二次回退 ── + + def test_session_tag_wrapping_user_text(self): + """当 标签包裹用户文本时,二次回退应提取内部文本. + + 注: session 元数据可能残留在标题前部,但用户文本现在可见, + 远优于完全回退到 '[Session] model_name'. + """ + raw = "session metadata\n用户真实提问内容" + result = _sanitize_user_text(raw) + assert "用户真实提问内容" in result + + def test_session_tag_wrapping_with_inner_noise(self): + """ 内部混合噪声标签时,二次回退应正确剥离噪声.""" + raw = ( + "session_key: abc\n" + "噪声内容" + "用户真实输入" + "" + ) + result = _sanitize_user_text(raw) + assert "用户真实输入" in result + assert "噪声内容" not in result + + def test_session_tag_prefix_still_works(self): + """用户文本在 标签之后(原有行为)仍正确.""" + raw = "metadata用户文本在外部" + assert _sanitize_user_text(raw) == "用户文本在外部" + + def test_all_noise_inside_session_tag(self): + """ 内部全是噪声时,二次回退仍返回空.""" + raw = "纯噪声" + assert _sanitize_user_text(raw) == "" + class TestExtractSessionTitle: """``_extract_session_title`` — 端到端从 CanonicalRequest 抽取标题.""" @@ -2253,7 +2335,7 @@ def _build_request(messages: list[dict]): return build_canonical_request({"model": "test", "messages": messages}, {}) def test_truncates_to_max_len(self): - long_text = "用户输入文本" * 20 + long_text = "用户输入文本" * 200 req = self._build_request([{"role": "user", "content": long_text}]) title = _extract_session_title(req) assert len(title) == _SESSION_TITLE_MAX_LEN @@ -2290,14 +2372,16 @@ def test_extracts_slash_command(self): req = self._build_request([{"role": "user", "content": raw}]) assert _extract_session_title(req) == "/commit feat: 新增标题清洗" - def test_returns_empty_when_only_noise(self): + def test_returns_metadata_fallback_when_only_noise(self): + """纯噪声文本回退到 Level 4 元数据兜底(使用 model 名称).""" raw = "纯噪声" req = self._build_request([{"role": "user", "content": raw}]) - assert _extract_session_title(req) == "" + assert _extract_session_title(req) == "[Session] test" - def test_returns_empty_for_no_user_messages(self): + def test_returns_metadata_fallback_for_no_user_messages(self): + """无 user 消息时回退到 Level 4 元数据兜底.""" req = self._build_request([{"role": "assistant", "content": "你好"}]) - assert _extract_session_title(req) == "" + assert _extract_session_title(req) == "[Session] test" def test_skips_noise_only_part_to_find_real_input(self): """首个 user text part 全噪声时,fallback 到下一个非空 user part.""" @@ -2324,3 +2408,906 @@ def test_skips_assistant_role(self): ] req = self._build_request(messages) assert _extract_session_title(req) == "新的用户问题" + + # ── 豁免前缀端到端（exempt_prefixes 透传至 Level 1）── + + def test_exempt_prefix_falls_through_to_next_user_text(self): + """端到端：L1 首条命中豁免 → 取第二条 user text 作为标题.""" + messages = [ + {"role": "user", "content": "Write the title in the language ..."}, + {"role": "user", "content": "端到端业务标题"}, + ] + req = self._build_request(messages) + assert ( + _extract_session_title( + req, exempt_prefixes=["Write the title in the language"] + ) + == "端到端业务标题" + ) + + def test_exempt_l1_falls_back_to_l2_tool_result(self): + """L1 命中豁免且无其他 user text → 回退 Level 2 tool_result.""" + messages = [ + {"role": "user", "content": "Write the title in the language ..."}, + { + "role": "user", + "content": [ + { + "type": "tool_result", + "tool_use_id": "tu_1", + "content": [{"type": "text", "text": "文件内容摘要"}], + } + ], + }, + ] + req = self._build_request(messages) + title = _extract_session_title( + req, exempt_prefixes=["Write the title in the language"] + ) + assert title == "[Tool output] 文件内容摘要" + + def test_exempt_l1_falls_back_to_l4_metadata(self): + """L1 命中豁免且无 L2/L3 → 回退 Level 4 元数据兜底.""" + messages = [{"role": "user", "content": "Write the title in the language ..."}] + req = self._build_request(messages) + assert ( + _extract_session_title( + req, exempt_prefixes=["Write the title in the language"] + ) + == "[Session] test" + ) + + def test_exempt_does_not_affect_l2(self): + """豁免仅作用于 L1：tool_result 文本以豁免前缀开头，L2 仍正常提取.""" + messages = [ + { + "role": "user", + "content": [ + { + "type": "tool_result", + "tool_use_id": "tu_1", + "content": [ + { + "type": "text", + "text": "Write the title in the language ...", + } + ], + } + ], + }, + ] + req = self._build_request(messages) + title = _extract_session_title( + req, exempt_prefixes=["Write the title in the language"] + ) + # L2 不受豁免影响，正常提取并加 [Tool output] 前缀 + assert title == "[Tool output] Write the title in the language ..." + + # ── 豁免前缀对合成兜底标题（Level 2/3/4）统一生效 ── + + def test_exempt_l4_session_title_returns_empty(self): + """L4 兜底 [Session] 标题命中豁免前缀 → 回退耗尽 → 返回空串.""" + req = self._build_request( + [{"role": "user", "content": "纯噪声"}] + ) + # L1 噪声剥离为空 → L4 产出 "[Session] test" → 命中 "[Session]" 豁免 + assert _extract_session_title(req, exempt_prefixes=["[Session]"]) == "" + + def test_exempt_l4_tool_call_title_returns_empty(self): + """L4 兜底 [Tool call] 标题命中豁免前缀 → 返回空串.""" + req = build_canonical_request( + {"model": "test", "messages": [], "tools": [{"name": "Bash"}]}, {} + ) + assert _extract_session_title(req, exempt_prefixes=["[Tool call]"]) == "" + + def test_exempt_l4_unmatched_keeps_original(self): + """豁免前缀不命中 L4 实际标题时维持原行为（不误豁免）.""" + req = build_canonical_request( + {"model": "test", "messages": [], "tools": [{"name": "Bash"}]}, {} + ) + # L4 产出 "[Tool call] Bash"，豁免前缀 "[Session]" 不命中 → 原样返回 + assert ( + _extract_session_title(req, exempt_prefixes=["[Session]"]) + == "[Tool call] Bash" + ) + + def test_exempt_l2_tool_output_falls_through_to_l4(self): + """L2 合成标题 [Tool output] 命中豁免 → 跳过 L2、回退到 L4.""" + messages = [ + { + "role": "user", + "content": [ + { + "type": "tool_result", + "tool_use_id": "tu_1", + "content": [{"type": "text", "text": "文件内容摘要"}], + } + ], + }, + ] + req = self._build_request(messages) + # L1 无 user TEXT → L2 "[Tool output] 文件内容摘要" 命中豁免 → 跳过 → + # L3 无图 → L4 "[Session] test"（未命中豁免，正常返回） + assert ( + _extract_session_title(req, exempt_prefixes=["[Tool output]"]) + == "[Session] test" + ) + + def test_exempt_l3_image_falls_through_to_l4(self): + """L3 合成标题 [1 Image] 命中豁免 → 跳过 L3、回退到 L4.""" + messages = [ + { + "role": "user", + "content": [ + { + "type": "image", + "source": { + "type": "base64", + "media_type": "image/png", + "data": "abc", + }, + } + ], + } + ] + req = self._build_request(messages) + assert ( + _extract_session_title(req, exempt_prefixes=["[1 Image]"]) + == "[Session] test" + ) + + def test_exempt_all_synthetic_layers_returns_empty(self): + """四层合成标题全部进豁免名单 → 全部被拦截 → 返回空串（永久空标题可接受）.""" + req = self._build_request( + [{"role": "user", "content": "纯噪声"}] + ) + assert ( + _extract_session_title( + req, + exempt_prefixes=[ + "[Tool output]", + "[1 Image]", + "[Tool call]", + "[Session]", + ], + ) + == "" + ) + + +# ═══════════════════════════════════════════════════════════════════ +# 多层级回退标题提取测试 +# ═══════════════════════════════════════════════════════════════════ + + +class TestExtractTitleFromUserText: + """Level 1 辅助函数 ``_extract_title_from_user_text``.""" + + def test_returns_first_non_empty_user_text(self): + from coding.proxy.model.compat import CanonicalMessagePart, CanonicalPartType + + msgs = [ + CanonicalMessagePart( + type=CanonicalPartType.TEXT, role="user", text="用户输入" + ), + ] + assert _extract_title_from_user_text(msgs) == "用户输入" + + def test_skips_assistant_text(self): + from coding.proxy.model.compat import CanonicalMessagePart, CanonicalPartType + + msgs = [ + CanonicalMessagePart( + type=CanonicalPartType.TEXT, role="assistant", text="助手回复" + ), + CanonicalMessagePart( + type=CanonicalPartType.TEXT, role="user", text="用户问题" + ), + ] + assert _extract_title_from_user_text(msgs) == "用户问题" + + def test_returns_empty_for_noise_only(self): + from coding.proxy.model.compat import CanonicalMessagePart, CanonicalPartType + + msgs = [ + CanonicalMessagePart( + type=CanonicalPartType.TEXT, + role="user", + text="纯噪声", + ), + ] + assert _extract_title_from_user_text(msgs) == "" + + # ── 豁免前缀（exempt_prefixes）—— 过滤注入式 Prompt ── + + def test_exempt_prefix_skips_first_falls_through_to_second(self): + """豁免前缀命中首条 user text → 跳过、返回第二条.""" + from coding.proxy.model.compat import CanonicalMessagePart, CanonicalPartType + + msgs = [ + CanonicalMessagePart( + type=CanonicalPartType.TEXT, + role="user", + text="Write the title in the language the user wrote in, regardless.", + ), + CanonicalMessagePart( + type=CanonicalPartType.TEXT, role="user", text="帮我重构 executor" + ), + ] + assert ( + _extract_title_from_user_text( + msgs, exempt_prefixes=["Write the title in the language"] + ) + == "帮我重构 executor" + ) + + def test_exempt_prefix_case_sensitive(self): + """大小写敏感：小写前缀不命中大写开头的输入.""" + from coding.proxy.model.compat import CanonicalMessagePart, CanonicalPartType + + msgs = [ + CanonicalMessagePart( + type=CanonicalPartType.TEXT, + role="user", + text="Write the title in the language the user wrote in.", + ), + ] + # 前缀小写、输入首字母大写 → 不命中，正常返回 + assert ( + _extract_title_from_user_text(msgs, exempt_prefixes=["write the title"]) + == "Write the title in the language the user wrote in." + ) + + def test_exempt_no_param_backward_compatible(self): + """不传 exempt_prefixes → 行为与历史一致（首条非空即返回）.""" + from coding.proxy.model.compat import CanonicalMessagePart, CanonicalPartType + + msgs = [ + CanonicalMessagePart( + type=CanonicalPartType.TEXT, + role="user", + text="Write the title in the language the user wrote in.", + ), + ] + assert _extract_title_from_user_text(msgs) == ( + "Write the title in the language the user wrote in." + ) + + def test_exempt_empty_list_no_op(self): + """传空列表 → 等价于不豁免.""" + from coding.proxy.model.compat import CanonicalMessagePart, CanonicalPartType + + msgs = [ + CanonicalMessagePart( + type=CanonicalPartType.TEXT, + role="user", + text="Write the title in the language the user wrote in.", + ), + ] + assert _extract_title_from_user_text(msgs, exempt_prefixes=[]) == ( + "Write the title in the language the user wrote in." + ) + + def test_exempt_all_user_inputs_skipped_returns_empty(self): + """全部 user text 命中豁免 → 返回空字符串.""" + from coding.proxy.model.compat import CanonicalMessagePart, CanonicalPartType + + msgs = [ + CanonicalMessagePart( + type=CanonicalPartType.TEXT, + role="user", + text="Write the title in the language variant A", + ), + CanonicalMessagePart( + type=CanonicalPartType.TEXT, + role="user", + text="Write the title in the language variant B", + ), + ] + assert ( + _extract_title_from_user_text( + msgs, exempt_prefixes=["Write the title in the language"] + ) + == "" + ) + + def test_exempt_matches_cleaned_not_raw(self): + """豁免判断作用于清洗后文本：raw 含 system-reminder 包裹、清洗后命中前缀 → 跳过.""" + from coding.proxy.model.compat import CanonicalMessagePart, CanonicalPartType + + raw = ( + "注入的系统上下文" + "Write the title in the language the user wrote in." + ) + msgs = [ + CanonicalMessagePart(type=CanonicalPartType.TEXT, role="user", text=raw), + CanonicalMessagePart( + type=CanonicalPartType.TEXT, role="user", text="真实业务问题" + ), + ] + assert ( + _extract_title_from_user_text( + msgs, exempt_prefixes=["Write the title in the language"] + ) + == "真实业务问题" + ) + + def test_exempt_multi_prefix_any_match(self): + """多前缀：命中任一即跳过.""" + from coding.proxy.model.compat import CanonicalMessagePart, CanonicalPartType + + msgs = [ + CanonicalMessagePart( + type=CanonicalPartType.TEXT, + role="user", + text="IGNORE_PREFIX_A something", + ), + CanonicalMessagePart( + type=CanonicalPartType.TEXT, role="user", text="第二输入" + ), + ] + assert ( + _extract_title_from_user_text( + msgs, exempt_prefixes=["IGNORE_PREFIX_A", "IGNORE_PREFIX_B"] + ) + == "第二输入" + ) + + +class TestExtractTitleFromToolResults: + """Level 2 辅助函数 ``_extract_title_from_tool_results``.""" + + def test_extracts_tool_result_text(self): + from coding.proxy.model.compat import CanonicalMessagePart, CanonicalPartType + + msgs = [ + CanonicalMessagePart( + type=CanonicalPartType.TOOL_RESULT, + role="user", + text="file contents here", + ), + ] + title = _extract_title_from_tool_results(msgs) + assert title == "[Tool output] file contents here" + + def test_skips_empty_tool_result(self): + from coding.proxy.model.compat import CanonicalMessagePart, CanonicalPartType + + msgs = [ + CanonicalMessagePart( + type=CanonicalPartType.TOOL_RESULT, role="user", text="" + ), + ] + assert _extract_title_from_tool_results(msgs) == "" + + def test_truncates_long_tool_result(self): + from coding.proxy.model.compat import CanonicalMessagePart, CanonicalPartType + + long_text = "A" * 200 + msgs = [ + CanonicalMessagePart( + type=CanonicalPartType.TOOL_RESULT, role="user", text=long_text + ), + ] + title = _extract_title_from_tool_results(msgs) + assert title.startswith("[Tool output] ") + assert len(title) <= len("[Tool output] ") + _FALLBACK_TITLE_MAX_LEN + + def test_sanitizes_noise_in_tool_result(self): + from coding.proxy.model.compat import CanonicalMessagePart, CanonicalPartType + + msgs = [ + CanonicalMessagePart( + type=CanonicalPartType.TOOL_RESULT, + role="user", + text="noiseclean output", + ), + ] + title = _extract_title_from_tool_results(msgs) + assert title == "[Tool output] clean output" + + def test_returns_empty_when_all_noise(self): + from coding.proxy.model.compat import CanonicalMessagePart, CanonicalPartType + + msgs = [ + CanonicalMessagePart( + type=CanonicalPartType.TOOL_RESULT, + role="user", + text="纯噪声", + ), + ] + assert _extract_title_from_tool_results(msgs) == "" + + +class TestExtractTitleFromImages: + """Level 3 辅助函数 ``_extract_title_from_images``.""" + + def test_single_image(self): + from coding.proxy.model.compat import CanonicalMessagePart, CanonicalPartType + + msgs = [ + CanonicalMessagePart(type=CanonicalPartType.IMAGE, role="user"), + ] + assert _extract_title_from_images(msgs) == "[1 Image]" + + def test_multiple_images(self): + from coding.proxy.model.compat import CanonicalMessagePart, CanonicalPartType + + msgs = [ + CanonicalMessagePart(type=CanonicalPartType.IMAGE, role="user"), + CanonicalMessagePart(type=CanonicalPartType.IMAGE, role="user"), + CanonicalMessagePart(type=CanonicalPartType.IMAGE, role="user"), + ] + assert _extract_title_from_images(msgs) == "[3 Images]" + + def test_no_images(self): + from coding.proxy.model.compat import CanonicalMessagePart, CanonicalPartType + + msgs = [ + CanonicalMessagePart(type=CanonicalPartType.TEXT, role="user", text="文本"), + ] + assert _extract_title_from_images(msgs) == "" + + def test_skips_assistant_images(self): + from coding.proxy.model.compat import CanonicalMessagePart, CanonicalPartType + + msgs = [ + CanonicalMessagePart(type=CanonicalPartType.IMAGE, role="assistant"), + ] + assert _extract_title_from_images(msgs) == "" + + +class TestExtractTitleFromMetadata: + """Level 4 辅助函数 ``_extract_title_from_metadata``.""" + + @staticmethod + def _build_request_with_meta(tool_names: list[str] | None = None, model: str = ""): + body: dict = {"model": model, "messages": []} + if tool_names: + body["tools"] = [{"name": n} for n in tool_names] + return build_canonical_request(body, {}) + + def test_uses_tool_names(self): + req = self._build_request_with_meta( + tool_names=["Bash", "Read", "Edit"], model="claude-opus-4-8" + ) + assert _extract_title_from_metadata(req) == "[Tool call] Bash, Read, Edit" + + def test_limits_to_three_tool_names(self): + req = self._build_request_with_meta( + tool_names=["Bash", "Read", "Edit", "Write", "Grep"], model="test" + ) + assert _extract_title_from_metadata(req) == "[Tool call] Bash, Read, Edit" + + def test_uses_model_when_no_tools(self): + req = self._build_request_with_meta(tool_names=[], model="claude-sonnet-4-6") + assert _extract_title_from_metadata(req) == "[Session] claude-sonnet-4-6" + + def test_returns_empty_when_nothing(self): + req = self._build_request_with_meta(tool_names=[], model="") + assert _extract_title_from_metadata(req) == "" + + +class TestExtractSessionTitleFallback: + """``_extract_session_title`` 多层级回退集成测试.""" + + @staticmethod + def _build_request(messages: list[dict], **extra): + body: dict = {"model": "test-model", "messages": messages, **extra} + return build_canonical_request(body, {}) + + def test_level1_takes_priority(self): + """Level 1 命中时不回退到 Level 2.""" + messages = [ + { + "role": "user", + "content": [ + {"type": "text", "text": "用户真实问题"}, + { + "type": "tool_result", + "tool_use_id": "tu_1", + "content": "工具输出", + }, + ], + } + ] + req = self._build_request(messages) + assert _extract_session_title(req) == "用户真实问题" + + def test_level2_when_no_text(self): + """无 user TEXT 时,回退到 Level 2 TOOL_RESULT.""" + messages = [ + { + "role": "user", + "content": [ + { + "type": "tool_result", + "tool_use_id": "tu_1", + "content": [{"type": "text", "text": "文件内容摘要"}], + }, + ], + } + ] + req = self._build_request(messages) + assert _extract_session_title(req) == "[Tool output] 文件内容摘要" + + def test_level3_when_only_images(self): + """无 TEXT 和 TOOL_RESULT 时,回退到 Level 3 IMAGE.""" + messages = [ + { + "role": "user", + "content": [ + { + "type": "image", + "source": { + "type": "base64", + "media_type": "image/png", + "data": "abc", + }, + }, + ], + } + ] + req = self._build_request(messages) + assert _extract_session_title(req) == "[1 Image]" + + def test_level4_uses_tool_names(self): + """所有消息级别均无内容时,回退到 Level 4 元数据.""" + req = self._build_request([], tools=[{"name": "Bash"}, {"name": "Read"}]) + assert _extract_session_title(req) == "[Tool call] Bash, Read" + + def test_level4_uses_model_name(self): + """无 tools 时,Level 4 使用 model 名称.""" + req = self._build_request([]) + assert _extract_session_title(req) == "[Session] test-model" + + def test_fallback_cascade_full(self): + """Level 1 全噪声 → Level 2 全噪声 → Level 3 无图 → Level 4 模型名.""" + messages = [ + { + "role": "user", + "content": "纯噪声", + }, + ] + req = self._build_request(messages) + assert _extract_session_title(req) == "[Session] test-model" + + +class TestApplyTitleBasedPolicy: + """``_apply_title_based_policy`` 标题前缀自动绑定测试.""" + + def test_prefix_match_triggers_upsert(self): + """标题以配置前缀开头 → 触发 upsert 绑定到目标 vendor.""" + resolver = SessionPolicyResolver() + executor = _executor( + session_policy_resolver=resolver, + title_vendor_bindings=[TitleVendorBinding(prefix="# 目标", vendor="zhipu")], + ) + executor._apply_title_based_policy("sess-1", "# 目标 (Goal) 实现功能 X") + policy = resolver.resolve("sess-1") + assert policy is not None + assert policy.tiers == ["zhipu"] + assert policy.name == "runtime:sess-1" + + def test_prefix_match_without_parenthesis(self): + """前缀匹配不要求括号后缀,纯 '# 目标' 开头即命中.""" + resolver = SessionPolicyResolver() + executor = _executor( + session_policy_resolver=resolver, + title_vendor_bindings=[TitleVendorBinding(prefix="# 目标", vendor="zhipu")], + ) + executor._apply_title_based_policy("sess-2", "# 目标 详细计划") + policy = resolver.resolve("sess-2") + assert policy is not None + assert policy.tiers == ["zhipu"] + + def test_non_matching_title_no_binding(self): + """非匹配标题 → 不创建绑定.""" + resolver = SessionPolicyResolver() + executor = _executor( + session_policy_resolver=resolver, + title_vendor_bindings=[TitleVendorBinding(prefix="# 目标", vendor="zhipu")], + ) + executor._apply_title_based_policy("sess-3", "普通会话标题") + assert resolver.resolve("sess-3") is None + + def test_empty_title_no_binding(self): + """空标题 → 提前返回,不创建绑定.""" + resolver = SessionPolicyResolver() + executor = _executor( + session_policy_resolver=resolver, + title_vendor_bindings=[TitleVendorBinding(prefix="# 目标", vendor="zhipu")], + ) + executor._apply_title_based_policy("sess-4", "") + assert resolver.resolve("sess-4") is None + + def test_no_bindings_configured_no_binding(self): + """未配置任何绑定规则 → 提前返回,等效禁用.""" + resolver = SessionPolicyResolver() + executor = _executor( + session_policy_resolver=resolver, + title_vendor_bindings=[], + ) + executor._apply_title_based_policy("sess-5", "# 目标 任意标题") + assert resolver.resolve("sess-5") is None + + def test_prefix_in_middle_no_match(self): + """前缀出现在标题中间 → startswith 不匹配,不绑定.""" + resolver = SessionPolicyResolver() + executor = _executor( + session_policy_resolver=resolver, + title_vendor_bindings=[TitleVendorBinding(prefix="# 目标", vendor="zhipu")], + ) + executor._apply_title_based_policy("sess-6", "前缀 # 目标 在中间") + assert resolver.resolve("sess-6") is None + + def test_multiple_bindings_first_match_wins(self): + """多条规则按顺序匹配,首次命中生效.""" + resolver = SessionPolicyResolver() + executor = _executor( + session_policy_resolver=resolver, + title_vendor_bindings=[ + TitleVendorBinding(prefix="# 目标", vendor="zhipu"), + TitleVendorBinding(prefix="# Review", vendor="anthropic"), + ], + ) + executor._apply_title_based_policy("sess-7", "# Review 代码审查") + policy = resolver.resolve("sess-7") + assert policy is not None + assert policy.tiers == ["anthropic"] + + def test_bound_tier_promoted_to_front(self): + """绑定后 _resolve_effective_tiers 将目标 vendor 提升至首位.""" + resolver = SessionPolicyResolver() + tiers = [ + _make_tier(_mock_vendor("anthropic")), + _make_tier(_mock_vendor("zhipu")), + ] + executor = _executor( + tiers=tiers, + session_policy_resolver=resolver, + title_vendor_bindings=[TitleVendorBinding(prefix="# 目标", vendor="zhipu")], + ) + executor._apply_title_based_policy("sess-8", "# 目标 实现 X") + effective = executor._resolve_effective_tiers("sess-8") + assert effective[0].name == "zhipu" + # 未提及的 vendor 仍保留在末尾 + assert {t.name for t in effective} == {"zhipu", "anthropic"} + + def test_non_matching_session_uses_default_order(self): + """非匹配 session 的 tier 顺序保持全局默认.""" + resolver = SessionPolicyResolver() + tiers = [ + _make_tier(_mock_vendor("anthropic")), + _make_tier(_mock_vendor("zhipu")), + ] + executor = _executor( + tiers=tiers, + session_policy_resolver=resolver, + title_vendor_bindings=[TitleVendorBinding(prefix="# 目标", vendor="zhipu")], + ) + executor._apply_title_based_policy("sess-9", "普通标题") + effective = executor._resolve_effective_tiers("sess-9") + assert [t.name for t in effective] == ["anthropic", "zhipu"] + + def test_nonexistent_vendor_skipped_in_resolution(self): + """绑定不存在的 vendor → upsert 成功但 tier 解析跳过该 vendor.""" + resolver = SessionPolicyResolver() + tiers = [ + _make_tier(_mock_vendor("anthropic")), + _make_tier(_mock_vendor("zhipu")), + ] + executor = _executor( + tiers=tiers, + session_policy_resolver=resolver, + title_vendor_bindings=[ + TitleVendorBinding(prefix="# 目标", vendor="nonexistent") + ], + ) + executor._apply_title_based_policy("sess-10", "# 目标 X") + effective = executor._resolve_effective_tiers("sess-10") + # 不存在的 vendor 被跳过,回退到全局默认顺序 + assert [t.name for t in effective] == ["anthropic", "zhipu"] + + @pytest.mark.asyncio + async def test_execute_message_end_to_end_binding(self): + """端到端: 新 session 首请求标题命中前缀 → 创建绑定并路由到 zhipu.""" + resolver = SessionPolicyResolver() + tiers = [ + _make_tier(_mock_vendor("anthropic")), + _make_tier(_mock_vendor("zhipu")), + ] + executor = _executor( + tiers=tiers, + session_mgr=_stub_session_manager(is_new=True), + session_policy_resolver=resolver, + title_vendor_bindings=[TitleVendorBinding(prefix="# 目标", vendor="zhipu")], + ) + body = { + "model": "test", + "metadata": {"user_id": "session-abc"}, + "messages": [ + {"role": "user", "content": [{"type": "text", "text": "# 目标 实现 X"}]} + ], + } + resp = await executor.execute_message(body, {}) + assert resp.status_code == 200 + # 从 body 解析出的 session_key 应已建立运行时绑定 + canonical = build_canonical_request(body, {}) + policy = resolver.resolve(canonical.session_key) + assert policy is not None + assert policy.tiers == ["zhipu"] + + @pytest.mark.asyncio + async def test_execute_message_existing_session_no_binding(self): + """端到端: 已存在 session(is_new=False) 不触发标题绑定.""" + resolver = SessionPolicyResolver() + tiers = [ + _make_tier(_mock_vendor("anthropic")), + _make_tier(_mock_vendor("zhipu")), + ] + executor = _executor( + tiers=tiers, + session_mgr=_stub_session_manager(is_new=False), + session_policy_resolver=resolver, + title_vendor_bindings=[TitleVendorBinding(prefix="# 目标", vendor="zhipu")], + ) + body = { + "model": "test", + "metadata": {"user_id": "session-existing"}, + "messages": [ + {"role": "user", "content": [{"type": "text", "text": "# 目标 实现 X"}]} + ], + } + await executor.execute_message(body, {}) + canonical = build_canonical_request(body, {}) + # is_new=False → 不调用 _apply_title_based_policy,无运行时绑定 + assert resolver.resolve(canonical.session_key) is None + + @pytest.mark.asyncio + async def test_execute_stream_end_to_end_binding(self): + """端到端(流式): 新 session 首请求标题命中前缀 → 创建绑定.""" + resolver = SessionPolicyResolver() + zhipu_vendor = _mock_vendor("zhipu") + zhipu_vendor.send_message_stream = MagicMock( + return_value=_async_chunks([b'{"type":"message_stop"}']) + ) + tiers = [ + _make_tier(_mock_vendor("anthropic")), + _make_tier(zhipu_vendor), + ] + executor = _executor( + tiers=tiers, + session_mgr=_stub_session_manager(is_new=True), + session_policy_resolver=resolver, + title_vendor_bindings=[TitleVendorBinding(prefix="# 目标", vendor="zhipu")], + ) + body = { + "model": "test", + "metadata": {"user_id": "session-stream"}, + "messages": [ + { + "role": "user", + "content": [{"type": "text", "text": "# 目标 流式任务"}], + } + ], + } + chunks = [chunk async for chunk, _ in executor.execute_stream(body, {})] + assert chunks # 有数据返回 + canonical = build_canonical_request(body, {}) + policy = resolver.resolve(canonical.session_key) + assert policy is not None + assert policy.tiers == ["zhipu"] + + def test_empty_prefix_rejected_by_validation(self): + """空 prefix 在模型校验阶段即被拒绝,杜绝全量误绑定.""" + import pydantic + + with pytest.raises(pydantic.ValidationError): + TitleVendorBinding(prefix="", vendor="zhipu") + + def test_empty_vendor_rejected_by_validation(self): + """空 vendor 在模型校验阶段即被拒绝.""" + import pydantic + + with pytest.raises(pydantic.ValidationError): + TitleVendorBinding(prefix="# 目标", vendor="") + + def test_unknown_vendor_warns_at_startup(self, caplog): + """构造时引用未知 vendor → 记录启动告警.""" + import logging as _logging + + tiers = [_make_tier(_mock_vendor("anthropic"))] + with caplog.at_level(_logging.WARNING, logger="coding.proxy.routing.executor"): + _executor( + tiers=tiers, + session_policy_resolver=SessionPolicyResolver(), + title_vendor_bindings=[ + TitleVendorBinding(prefix="# 目标", vendor="nonexistent") + ], + ) + warnings = [r for r in caplog.records if r.levelno == _logging.WARNING] + assert any("nonexistent" in r.message for r in warnings) + + def test_known_vendor_no_startup_warning(self, caplog): + """构造时引用合法 vendor → 不产生告警.""" + import logging as _logging + + tiers = [_make_tier(_mock_vendor("zhipu"))] + with caplog.at_level(_logging.WARNING, logger="coding.proxy.routing.executor"): + _executor( + tiers=tiers, + session_policy_resolver=SessionPolicyResolver(), + title_vendor_bindings=[ + TitleVendorBinding(prefix="# 目标", vendor="zhipu") + ], + ) + binding_warnings = [ + r for r in caplog.records if "title_vendor_bindings" in r.message + ] + assert not binding_warnings + + +class TestExemptPrefixInjection: + """``_RouteExecutor._extract_session_title`` 实例方法注入豁免前缀名单.""" + + @staticmethod + def _build_request(messages: list[dict]): + return build_canonical_request({"model": "test", "messages": messages}, {}) + + def test_executor_injects_exempt_prefixes_to_l1(self): + """executor 持有的豁免前缀经实例方法注入，L1 跳过注入式 Prompt.""" + req = self._build_request( + [ + {"role": "user", "content": "Write the title in the language ..."}, + {"role": "user", "content": "注入后的真实标题"}, + ] + ) + executor = _executor(title_exempt_prefixes=["Write the title in the language"]) + assert executor._extract_session_title(req) == "注入后的真实标题" + + def test_executor_without_exempt_prefixes_backward_compatible(self): + """未配置豁免前缀 → 实例方法行为与模块级默认一致（不跳过）.""" + req = self._build_request( + [{"role": "user", "content": "Write the title in the language ..."}] + ) + executor = _executor() # 不传 title_exempt_prefixes + assert executor._extract_session_title(req) == ( + "Write the title in the language ..." + ) + + def test_executor_filters_empty_string_prefix_at_construction(self): + """构造期过滤空串：即便传入空串/纯空白也不会豁免一切.""" + req = self._build_request([{"role": "user", "content": "正常标题"}]) + executor = _executor(title_exempt_prefixes=["", " ", "\t"]) + assert executor._title_exempt_prefixes == [] + assert executor._extract_session_title(req) == "正常标题" + + def test_executor_exempt_l4_returns_empty(self): + """executor 持有 [Session] 豁免 → L4 兜底标题被豁免 → 实例方法返回空串.""" + req = self._build_request( + [{"role": "user", "content": "纯噪声"}] + ) + executor = _executor(title_exempt_prefixes=["[Session]"]) + # L1 噪声剥离为空 → L4 "[Session] test" 命中豁免 → 返回 "" + assert executor._extract_session_title(req) == "" + + @pytest.mark.asyncio + async def test_execute_skips_title_write_when_l4_exempt(self): + """端到端: L4 兜底 [Session] 标题命中豁免 → 标题为空 → set_session_title 不被调用.""" + recorder = UsageRecorder() + executor = _executor( + recorder=recorder, + session_mgr=_stub_session_manager(is_new=True), + title_exempt_prefixes=["[Session]"], + ) + body = { + "model": "claude-opus-4-8", + "metadata": {"user_id": "session-l4"}, + "messages": [ + {"role": "user", "content": "纯噪声"} + ], + } + with patch.object(recorder, "set_session_title", new=AsyncMock()) as spy: + await executor.execute_message(body, {}) + spy.assert_not_called() diff --git a/tests/test_session_aware.py b/tests/test_session_aware.py index 29518e5..9f3441a 100644 --- a/tests/test_session_aware.py +++ b/tests/test_session_aware.py @@ -462,6 +462,30 @@ def test_config_parse(): assert config.policies[0].tiers == ["anthropic", "copilot"] +def test_config_exempt_prefixes_default_empty(): + """未配置时默认空列表.""" + config = SessionPoliciesConfig() + assert config.title_exempt_prefixes == [] + + +def test_config_exempt_prefixes_strips_whitespace(): + """加载时 strip 每个前缀的前后空白.""" + config = SessionPoliciesConfig(title_exempt_prefixes=[" Write ", "\tRead\t"]) + assert config.title_exempt_prefixes == ["Write", "Read"] + + +def test_config_exempt_prefixes_drops_empty_string(): + """空字符串/纯空白前缀必须丢弃——防空串 startswith 恒真导致全量误豁免.""" + config = SessionPoliciesConfig(title_exempt_prefixes=["", " ", "Write"]) + assert config.title_exempt_prefixes == ["Write"] + + +def test_config_exempt_prefixes_dedupes_preserves_order(): + """去重并保留首次出现顺序.""" + config = SessionPoliciesConfig(title_exempt_prefixes=["a", "b", "a", "b", "c"]) + assert config.title_exempt_prefixes == ["a", "b", "c"] + + # ── 9. SessionPolicyResolver 运行时可变性 ──────────────────────── diff --git a/tests/test_vendor_channels.py b/tests/test_vendor_channels.py index f9c9bb5..d99a4c6 100644 --- a/tests/test_vendor_channels.py +++ b/tests/test_vendor_channels.py @@ -2225,3 +2225,221 @@ def test_preserves_supported_params(self): assert result["stream"] is True assert result["metadata"] == {"user_id": "test"} assert adaptations == [] + + +class TestDumpMessageDigest: + """``_dump_message_digest`` 诊断快照函数测试.""" + + def test_outputs_nothing_on_empty_messages(self, caplog): + import logging + + from coding.proxy.convert.vendor_channels import _dump_message_digest + + with caplog.at_level( + logging.DEBUG, logger="coding.proxy.convert.vendor_channels" + ): + _dump_message_digest([], label="test") + assert "Transition digest" not in caplog.text + + def test_outputs_structure_for_first_n_messages(self, caplog): + import logging + + from coding.proxy.convert.vendor_channels import _dump_message_digest + + messages = [ + {"role": "user", "content": [{"type": "text", "text": "hi"}]}, + { + "role": "assistant", + "content": [ + {"type": "tool_use", "id": "toolu_1", "name": "bash", "input": {}}, + ], + }, + { + "role": "user", + "content": [ + {"type": "tool_result", "tool_use_id": "toolu_1", "content": "ok"}, + ], + }, + ] + with caplog.at_level( + logging.DEBUG, logger="coding.proxy.convert.vendor_channels" + ): + _dump_message_digest(messages, label="test") + assert "test" in caplog.text + assert "0:user" in caplog.text + assert "1:assistant" in caplog.text + assert "tool_use:1" in caplog.text + + +class TestValidateAnthropicPairing: + """``_validate_anthropic_pairing`` 独立配对自检测试.""" + + def test_no_issues_for_correct_pairing(self): + from coding.proxy.convert.vendor_channels import _validate_anthropic_pairing + + messages = [ + {"role": "user", "content": "go"}, + { + "role": "assistant", + "content": [ + {"type": "tool_use", "id": "toolu_1", "name": "bash", "input": {}}, + ], + }, + { + "role": "user", + "content": [ + {"type": "tool_result", "tool_use_id": "toolu_1", "content": "ok"}, + ], + }, + ] + issues = _validate_anthropic_pairing(messages) + assert issues == [] + + def test_detects_missing_tool_result(self): + from coding.proxy.convert.vendor_channels import _validate_anthropic_pairing + + messages = [ + {"role": "user", "content": "go"}, + { + "role": "assistant", + "content": [ + {"type": "tool_use", "id": "toolu_1", "name": "bash", "input": {}}, + ], + }, + {"role": "user", "content": [{"type": "text", "text": "no result"}]}, + ] + issues = _validate_anthropic_pairing(messages) + assert len(issues) == 1 + assert "toolu_1" in issues[0] + assert "messages[1]" in issues[0] + + def test_detects_non_user_after_assistant_with_tool_use(self): + from coding.proxy.convert.vendor_channels import _validate_anthropic_pairing + + messages = [ + {"role": "user", "content": "go"}, + { + "role": "assistant", + "content": [ + {"type": "tool_use", "id": "toolu_1", "name": "bash", "input": {}}, + ], + }, + { + "role": "assistant", + "content": [{"type": "text", "text": "another assistant"}], + }, + ] + issues = _validate_anthropic_pairing(messages) + assert len(issues) == 1 + assert "not user" in issues[0] + + def test_detects_assistant_with_tool_use_at_end_of_list(self): + from coding.proxy.convert.vendor_channels import _validate_anthropic_pairing + + messages = [ + {"role": "user", "content": "go"}, + { + "role": "assistant", + "content": [ + {"type": "tool_use", "id": "toolu_1", "name": "bash", "input": {}}, + ], + }, + ] + issues = _validate_anthropic_pairing(messages) + assert len(issues) == 1 + assert "end of list" in issues[0] + + def test_partial_missing_only_reports_missing_ids(self): + from coding.proxy.convert.vendor_channels import _validate_anthropic_pairing + + messages = [ + {"role": "user", "content": "go"}, + { + "role": "assistant", + "content": [ + {"type": "tool_use", "id": "toolu_1", "name": "bash", "input": {}}, + {"type": "tool_use", "id": "toolu_2", "name": "read", "input": {}}, + ], + }, + { + "role": "user", + "content": [ + {"type": "tool_result", "tool_use_id": "toolu_1", "content": "ok"}, + ], + }, + ] + issues = _validate_anthropic_pairing(messages) + assert len(issues) == 1 + assert "toolu_2" in issues[0] + assert "1/2" in issues[0] + + def test_integration_with_zhipu_to_anthropic_channel(self): + """验证 prepare_zhipu_to_anthropic 在末端执行自检且 adaptations 包含标签.""" + from coding.proxy.convert.vendor_channels import prepare_zhipu_to_anthropic + + body = { + "model": "claude-opus-4-7", + "messages": [ + {"role": "user", "content": "go"}, + { + "role": "assistant", + "content": [ + { + "type": "server_tool_use", + "id": "srvtoolu_01", + "name": "bash", + "input": {}, + }, + ], + }, + { + "role": "user", + "content": [ + { + "type": "tool_result", + "tool_use_id": "srvtoolu_01", + "content": "ok", + }, + ], + }, + ], + } + result, adaptations = prepare_zhipu_to_anthropic(body) + # 自检通过，不应包含 validation_issues 标签 + assert "anthropic_pairing_validation_issues" not in adaptations + + def test_integration_detects_enforce_missed_issue(self): + """构造一个理论上 enforce 可能遗漏的场景，验证自检能捕获. + + 场景：两条连续 assistant 消息，第一条的 tool_result 被第二条的 + existing_result_ids"冒领"（相同 ID 碰撞场景的模拟）。 + 虽然当前 enforce 实现下不太可能自然产生此场景，但自检应能捕获。 + """ + from coding.proxy.convert.vendor_channels import ( + _validate_anthropic_pairing, + ) + + # 手动构造一个 enforce 后仍存在配对缺陷的 body + messages = [ + {"role": "user", "content": "go"}, + { + "role": "assistant", + "content": [ + {"type": "tool_use", "id": "toolu_x", "name": "bash", "input": {}}, + ], + }, + { + "role": "user", + "content": [ + # tool_result 缺失 tolu_x，但有不相关的 tool_result + { + "type": "tool_result", + "tool_use_id": "toolu_other", + "content": "wrong", + }, + ], + }, + ] + issues = _validate_anthropic_pairing(messages) + assert len(issues) == 1 + assert "toolu_x" in issues[0] diff --git a/tests/test_zhipu.py b/tests/test_zhipu.py index aa05b21..cf44e7b 100644 --- a/tests/test_zhipu.py +++ b/tests/test_zhipu.py @@ -356,6 +356,20 @@ def _make_429_response( ) +def _make_529_response( + headers: dict[str, str] | None = None, +) -> httpx.Response: + """构造 529 HTTP 响应（Overloaded / 并发过载）.""" + return httpx.Response( + status_code=529, + content=b'{"error":{"type":"overloaded_error","message":"Overloaded"}}', + headers=headers or {}, + request=httpx.Request( + "POST", "https://open.bigmodel.cn/api/anthropic/v1/messages" + ), + ) + + def _make_200_response() -> httpx.Response: """构造 200 HTTP 响应.""" body = json.dumps( @@ -666,3 +680,212 @@ async def test_missing_api_key_skips_retry(self): {}, ) assert resp.status_code == 401 + + # ── 529 Overloaded（并发过载）重试，行为与 429 一致 ────── + + @pytest.mark.asyncio + async def test_nonstream_529_retries_and_succeeds(self): + """非流式 529 两次后 200，重试成功.""" + vendor = _make_zhipu_vendor() + call_count = 0 + + async def mock_post(*args, **kwargs): + nonlocal call_count + call_count += 1 + if call_count <= 2: + return _make_529_response() + return _make_200_response() + + with ( + patch.object(vendor, "_get_client") as mock_client, + patch("asyncio.sleep", new_callable=AsyncMock), + ): + client = AsyncMock() + client.post = mock_post + mock_client.return_value = client + + resp = await vendor.send_message( + {"model": "claude-sonnet-4-20250514", "messages": []}, + {}, + ) + + assert resp.status_code == 200 + assert call_count == 3 + + @pytest.mark.asyncio + async def test_nonstream_529_exhausted_retries(self): + """非流式连续 5 次 529，耗尽重试后返回 529.""" + vendor = _make_zhipu_vendor() + call_count = 0 + + async def mock_post(*args, **kwargs): + nonlocal call_count + call_count += 1 + return _make_529_response() + + with ( + patch.object(vendor, "_get_client") as mock_client, + patch("asyncio.sleep", new_callable=AsyncMock), + ): + client = AsyncMock() + client.post = mock_post + mock_client.return_value = client + + resp = await vendor.send_message( + {"model": "claude-sonnet-4-20250514", "messages": []}, + {}, + ) + + assert resp.status_code == 529 + assert call_count == 5 + + @pytest.mark.asyncio + async def test_stream_529_retries_and_succeeds(self): + """流式 529 两次后成功.""" + call_count = 0 + + async def fake_stream(self, body, headers): + nonlocal call_count + call_count += 1 + if call_count <= 2: + resp = _make_529_response() + raise httpx.HTTPStatusError( + "529", + request=resp.request, + response=resp, + ) + yield b'data: {"type":"content_block_start"}\n\n' + yield b'data: {"type":"content_block_delta"}\n\n' + + vendor = _make_zhipu_vendor() + chunks = [] + with ( + patch.object(NativeAnthropicVendor, "send_message_stream", fake_stream), + patch("asyncio.sleep", new_callable=AsyncMock), + ): + async for chunk in vendor.send_message_stream( + {"model": "claude-sonnet-4-20250514", "messages": []}, + {}, + ): + chunks.append(chunk) + + assert len(chunks) == 2 + assert call_count == 3 + + @pytest.mark.asyncio + async def test_stream_529_exhausted_retries_raises(self): + """流式连续 529，耗尽重试后 raise.""" + call_count = 0 + + async def fake_stream(self, body, headers): + nonlocal call_count + call_count += 1 + resp = _make_529_response() + raise httpx.HTTPStatusError( + "529", + request=resp.request, + response=resp, + ) + yield # 使函数成为 async generator（不可达，仅影响类型） + + vendor = _make_zhipu_vendor() + with ( + patch.object(NativeAnthropicVendor, "send_message_stream", fake_stream), + patch("asyncio.sleep", new_callable=AsyncMock), + pytest.raises(httpx.HTTPStatusError) as exc_info, + ): + async for _ in vendor.send_message_stream( + {"model": "claude-sonnet-4-20250514", "messages": []}, + {}, + ): + pass + + assert exc_info.value.response.status_code == 529 + assert call_count == 5 + + @pytest.mark.asyncio + async def test_stream_529_respects_retry_after(self): + """流式 529 响应含 retry-after 时使用 server 建议延迟. + + 回归保障：修复前流式延迟计算把真实状态码 529 传给 + parse_rate_limit_headers（仅对 429/403 解析），导致 529 忽略 + retry-after 而回退指数退避（首次最多 1s）。修复后固定按 429 + 语义解析，529 与 429 一样尊重 server retry-after。 + """ + call_count = 0 + sleep_delays = [] + + async def fake_stream(self, body, headers): + nonlocal call_count + call_count += 1 + if call_count == 1: + resp = _make_529_response(headers={"retry-after": "2"}) + raise httpx.HTTPStatusError( + "529", + request=resp.request, + response=resp, + ) + yield b'data: {"type":"content_block_start"}\n\n' + + async def mock_sleep(delay): + sleep_delays.append(delay) + + vendor = _make_zhipu_vendor() + chunks = [] + with ( + patch.object(NativeAnthropicVendor, "send_message_stream", fake_stream), + patch("asyncio.sleep", side_effect=mock_sleep), + ): + async for chunk in vendor.send_message_stream( + {"model": "claude-sonnet-4-20250514", "messages": []}, + {}, + ): + chunks.append(chunk) + + assert len(chunks) == 1 + assert call_count == 2 + assert len(sleep_delays) == 1 + # retry-after=2 → 2 * 1.1 = 2.2s（>1s 指数退避首跳，证明用了 server 信号） + assert 2.0 <= sleep_delays[0] <= 2.2 + + @pytest.mark.asyncio + async def test_529_equal_jitter_delay_in_expected_band(self): + """非流式 529 无 retry-after 时，Equal Jitter 首跳落在 [0.5, 1.0]s。 + + 回归保障：Full Jitter 时首跳为 uniform(0, 1000ms)，可能接近 0ms + （用户报告的 418.8ms 落在 (0, 1000] 全区间，且整体序列非单调）。 + Equal Jitter 后区间收窄为 [500, 1000]ms，下界抬升至 500ms， + 呈现单调非递减的指数退避形态，429/529 同步受益（共用 calculate_delay）。 + """ + vendor = _make_zhipu_vendor() + sleep_delays = [] + + async def mock_sleep(delay): + sleep_delays.append(delay) + + call_count = 0 + + async def mock_post(*args, **kwargs): + nonlocal call_count + call_count += 1 + if call_count == 1: + return _make_529_response() # 无 retry-after + return _make_200_response() + + with ( + patch.object(vendor, "_get_client") as mock_client, + patch("asyncio.sleep", side_effect=mock_sleep), + ): + client = AsyncMock() + client.post = mock_post + mock_client.return_value = client + + resp = await vendor.send_message( + {"model": "claude-sonnet-4-20250514", "messages": []}, + {}, + ) + + assert resp.status_code == 200 + assert len(sleep_delays) == 1 + # Equal Jitter: attempt 0 → temp=1000ms → [500, 1000]ms → sleep([0.5, 1.0]) + assert 0.5 <= sleep_delays[0] <= 1.0 diff --git a/tests/test_zhipu_concurrency.py b/tests/test_zhipu_concurrency.py index 7566b24..d7be1ca 100644 --- a/tests/test_zhipu_concurrency.py +++ b/tests/test_zhipu_concurrency.py @@ -1,13 +1,16 @@ """Zhipu 每模型并发限制专项测试. -验证 ``ModelConcurrencyLimiter`` 与 ``ZhipuVendor`` 集成后的并发控制行为： +验证 ``ModelConcurrencyController`` 与 ``ZhipuVendor`` 集成后的并发控制行为： - 默认 ``concurrency.default=3`` 时同一模型最多 3 个并发 - 超出上限时按 FIFO 排队，槽位释放后才唤醒 - 不同模型彼此独立，互不阻塞 - - 异常路径下 Semaphore 仍能释放，避免泄漏 - - 流式请求与非流式请求共享同一信号量 + - 异常路径下槽位仍能释放，避免泄漏 + - 流式请求与非流式请求共享同一槽位 - 与 429 重试机制兼容（重试期间持续占用槽位） - - ``concurrency=None`` 时禁用限制（向后兼容） + +注意：并发限流由 BaseVendor._concurrency_controller 统一管控， +executor 层通过 ``vendor.track_in_flight(mapped_model)`` 上下文管理器 +包裹 send_message[_stream] 调用。本测试用同样的包裹模拟 executor 语义。 """ from __future__ import annotations @@ -25,7 +28,10 @@ ZhipuConfig, ) from coding.proxy.routing.model_mapper import ModelMapper -from coding.proxy.vendors.concurrency import ModelConcurrencyLimiter +from coding.proxy.vendors.concurrency import ( + ModelConcurrencyController, + ModelConcurrencyLimiter, +) from coding.proxy.vendors.native_anthropic import NativeAnthropicVendor from coding.proxy.vendors.zhipu import ZhipuVendor @@ -69,6 +75,29 @@ def _make_vendor( return ZhipuVendor(ZhipuConfig(**cfg_kwargs), _make_mapper()) +async def _send_with_tracking( + vendor: ZhipuVendor, + body: dict, + headers: dict, +): + """模拟 executor 行为：track_in_flight 包裹 send_message 调用.""" + mapped = vendor.map_model(body.get("model", "")) + async with vendor.track_in_flight(mapped): + return await vendor.send_message(body, headers) + + +async def _stream_with_tracking( + vendor: ZhipuVendor, + body: dict, + headers: dict, +): + """模拟 executor 行为：track_in_flight 包裹 send_message_stream 调用.""" + mapped = vendor.map_model(body.get("model", "")) + async with vendor.track_in_flight(mapped): + async for chunk in vendor.send_message_stream(body, headers): + yield chunk + + def _make_200_response() -> httpx.Response: body = json.dumps( { @@ -132,72 +161,128 @@ def test_zhipu_config_default_concurrency(self) -> None: assert cfg.concurrency.default == 3 -# ─── ModelConcurrencyLimiter 单元测试 ────────────────────── +# ─── ModelConcurrencyController 单元测试（limited 模式）───── + +class TestModelConcurrencyControllerLimited: + """ModelConcurrencyController limited 模式基础行为.""" -class TestModelConcurrencyLimiter: - """ModelConcurrencyLimiter 基础行为.""" + def test_alias_compatibility(self) -> None: + """ModelConcurrencyLimiter 别名指向 ModelConcurrencyController.""" + assert ModelConcurrencyLimiter is ModelConcurrencyController + + def test_mode_is_limited(self) -> None: + ctrl = ModelConcurrencyController(ZhipuConcurrencyConfig(default=2)) + assert ctrl.mode == "limited" @pytest.mark.asyncio - async def test_lazy_semaphore_creation(self) -> None: - limiter = ModelConcurrencyLimiter(ZhipuConcurrencyConfig(default=2)) - slot_a = limiter._get_or_create_slot("model-a") - slot_b = limiter._get_or_create_slot("model-b") + async def test_lazy_slot_creation(self) -> None: + ctrl = ModelConcurrencyController(ZhipuConcurrencyConfig(default=2)) + slot_a = ctrl._get_or_create_slot("model-a") + slot_b = ctrl._get_or_create_slot("model-b") # 不同模型独立 slot assert slot_a is not slot_b # 相同模型复用 slot - assert limiter._get_or_create_slot("model-a") is slot_a + assert ctrl._get_or_create_slot("model-a") is slot_a @pytest.mark.asyncio - async def test_acquire_blocks_when_full(self) -> None: - limiter = ModelConcurrencyLimiter(ZhipuConcurrencyConfig(default=2)) + async def test_track_blocks_when_full(self) -> None: + ctrl = ModelConcurrencyController(ZhipuConcurrencyConfig(default=2)) + + # 通过 acquire 占满 2 个槽位 + slot = ctrl._get_or_create_slot("glm-5.1") + await slot.acquire() + await slot.acquire() + assert slot.in_use == 2 - # 占满 2 个槽位 - sem1 = await limiter.acquire("glm-5.1") - sem2 = await limiter.acquire("glm-5.1") - assert sem1 is sem2 # 同一 semaphore + # 第 3 次 track 必须阻塞 + async def third(): + async with ctrl.track("glm-5.1"): + pass - # 第 3 次 acquire 必须阻塞 - task = asyncio.create_task(limiter.acquire("glm-5.1")) + task = asyncio.create_task(third()) await asyncio.sleep(0.05) assert not task.done(), "第三个请求应在排队等待" + # 排队时 pending 应递增 + assert slot.pending == 1 # 释放一个槽位后，等待者被唤醒 - sem1.release() + slot.release() await asyncio.sleep(0.05) assert task.done() - (await task).release() - sem2.release() + slot.release() @pytest.mark.asyncio async def test_per_model_independent(self) -> None: - limiter = ModelConcurrencyLimiter( + ctrl = ModelConcurrencyController( ZhipuConcurrencyConfig(default=1, models={"glm-5.1": 1}) ) - # 占满 glm-5.1 - sem_51 = await limiter.acquire("glm-5.1") - # glm-5v-turbo 仍可立即获取 - sem_5v = await asyncio.wait_for(limiter.acquire("glm-5v-turbo"), timeout=0.5) - assert sem_51 is not sem_5v - sem_51.release() - sem_5v.release() - def test_diagnostics_snapshot(self) -> None: - limiter = ModelConcurrencyLimiter(ZhipuConcurrencyConfig(default=3)) + async def hold(model: str, event: asyncio.Event) -> None: + async with ctrl.track(model): + await event.wait() + + gate_51 = asyncio.Event() + gate_5v = asyncio.Event() + t51 = asyncio.create_task(hold("glm-5.1", gate_51)) + await asyncio.sleep(0.02) + # glm-5v-turbo 仍可立即获取 + t5v = asyncio.create_task(hold("glm-5v-turbo", gate_5v)) + await asyncio.sleep(0.02) + # 两个任务都尚未结束（都在 await event） + assert not t51.done() + assert not t5v.done() + gate_51.set() + gate_5v.set() + await asyncio.gather(t51, t5v) + + def test_diagnostics_snapshot_limited(self) -> None: + ctrl = ModelConcurrencyController(ZhipuConcurrencyConfig(default=3)) # 触发 slot 创建 - limiter._get_or_create_slot("glm-5.1") - snap = limiter.get_diagnostics() + ctrl._get_or_create_slot("glm-5.1") + snap = ctrl.get_diagnostics() assert "glm-5.1" in snap + assert snap["glm-5.1"]["mode"] == "limited" assert snap["glm-5.1"]["limit"] == 3 assert snap["glm-5.1"]["available"] == 3 assert snap["glm-5.1"]["in_use"] == 0 + assert snap["glm-5.1"]["pending"] == 0 + assert "peak_pending_recent" in snap["glm-5.1"] + + @pytest.mark.asyncio + async def test_peak_pending_recent_tracking(self) -> None: + """触发排队时记录 peak，释放后仍可读到余晖.""" + ctrl = ModelConcurrencyController(ZhipuConcurrencyConfig(default=1)) + slot = ctrl._get_or_create_slot("glm-5.1") + + # 占满 + await slot.acquire() + # 触发两个排队 + t1 = asyncio.create_task(slot.acquire()) + t2 = asyncio.create_task(slot.acquire()) + await asyncio.sleep(0.05) + assert slot.pending == 2 + snap = ctrl.get_diagnostics() + assert snap["glm-5.1"]["peak_pending_recent"] == 2 + + # 释放并完成所有任务 + slot.release() + await t1 + slot.release() + await t2 + slot.release() + + # pending 已归零，但 peak_pending_recent 仍记得最近的峰值 + snap2 = ctrl.get_diagnostics() + assert snap2["glm-5.1"]["pending"] == 0 + assert snap2["glm-5.1"]["peak_pending_recent"] == 2 # ─── ZhipuVendor 集成测试：非流式 ──────────────────────────── class TestZhipuVendorNonStreamConcurrency: - """非流式 send_message 的并发限制行为.""" + """非流式 send_message 的并发限制行为（通过 track_in_flight 包裹）.""" @pytest.mark.asyncio async def test_limits_parallel_requests(self) -> None: @@ -223,7 +308,8 @@ async def mock_post(*_, **__) -> httpx.Response: tasks = [ asyncio.create_task( - vendor.send_message( + _send_with_tracking( + vendor, {"model": "claude-opus-4-6", "messages": []}, {}, ) @@ -265,15 +351,17 @@ async def mock_post(*_args, **kwargs) -> httpx.Response: mock_client.return_value = client # claude-opus → glm-5.1, claude-sonnet → glm-5v-turbo， - # 分属两个独立信号量，应同时执行 + # 分属两个独立槽位，应同时执行 task_opus = asyncio.create_task( - vendor.send_message( + _send_with_tracking( + vendor, {"model": "claude-opus-4-6", "messages": []}, {}, ) ) task_sonnet = asyncio.create_task( - vendor.send_message( + _send_with_tracking( + vendor, {"model": "claude-sonnet-4-6", "messages": []}, {}, ) @@ -289,8 +377,8 @@ async def mock_post(*_args, **kwargs) -> httpx.Response: await asyncio.gather(task_opus, task_sonnet) @pytest.mark.asyncio - async def test_semaphore_released_on_exception(self) -> None: - """上游抛异常时 Semaphore 仍应释放，后续请求不阻塞.""" + async def test_slot_released_on_exception(self) -> None: + """上游抛异常时槽位仍应释放，后续请求不阻塞.""" vendor = _make_vendor(ZhipuConcurrencyConfig(default=1)) call_count = 0 @@ -307,14 +395,16 @@ async def mock_post(*_, **__) -> httpx.Response: mock_client.return_value = client with pytest.raises(RuntimeError): - await vendor.send_message( + await _send_with_tracking( + vendor, {"model": "claude-opus-4-6", "messages": []}, {}, ) # 槽位应已释放，第二次请求可正常完成 resp = await asyncio.wait_for( - vendor.send_message( + _send_with_tracking( + vendor, {"model": "claude-opus-4-6", "messages": []}, {}, ), @@ -343,7 +433,8 @@ async def mock_post(*_, **__) -> httpx.Response: client.post = mock_post mock_client.return_value = client - resp = await vendor.send_message( + resp = await _send_with_tracking( + vendor, {"model": "claude-opus-4-6", "messages": []}, {}, ) @@ -351,13 +442,13 @@ async def mock_post(*_, **__) -> httpx.Response: assert call_count == 3 # 两次 429 + 一次成功，且共用同一槽位 @pytest.mark.asyncio - async def test_no_concurrency_when_config_is_none(self) -> None: - """concurrency=None 时禁用并发限制，行为与旧版完全一致.""" - # 强制构造一个 concurrency=None 的 ZhipuConfig（绕过默认工厂） - cfg = ZhipuConfig(api_key="key") - cfg = cfg.model_copy(update={"concurrency": None}) - vendor = ZhipuVendor(cfg, _make_mapper()) - assert vendor._concurrency_limiter is None + async def test_monitor_mode_no_throttling(self) -> None: + """BaseVendor 默认 monitor 模式：高并发不限流，仅计数.""" + # 显式构造一个 monitor 模式的 zhipu vendor（用于验证 BaseVendor 默认行为等价） + vendor = _make_vendor(ZhipuConcurrencyConfig(default=20)) + from coding.proxy.vendors.concurrency import ModelConcurrencyController as MCC + + vendor._concurrency_controller = MCC(None) gate = asyncio.Event() active = 0 @@ -378,7 +469,8 @@ async def mock_post(*_, **__) -> httpx.Response: tasks = [ asyncio.create_task( - vendor.send_message( + _send_with_tracking( + vendor, {"model": "claude-opus-4-6", "messages": []}, {}, ) @@ -390,7 +482,7 @@ async def mock_post(*_, **__) -> httpx.Response: break await asyncio.sleep(0.01) - assert peak == 5, "无并发限制时应全部并行" + assert peak == 5, "monitor 模式应允许全部并行" gate.set() await asyncio.gather(*tasks) @@ -399,7 +491,7 @@ async def mock_post(*_, **__) -> httpx.Response: class TestZhipuVendorStreamConcurrency: - """流式 send_message_stream 的并发限制行为.""" + """流式 send_message_stream 的并发限制行为（通过 track_in_flight 包裹）.""" @pytest.mark.asyncio async def test_stream_limits_parallel_requests(self) -> None: @@ -421,8 +513,8 @@ async def fake_stream(self, _body, _headers): # noqa: ARG001 async def consume(model: str) -> int: chunks: list[bytes] = [] - async for chunk in vendor.send_message_stream( - {"model": model, "messages": []}, {} + async for chunk in _stream_with_tracking( + vendor, {"model": model, "messages": []}, {} ): chunks.append(chunk) return len(chunks) @@ -453,15 +545,14 @@ async def fake_stream(self, _body, _headers): # noqa: ARG001 # 连续两次流式请求都能完成（说明槽位被释放） for _ in range(2): chunks = [] - async for chunk in vendor.send_message_stream( - {"model": "claude-opus-4-6", "messages": []}, {} + async for chunk in _stream_with_tracking( + vendor, {"model": "claude-opus-4-6", "messages": []}, {} ): chunks.append(chunk) assert len(chunks) == 2 # 确认 slot 当前完全可用 - assert vendor._concurrency_limiter is not None - slot = vendor._concurrency_limiter._get_or_create_slot("glm-5.1") + slot = vendor._concurrency_controller._get_or_create_slot("glm-5.1") assert slot.available == 1 @pytest.mark.asyncio @@ -485,22 +576,22 @@ async def fake_stream(self, _body, _headers): # noqa: ARG001 with patch.object(NativeAnthropicVendor, "send_message_stream", fake_stream): with pytest.raises(httpx.HTTPStatusError): - async for _ in vendor.send_message_stream( - {"model": "claude-opus-4-6", "messages": []}, {} + async for _ in _stream_with_tracking( + vendor, {"model": "claude-opus-4-6", "messages": []}, {} ): pass # 槽位应已释放，第二次请求可正常推进 chunks = [] - async for chunk in vendor.send_message_stream( - {"model": "claude-opus-4-6", "messages": []}, {} + async for chunk in _stream_with_tracking( + vendor, {"model": "claude-opus-4-6", "messages": []}, {} ): chunks.append(chunk) assert chunks == [b'data: {"type":"message_start"}\n\n'] @pytest.mark.asyncio - async def test_stream_and_nonstream_share_semaphore(self) -> None: - """流式与非流式请求共用同一信号量（按映射后模型分组）.""" + async def test_stream_and_nonstream_share_slot(self) -> None: + """流式与非流式请求共用同一槽位（按映射后模型分组）.""" vendor = _make_vendor(ZhipuConcurrencyConfig(default=1)) gate = asyncio.Event() active = 0 @@ -530,8 +621,8 @@ async def mock_post(*_, **__) -> httpx.Response: # 启动流式请求并等待它占用槽位 async def consume_stream() -> None: - async for _ in vendor.send_message_stream( - {"model": "claude-opus-4-6", "messages": []}, {} + async for _ in _stream_with_tracking( + vendor, {"model": "claude-opus-4-6", "messages": []}, {} ): pass @@ -542,9 +633,10 @@ async def consume_stream() -> None: await asyncio.sleep(0.01) assert active == 1 - # 非流式请求应被同一信号量阻塞 + # 非流式请求应被同一槽位阻塞 nonstream_task = asyncio.create_task( - vendor.send_message( + _send_with_tracking( + vendor, {"model": "claude-opus-4-6", "messages": []}, {}, ) diff --git a/uv.lock b/uv.lock index d04ad46..b9197bf 100644 --- a/uv.lock +++ b/uv.lock @@ -74,7 +74,7 @@ wheels = [ [[package]] name = "coding-proxy" -version = "0.5.0" +version = "0.5.2a5" source = { editable = "." } dependencies = [ { name = "aiosqlite" },