From c80b49ab118e7acc3d110817afa0f3cbda8499c5 Mon Sep 17 00:00:00 2001 From: Eli Ma Date: Wed, 7 Oct 2026 22:31:51 +0800 Subject: [PATCH 1/5] HP-22 enable read-only view clone and fetch packs Signed-off-by: Eli Ma --- Cargo.lock | 2 +- Cargo.toml | 2 +- docs/architecture.md | 2 + docs/architecture.zh.md | 2 + docs/deploy-trunk.md | 2 +- docs/plan/plan-20261002.md | 45 +- docs/refactoring/protocol.md | 7 +- docs/user-guide.md | 10 + docs/user-guide.zh.md | 10 + src/ceres/pack/view_repo.rs | 880 +++++++++++++++++- .../storage/view_projection_storage.rs | 92 ++ tests/fixtures/views/josh_proxy/README.md | 11 + .../views/josh_proxy/prefix.fetch.log | 3 + .../views/josh_proxy/prefix.fetch.tree | 3 + tests/fixtures/views/josh_proxy/prefix.log | 2 + tests/fixtures/views/josh_proxy/prefix.tree | 2 + .../views/josh_proxy/subsubtree.fetch.log | 2 + .../views/josh_proxy/subsubtree.fetch.tree | 2 + .../fixtures/views/josh_proxy/subsubtree.log | 1 + .../fixtures/views/josh_proxy/subsubtree.tree | 1 + .../views/josh_proxy/subtree.fetch.log | 2 + .../views/josh_proxy/subtree.fetch.tree | 2 + tests/fixtures/views/josh_proxy/subtree.log | 1 + tests/fixtures/views/josh_proxy/subtree.tree | 1 + tests/integration_git_cli.rs | 512 ++++++++++ tests/integration_git_ssh.rs | 259 ++++++ 26 files changed, 1805 insertions(+), 53 deletions(-) create mode 100644 tests/fixtures/views/josh_proxy/README.md create mode 100644 tests/fixtures/views/josh_proxy/prefix.fetch.log create mode 100644 tests/fixtures/views/josh_proxy/prefix.fetch.tree create mode 100644 tests/fixtures/views/josh_proxy/prefix.log create mode 100644 tests/fixtures/views/josh_proxy/prefix.tree create mode 100644 tests/fixtures/views/josh_proxy/subsubtree.fetch.log create mode 100644 tests/fixtures/views/josh_proxy/subsubtree.fetch.tree create mode 100644 tests/fixtures/views/josh_proxy/subsubtree.log create mode 100644 tests/fixtures/views/josh_proxy/subsubtree.tree create mode 100644 tests/fixtures/views/josh_proxy/subtree.fetch.log create mode 100644 tests/fixtures/views/josh_proxy/subtree.fetch.tree create mode 100644 tests/fixtures/views/josh_proxy/subtree.log create mode 100644 tests/fixtures/views/josh_proxy/subtree.tree diff --git a/Cargo.lock b/Cargo.lock index fb413935..4853c9bd 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4172,7 +4172,7 @@ checksum = "7ebb8d8732c6a6df3d8f032a82911cfc747e00efb95cc46e8d0acd5b5b88570c" [[package]] name = "mega2" -version = "0.42.20" +version = "0.42.21" dependencies = [ "anyhow", "async-recursion", diff --git a/Cargo.toml b/Cargo.toml index 3780a22e..85e176d0 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "mega2" -version = "0.42.20" +version = "0.42.21" edition = "2024" license = "MIT OR Apache-2.0" description = "Monorepo engine and Git hosting server for AI agents: trunk-based storage, Git Smart HTTP, Git LFS, OCI registry, and Agent Session Capture" diff --git a/docs/architecture.md b/docs/architecture.md index 61447bf7..d97d1272 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -115,6 +115,8 @@ See the [Configuration Guide](./configuration.md) for SecretRef settings and val | SSH | `service ssh` (dedicated port) | read-only (upload-pack) in storage-only; `git.ssh_receive_pack=false` is mandatory — omitting it refuses startup | | Git LFS | `/info/lfs` + `/api/v1/lfs` | mounted in both modes; write authorization follows `push_auth` | | Product API | `/api/v1/*` (status, file/blob, file/tree, preview reads + create-entry/delete-entry/move-entry/edit/save writes + tags) | trunk / storage-only mode | +| Git view URLs (read-only) | `/.filter/.git`, `/.view/.git`, `/.view/@.git` (HTTP and SSH upload-pack) | `[views].enabled` defaults off and requires restart; receive-pack is rejected and LFS under view prefixes returns 404 | +| Views API | `/api/v1/views` (`POST ?wait=true` registration and `GET ?name=&version=`), `GET /api/v1/views/{filter_id}`, `GET /api/v1/views/metrics`; see [views operations](./deploy-trunk.md) | `[views].enabled` defaults off and requires restart; mounted only in trunk / storage-only, returns 404 when disabled | | OCI registry | `/v2/` | `[oci].enabled` and storage-only; fail-closed (not registered) under review | | Agent Capture | `/api/v1/agent-capture` | `[agent_capture].enabled` and storage-only | | MST/2 snapshot surface | `/api/v2` | nest is static; handlers fail closed unless `[mst2].enabled`; toggling needs a restart | diff --git a/docs/architecture.zh.md b/docs/architecture.zh.md index 71358f91..ff689f08 100644 --- a/docs/architecture.zh.md +++ b/docs/architecture.zh.md @@ -114,6 +114,8 @@ HTTP router 在 `server::http_server::app`（`src/server/http_server.rs:681`） | SSH | `service ssh`（独立端口） | storage-only 下只读（upload-pack）；`git.ssh_receive_pack=false` 为强制项，省略会拒绝启动 | | Git LFS | `/info/lfs` + `/api/v1/lfs` | 两种形态均挂载；写授权随 `push_auth` | | 产品 API | `/api/v1/*`（status、file/blob、file/tree、preview 读 + create-entry/delete-entry/move-entry/edit/save 写 + tags） | trunk / storage-only 形态 | +| Git 视图 URL（只读） | `/.filter/.git`、`/.view/.git`、`/.view/@.git`（HTTP 与 SSH upload-pack） | `[views].enabled` 缺省关闭，重启生效；receive-pack 拒绝，视图前缀 LFS 返回 404 | +| 视图 API | `/api/v1/views`（`POST ?wait=true` 注册、`GET ?name=&version=`）、`GET /api/v1/views/{filter_id}`、`GET /api/v1/views/metrics`；请求见[视图运维](./deploy-trunk.md) | `[views].enabled` 缺省关闭，重启生效；只在 trunk / storage-only 挂载，关闭时返回 404 | | OCI registry | `/v2/` | `[oci].enabled` 且 storage-only；review 形态 fail-closed 不注册 | | Agent Capture | `/api/v1/agent-capture` | `[agent_capture].enabled` 且 storage-only | | MST/2 快照面 | `/api/v2` | nest 静态注册，handler 在 `[mst2].enabled` 为 false 时 fail-closed；切换需重启 | diff --git a/docs/deploy-trunk.md b/docs/deploy-trunk.md index 3e6e6d1f..07d60709 100644 --- a/docs/deploy-trunk.md +++ b/docs/deploy-trunk.md @@ -487,7 +487,7 @@ DELETE FROM mega_view_root_chain_scan; `recycle-filter` 与 `rebuild-all` 都有显式事务。`COMMIT` 之前的语句失败时，psql 会停止并断开连接，事务不会提交。recycle-filter 只回收目标过滤器的派生提交映射与对象引用，保留对象以供后续清扫；rebuild-all 清掉全部派生状态。两者把过滤器置为回收态：`projected_seq = 0`，且 `ready_seq`、`warming_since` 为 NULL。worker 只为活跃视图（`ready_seq` 或 `warming_since` 非空）执行 catch_up，因此回收态视图会等待重新预热。clear-root-chain-scan 只清暂存表，不改视图状态；下一次根链扩展会按当前的 `main@/` 重新判定。 -三段 SQL 都不用 `TRUNCATE`，不删除 `mega_view_filter`、`mega_view` 或 `mega_view_register_log`，也不改 `last_access_at` 与过滤器定义列。rebuild-all 是全表 `DELETE`，停服窗口随派生行数增长。本节只记录停服运维与恢复语义；视图 Git URL 的对象读取尚未开放。 +三段 SQL 都不用 `TRUNCATE`，不删除 `mega_view_filter`、`mega_view` 或 `mega_view_register_log`，也不改 `last_access_at` 与过滤器定义列。rebuild-all 是全表 `DELETE`，停服窗口随派生行数增长。本节记录停服运维与恢复语义；视图 Git URL 已支持只读 clone 与 fetch，用法见[用户指南](./user-guide.zh.md#27-视图-url只读)。 投影停止时，首个失败点会以 ERROR 事件给出。除 message 外，停止告警字段为 `metric=view_projection_stops_total`、`filter_id`、`s`、`commit_id`、`reason`；当 reason 是 `row_absent` 或 `unparsable` 时还带 `tree_id`。例如：`metric=view_projection_stops_total s=7 reason=row_absent`。按 reason 处理： diff --git a/docs/plan/plan-20261002.md b/docs/plan/plan-20261002.md index 2cae2223..b32ee375 100644 --- a/docs/plan/plan-20261002.md +++ b/docs/plan/plan-20261002.md @@ -2,7 +2,7 @@ **模板版本：** v2.1；依据 [`plan-template.md`](plan-template.md)。源码核对日期 2026-10-02；核对时 tip 为 `c9ae6df`（`v0.41.34`）。成稿时另一会话已发布 `fb5ac71`（`v0.41.35: BB-43 ART input validation case`），只改了 `Cargo.toml`、`Cargo.lock`、`docs/deploy-trunk.md`、`docs/plan/plan-20261001.md`、`scripts/artifacts_smoke_storage_only.sh`，不影响本文的源码锚点。工作区仍有该会话的未提交改动，本计划不触碰；每张卡开工时按 ER-01、ER-12 重新读取权威版本。 -**计划状态：** **执行中**。主链已完成 HP-32 与 HP-34；HP-30 承载发布为 `v0.42.7`；HP-13 承载发布为 `v0.42.8`；HP-31 承载发布为 `v0.42.9`；HP-14 由 `v0.42.10` 承载；HP-16 由 `v0.42.11` 承载；HP-17 由 `v0.42.12` 承载；HP-34 由 `v0.42.13` 承载；HP-32 由 `v0.42.14` 承载；HP-15 由 `v0.42.15` 承载；HP-33 由 `v0.42.16` 承载；HP-18 由 `v0.42.17` 承载；HP-19 由 `v0.42.18` 承载；HP-20 由 `v0.42.19` 承载；HP-21 由 `v0.42.20` 承载。按剩余依赖顺序从 HP-22 继续。Docker Build / Actions 按发布者指令不等待、不查询。 +**计划状态：** **执行中**。主链已完成 HP-32 与 HP-34；HP-30 承载发布为 `v0.42.7`；HP-13 承载发布为 `v0.42.8`；HP-31 承载发布为 `v0.42.9`；HP-14 由 `v0.42.10` 承载；HP-16 由 `v0.42.11` 承载；HP-17 由 `v0.42.12` 承载；HP-34 由 `v0.42.13` 承载；HP-32 由 `v0.42.14` 承载；HP-15 由 `v0.42.15` 承载；HP-33 由 `v0.42.16` 承载；HP-18 由 `v0.42.17` 承载；HP-19 由 `v0.42.18` 承载；HP-20 由 `v0.42.19` 承载；HP-21 由 `v0.42.20` 承载；HP-22 由 `v0.42.21` 承载。按剩余依赖顺序从 HP-23 继续。Docker Build / Actions 按发布者指令不等待、不查询。 **设计契约：** [`docs/refactoring/history-projection.md`](../refactoring/history-projection.md)（下称「设计」，v0.2，经 Codex 五轮审计，见设计附录 F）。本文用「设计 §x.y」引用其条款；设计节号冻结（ADR-HP-01）。 @@ -184,6 +184,7 @@ | 2026-10-07 | HP-20 实施、验收与三轮 Claude CLI 复审 | 视图 URL 经认证后解析并分派 ViewRepo，覆盖 HTTP/SSH v0/v2 的未知、未就绪、宣告后失效、定义损坏和成功宣告。Claude R1/R2 的测试缺口已修订，R3 `VERDICT: PASS`，Codex 复核通过；8 条精确验证、ER-04 A 组只读 smoke（8/8）、nightly 格式、全目标全特性 Clippy、完整 `cargo test --all` 与两个构建门通过；专用链接参数下构建 0 警告，默认 macOS 链接器保留既有 `__eh_frame` 提示。Docker Build / Actions 按发布者指令不等待、不查询。 | 无拆卡 | HP-20、`protocol.md`、Review log、ER-04 C/D | | 2026-10-07 | HP-21 设计阶段归属校准 | 设计 §6.1 第 3 层的读者触发重新预热未标（P1），而 §3.3、§7.5 与 §7.4 有指向 P1 的描述；本计划按 §6.1 与 ADR-HP-08 在 P0 实施。设计「事实校准」第 15 条登记该取舍，正文算法与错误契约不变。 | 无拆卡 | HP-21、设计「事实校准」、ADR-HP-01 | | 2026-10-07 | HP-21 实施、验收与三轮 Claude CLI 复审 | 回收态读请求经准入触发预热与 worker 信号，本次仍按原状态返回暂不可用；四种准入结果、拒绝原因、取消锁等待、热加载、重启恢复和入口闸门均有用例。Claude R1/R2 的测试证据缺口已关闭，R3 `VERDICT: PASS`；Codex 复核通过。8 条精确验证、ER-04 A 组只读 smoke 8/8、nightly 格式、全目标全特性 Clippy、完整 `cargo test --all` 和两个构建门均通过，专用链接参数下构建 0 警告。Docker Build / Actions 按发布者指令不等待、不查询。 | 无拆卡 | HP-21、`deploy-trunk.md`、设计「事实校准」、Review log、ER-04 C/D | +| 2026-10-07 | HP-22 打包、验收与三轮 Claude CLI 复审 | ViewRepo 提供只读视图 URL 的 HTTP/SSH v0/v2 clone 与 fetch；Josh 固定期望与本项目增量扩展按 ER-03 分开登记。Claude R1 的闭包与夹具证据缺口已修订，R2/R3 `VERDICT: PASS`；Codex 复核通过。8 条精确验证、ER-04 A 组四项本地只读 smoke、nightly 格式、全目标全特性 Clippy、受控并发的完整 `cargo test --all` 及两个构建门均通过，专用链接参数下构建 0 警告。首次完整测试的一个现有进程级用例遇到回环连接重置，单独复跑与受控并发的第二次完整运行通过。Docker Build / Actions 按发布者指令不等待、不查询。 | 无拆卡 | HP-22、`protocol.md`、双语用户指南与架构、Review log、ER-04 C/D | ## 已决议设计决策 @@ -4942,7 +4943,9 @@ ER-04 强制门（不计入 VER）：本卡改 `src/contract/git_protocol/mod.rs **Task type:** `implementation` -**Lifecycle / Acceptance:** `pending` / 空 +**Lifecycle / Acceptance:** `done` / `complete`（Claude HP22-R3 `VERDICT: PASS`；Codex HP22-R4 PASS；8/8 验收、8/8 精确 Verification、ER-04 A 组四项本地只读 smoke、nightly 格式、Clippy、完整测试与两个零警告构建门通过；承载发布 `v0.42.21`；Docker Build / Actions 按发布者指令不等待、不查询） + +**ER-03 修订（2026-10-07）：**固定上游 Josh revision `e6dfb95e` 的 `clone_subtree.t`、`clone_subsubtree.t`、`clone_prefix.t` 只有 clone 期望，没有后续 fetch 段。因此 `tests/fixtures/views/josh_proxy/*.log` 与 `*.tree` 逐字节移植 clone 期望；`*.fetch.*` 是本项目在同一过滤器上追加 `add file3` 的增量扩展，不声称来自上游。HP-22 仍以三个过滤器的 HTTP/SSH v0/v2 fetch 验证增量包。另将 `docs/deploy-trunk.md` 纳入本卡文档写集，只校正 §9 中“视图 Git URL 的对象读取尚未开放”的过时句，不改运维 SQL 与恢复语义。 **Description:** 实现 `ViewRepo` 的 `full_pack` 与 `incremental_pack`（设计 §6.3「打包」，以及方法表 `full_pack`、`incremental_pack` 两行），替换 HP-19 留下的 `view pack not implemented` 错误。blob 内容与元数据经 HP-19 按方法表 `get_blobs_by_hashes`、`get_blob_metadata_by_hashes` 两行实现的同名方法读取，本卡只调用这两个方法，不改它们。唯一的行为轴是「视图 URL 的 clone 与 fetch」：本卡发布后，`[views].enabled = true` 的部署上视图 URL 首次端到端可用（ADR-HP-04），双语用户文档随本卡写明视图 URL 的用法（GC-HP-08）。范围： - 计数步骤：用一条 SQL 读出 want 与 have 在本视图 commit_map 中的 `seq_from` 与 `view_tree`（数组参数绑定）。want 的最大 seq 为终点 s_w；have 中位于本视图链上的最大 seq 为起点 s_h，have 为空或全不在链上时 s_h = 0。diff 的起点 tree：s_h > 0 时取这条 SQL 返回的 seq_from = s_h 那一行的 `view_tree`（即 c_{s_h} 的 tree，c_{s_h} 本身不入包），s_h = 0 时为空树。再用一条 SQL 按 seq 区间 (s_h, s_w] 读出全部视图提交及其字节（`mega_view_commit_map` 连 `mega_view_object` 中 kind = 1 的行），NULL 段不产出提交。 @@ -5001,27 +5004,27 @@ ER-04 强制门（不计入 VER）：本卡改 `src/contract/git_protocol/mod.rs **Acceptance criteria:** -- [ ] have 为空时，`full_pack` 产出的 pack 中的对象（id 由对象字节重算）与期望闭包一一对应。期望闭包为 seq 不超过 want 最大 seq 的全部视图提交，加上它们 tree 闭包中的 tree 与 blob，不含 `160000` 条目指向的对象。 -- [ ] have 非空时，`incremental_pack` 产出的 pack 中的对象，与下述集合一一对应：起点为 have 中位于本视图链上的最大 seq（没有时为 0），按设计 §6.3「打包」的 have 侧排除与 gitlink 规则（含方法表 `incremental_pack` 行「链外的 have 忽略」）得出，每个对象只出现一次。 -- [ ] 闭包含 `160000` gitlink 条目的视图，git-cli 经 HTTP 以 v0 与 v2 clone 都成功。 -- [ ] 在 tree 深度与每层分块数相同的输入上，一次打包执行的 SQL 语句数与视图提交数无关。 -- [ ] Josh `clone_subtree.t`、`clone_subsubtree.t`、`clone_prefix.t` 中 clone 与 fetch 部分的 `git log --graph` 与 tree 输出，经 HTTP 与 SSH、v0 与 v2，都与 `tests/fixtures/views/josh_proxy/` 中的期望逐字节一致。 -- [ ] 已就绪但投影为空（tip 为 NULL 段）的视图，经 HTTP 与 SSH 读取的结果（含 v2 `ls-refs` 应答）与设计 §7.5 P0 验收 7 中「已就绪但投影为空的视图」一项一致。 -- [ ] 过滤后 tree 改变的 trunk 推送（N = 1 与 N > 1 各一例）之后，已有克隆以默认 v2 执行 `git fetch`，`origin/main` 恰好前进一个视图提交。 -- [ ] 经 HTTP 以 v0 或 v2 带 `--depth 1` clone 视图、以 v2 带 `--filter=blob:none` clone 视图，git 都以 `RPC failed; HTTP 400` 失败。 +- [x] have 为空时，`full_pack` 产出的 pack 中的对象（id 由对象字节重算）与期望闭包一一对应。期望闭包为 seq 不超过 want 最大 seq 的全部视图提交，加上它们 tree 闭包中的 tree 与 blob，不含 `160000` 条目指向的对象。 +- [x] have 非空时，`incremental_pack` 产出的 pack 中的对象，与下述集合一一对应：起点为 have 中位于本视图链上的最大 seq（没有时为 0），按设计 §6.3「打包」的 have 侧排除与 gitlink 规则（含方法表 `incremental_pack` 行「链外的 have 忽略」）得出，每个对象只出现一次。 +- [x] 闭包含 `160000` gitlink 条目的视图，git-cli 经 HTTP 以 v0 与 v2 clone 都成功。 +- [x] 在 tree 深度与每层分块数相同的输入上，一次打包执行的 SQL 语句数与视图提交数无关。 +- [x] Josh `clone_subtree.t`、`clone_subsubtree.t`、`clone_prefix.t` 中 clone 的 `git log --graph` 与 tree 输出，以及 ER-03 登记的 fetch 扩展输出，经 HTTP 与 SSH、v0 与 v2，都与 `tests/fixtures/views/josh_proxy/` 中的期望逐字节一致。 +- [x] 已就绪但投影为空（tip 为 NULL 段）的视图，经 HTTP 与 SSH 读取的结果（含 v2 `ls-refs` 应答）与设计 §7.5 P0 验收 7 中「已就绪但投影为空的视图」一项一致。 +- [x] 过滤后 tree 改变的 trunk 推送（N = 1 与 N > 1 各一例）之后，已有克隆以默认 v2 执行 `git fetch`，`origin/main` 恰好前进一个视图提交。 +- [x] 经 HTTP 以 v0 或 v2 带 `--depth 1` clone 视图、以 v2 带 `--filter=blob:none` clone 视图，git 都以 `RPC failed; HTTP 400` 失败。 **Verification:** 前置（环境准备，不单独计门）：`source .env.test && docker compose -p mega2-it -f docker/docker-compose.test.yml --profile git up -d --wait`。Postgres、Redis 缺失时下列用例会 panic，判为失败，不是跳过。lib 用例的根历史夹具复用 `crate::jupiter::storage::view_test_fixtures`，过滤器行与状态列在测试中直接写入（ADR-HP-06），再以 `ViewProjectionService` 的 `catch_up` 投影到就绪；blob 经 `storage.git_service.save_object_from_raw` 写入；除第 3 行外，ViewRepo 建在 `test_storage_with_config` 返回的存储上。进程级用例以 `MEGA_VIEWS__ENABLED=true` 启动服务，经 trunk 推送构造根历史，以 `POST /api/v1/views?wait=true`（请求体 `{filter_spec, name}`）注册视图并等到应答 `ready = true`；注册凭据覆盖视图的源路径，`push_auth = none` 时另设 `MEGA_VIEWS__ALLOW_ANONYMOUS_REGISTER=true`。「投影追平」规则（输入确认，不是断言，适用于第 4–7 行的全部进程级用例）：每次 trunk 推送（宿主 git；SSH 用例中经 HTTP 腿）返回之后、下一次对视图执行 clone 或 fetch 之前，宿主用不带凭据的 curl 每隔 250 毫秒请求一次将要访问的视图的 `GET /api/v1/views/{filter_id}`（filter_id 取注册应答中的值），URL 取 `git_cli::mega2_host_http_url(, "/api/v1/views/")`，SSH 用例同样取该服务的 HTTP 端口；直到状态码为 200 且响应 JSON 的 `data.lag_commits` 等于 0 为止，累计 60 秒仍未满足即用例失败。推送早于注册时（各用例的种子推送），在注册应答 `ready = true` 之后、首次 clone 之前轮询；推送晚于注册时，在推送命令返回之后、下一次 fetch 之前轮询。理由：推送命令返回时 `main@/` 已前进，C 段信号随即唤醒 worker，worker 在整批事务提交之前持有 L_V(F)；此时到达的 advertise 试锁失败，宣告已投影到的旧 tip，clone 或 fetch 取不到新提交（Current evidence「trunk 推送在客户端收到应答之前已提交根」「投影是异步的」两行）。`lag_commits` 为 0 即 `projected_seq ≥ seq(main@/)`，此后视图 tip 就是当前 `main@/` 的投影。轮询超时属于前置条件不成立；各行的断言都在轮询结束之后执行。本卡的进程级新用例都是顶层函数，完整路径即函数名；每个用例开头按 `tests/integration_git_ssh.rs:545-549` 的写法断言 `MEGA2_IT_SKIP_GIT_CLI` 不为 1，再调用 `git_cli::require_git_cli_runner()`，不使用 `tests/integration_git_cli.rs:1871-1874` 那种打印 SKIP 后返回的分支，跳过不算通过。执行位置：视图 URL 上的 git 步骤（clone、fetch，以及在所得克隆中执行的 `git log`、`git ls-tree`、`git rev-list`、`git for-each-ref`）一律在 compose 的 git-cli 容器 runner 中运行，与设计 §7.4 一致（`docs/refactoring/history-projection.md:1264`、`:1374`）：HTTP 经 `git_cli::git_cli_no_auth`，SSH 经 `git_cli::git_cli_ssh`，需要额外环境变量的那一步用第 5 行所述的用例内 helper。本卡的 SSH 用例不采用宿主回环写法。容器 runner 的 git 版本固定（`PINNED_GIT_CLI_VERSION`，`tests/common/git_cli.rs:29`、`:416-450`），逐字节比较只在它上面成立；验收时不设 `MEGA2_IT_ALLOW_HOST_GIT`。宿主 git 只用于构造根历史的推送，以及 gitlink 夹具的 `update-index`、`write-tree`、`commit-tree`，沿用既有 trunk 用例的 `trunk_host_git`（`tests/integration_git_cli.rs:1669-1697`）与 `git_host_ssh`（`tests/integration_git_ssh.rs:480-501`），不参与比对。HTTP 原始应答（第 6 行的 `ls-refs` POST）由宿主 curl 取得，与设计 §7.4「错误契约」导语「HTTP 在 cargo 黑盒中直接发请求」一致。宿主侧发出的 HTTP 请求（注册请求、「投影追平」轮询与第 6 行的 curl）一律发往 `git_cli::mega2_host_http_url(port, "<路径>")`（回环地址，`tests/common/git_cli.rs:314-317`）；容器中的 git 用 `git_cli::mega2_http_url`，它在容器 runner 下的主机 `host.docker.internal` 不一定能在宿主解析。ER-04 A 组（不计入 VER）：Git 协议行，按本计划 ER-04 A 组表中该卡那一行执行（`scripts/git_protocol_smoke_storage_only.sh`，compose 项目 `mega2-trunk`）。 -- [ ] `source .env.test && cargo test -p mega2 --lib 'ceres::pack::view_repo::tests::full_pack_closure_one_to_one' -- --exact`（new；AC 1。夹具：线性根历史 8 个根提交；Subdir、Prefix、Exclude、不相交 Compose 四类视图各一个，另有一个闭包含 `160000` 条目的 Subdir 视图，该条目的目标 id 不在任何表与对象存储中。每个视图以 tip 与一个较早的视图提交为 want 各调用一次 `full_pack`，收齐流后用 `Pack::new_with_hash_kind` 解码，写法同 `src/ceres/pack/monorepo.rs:3779-3809`。断言：解码成功；条目数等于 pack 头的对象数；按条目字节重算的 id 无重复，集合等于测试按夹具在内存中独立算出的期望闭包；Prefix 视图的 pack 含 `mega_view_object` 中的脊柱 tree；gitlink 视图的 pack 不含该目标 id） -- [ ] `source .env.test && cargo test -p mega2 --lib 'ceres::pack::view_repo::tests::incremental_pack_have_exclusion' -- --exact`（new；AC 2，逐项断言设计 §6.3「打包」的 have 侧排除规则与 gitlink 规则、方法表 `incremental_pack` 行。夹具：线性根历史 12 个根提交，视图 `:/a` 下有每个提交都不改动的子树 `a/big/`（1 500 个 blob，跨越分块边界），其余提交改动 `a/x/` 下的文件，其中一个提交把文件从 `a/x/m` 移到 `a/y/m`，一个提交加入 `160000` 条目、下一个提交改变它的目标。① have 为链上 seq = k 的视图提交 c_k 时，pack 恰含 (k, s_w] 内的视图提交，加上对其中每个视图提交逐路径比较它与链上前一个视图提交的 tree 得到的新增或改变的 tree 与 blob（同名同 id 的条目整棵跳过），每个对象一次；k 取 0 之外的三个值；② have 含链上视图提交时（① 与 ⑤），pack 不含 `a/big/` 下的对象；③ 测试在内存中把 pack 中的对象与 c_k 的完整闭包（c_k 及其全部祖先视图提交，以及这些提交的 tree 闭包）合并，合并结果包含 want 的完整闭包（seq ≤ s_w 的全部视图提交及其 tree 闭包，不含 `160000` 条目指向的对象）；④ have 全不在视图链上（`mega_commit` 中存在的根提交、另一个过滤器的视图提交、不存在的 40 位 hex）时，pack 与 have 为空时 `full_pack` 的结果一一对应；⑤ have 同时含链上 seq = 2、seq = 5 的视图提交与链外提交时，按 seq = 5 计算；⑥ 任何 pack 都不含 gitlink 的新旧目标） -- [ ] `source .env.test && cargo test -p mega2 --lib 'ceres::pack::view_repo::tests::pack_statement_count_independent_of_chain' -- --exact`（new；AC 4。用 `test_db_config` 取得 `(db_config, _schema)`，守卫持有到用例结束；经 `database_connection(&db_config)` 自建连接，以 `Storage::new_with_connection(config, Arc::new(conn), mock_object_storage())` 组装存储，在这条连接上设 `set_metric_callback` 计数；不在 `test_db_connection` 返回的连接上替换回调（GC-HP-10）。夹具：视图 `:/a` 有 200 个视图提交，视图 `:/b` 有 10 个视图提交，两者每个提交都改动深度为 3 的一个文件，每层待读 id 都少于 1000。每次调用前计数清零，收齐 pack 流后读取计数。断言：两个视图以 tip 为 want 的 `full_pack` 语句数相等；`:/a` 上 have 为 tip 之前 1 个与之前 150 个视图提交的 `incremental_pack` 语句数相等） -- [ ] `source .env.test && docker compose -p mega2-it -f docker/docker-compose.test.yml --profile git up -d --wait && cargo test -p mega2 --test integration_git_cli integration_git_cli_view_pack_http_josh_proxy -- --exact --test-threads=1`（new；AC 5 的 HTTP 部分。服务由 `boot_service_http_with_env`（`tests/integration_git_cli.rs:308-349`）启动，extra_env 为 `trunk_boot_env()`（`:1658-1664`）的三项，加上 `MEGA_VIEWS__ENABLED=true` 与 `MEGA_VIEWS__ALLOW_ANONYMOUS_REGISTER=true`。每个 Josh case 用独立路径 `/project/hp22-`。种子步骤的写法同 `seed_project_foo`（`tests/integration_git_cli.rs:1784-1809`，宿主 git 经 `trunk_host_git`）：clone `/project`，把 `.t` 第一个提交的文件放到 `hp22-/` 下，以 `.t` 中该提交的 message 提交，经 `/project` 以 N = 1 推送。不从空历史直接推到 `/project/hp22-.git`（trunk 以 `MONO_PATH_UNINITIALIZED` 拒绝），也不开通路径（开通写入的 `.gitkeep` 提交会破坏与 fixture 的逐字节一致）。`.t` 之后的提交按顺序逐个经宿主 git 以 N = 1 推送，经 `/project` 的克隆或 `/project/hp22-.git` 的克隆均可。注册把 Josh URL 中的过滤器套在该路径上的视图，对照关系写在 fixture 的 README 中。视图侧的 git 都经 `git_cli::git_cli_no_auth`（`tests/common/git_cli.rs:603-643`）在容器 runner 中执行，URL 取 `git_cli::mega2_http_url(port, "<视图路径>")`。每个 case 建两个克隆。v0 克隆以 `git -c protocol.version=0 clone ` 取得，此后在它里面的每次 fetch 都写作 `git -C -c protocol.version=0 fetch`：子命令之前的 `-c` 只对当次命令生效，不写进克隆的配置，所以每次 fetch 都显式带上；这些 fetch 带 have，经 v0 的 have 分支（`src/ceres/protocol/smart.rs:272-305`）到达 `incremental_pack`。v2 克隆的 clone 与 fetch 都用默认协议。按 `.t` 的 fetch 段，每次经宿主 git 推送并按前置「投影追平」规则轮询之后，两个克隆各 fetch 一次，再在本地以 `git -C merge --ff-only @{upstream}` 前进 HEAD（不经网络）。输入确认：每次 clone 与 fetch（含 v0 克隆中的 fetch）退出码都为 0。断言：clone 之后，以及每次 fetch 并前进 HEAD 之后，都在容器中执行 `git log --graph --pretty=%s` 与 `git ls-tree -r --name-only HEAD`，两个克隆的输出都与 fixture 逐字节相等。三个 case 中至少一个用 `/.filter/.git`，至少一个用 `/.view/@.git`） -- [ ] `source .env.test && docker compose -p mega2-it -f docker/docker-compose.test.yml --profile git up -d --wait && cargo test -p mega2 --test integration_git_cli integration_git_cli_view_pack_http_fetch_after_trunk_push -- --exact --test-threads=1`（new；AC 7。服务由 `boot_service_http_with_env`（`tests/integration_git_cli.rs:308-349`）启动，extra_env 为 `trunk_boot_env()`（`:1658-1664`）的三项加上 `MEGA_VIEWS__ENABLED=true` 与 `MEGA_VIEWS__ALLOW_ANONYMOUS_REGISTER=true`。视图为 `:/project/hp22-fetch`。种子步骤的写法同 `seed_project_foo`（`tests/integration_git_cli.rs:1784-1809`，宿主 git 经 `trunk_host_git`）：clone `/project`，写入 `hp22-fetch/seed.txt` 并提交，经 `/project` 以 N = 1 推送；注册视图后，以 `git_cli::git_cli_no_auth` 在容器 runner 中默认 v2 clone 视图，URL 取 `git_cli::mega2_http_url`。之后的推送都在 `/project/hp22-fetch.git` 的宿主克隆中提交，经 `trunk_host_git` 推送。带 trace 的 `git fetch` 由本用例文件中新增的用例内 helper 在容器中执行：`docker exec -e GIT_TRACE_PACKET=1`，其余 `-w` 与 `-e` 参数同 `git_cli_no_auth` 的容器分支（`tests/common/git_cli.rs:615-643`）；容器 id 用 `docker ps -q --filter label=com.docker.compose.project=mega2-it --filter label=com.docker.compose.service=git-cli --filter status=running` 取得，与该文件私有 helper（`:349-378`）写法相同，不走 Compose 的 `exec` 子命令；工作目录为 `/work/` 加克隆目录相对 `git_cli::git_cli_workdir()` 的路径，与私有 `container_path_for_host`（`:513-525`）相同。`git rev-list` 经 `git_cli_no_auth` 在容器中执行。N = 1：推送一个改动视图内文件的提交，按前置「投影追平」规则轮询到 `lag_commits` 为 0 之后，执行带 trace 的 `git fetch`（输入确认：fetch 退出码 0；trace 中的 fetch 请求含 `have` 行，说明本次 fetch 带 have、经 `incremental_pack` 发包）；断言：`git rev-list --count <旧 tip>..origin/main` 为 1。N > 1：一次推送两个都改动视图内文件的提交，同样先轮询，再执行带 trace 的 `git fetch`（输入确认同 N = 1）；断言：`git rev-list --count <旧 tip>..origin/main` 为 1。`<旧 tip>` 为该次 fetch 之前 `origin/main` 指向的提交。git clone 与 git fetch 自带对所收对象的连通性检查，本行不另跑 `git fsck`） -- [ ] `source .env.test && docker compose -p mega2-it -f docker/docker-compose.test.yml --profile git up -d --wait && cargo test -p mega2 --test integration_git_cli integration_git_cli_view_pack_http_clone_ -- --test-threads=1`（new；前缀过滤，`test result` 中 passed ≥ 3，输出中下列三个用例都为 `ok`。每个用例的服务都由 `boot_service_http_with_env`（`tests/integration_git_cli.rs:308-349`）启动，extra_env 为 `trunk_boot_env()`（`:1658-1664`）的三项加上 `MEGA_VIEWS__ENABLED=true` 与 `MEGA_VIEWS__ALLOW_ANONYMOUS_REGISTER=true`；种子推送与 gitlink 夹具的 `update-index`、`write-tree`、`commit-tree` 用宿主 git（`trunk_host_git`，`:1669-1697`）；视图侧的 clone、`git ls-tree`、`git for-each-ref` 都经 `git_cli::git_cli_no_auth` 在容器 runner 中执行，URL 取 `git_cli::mega2_http_url`。`integration_git_cli_view_pack_http_clone_gitlink`（AC 3）：clone `/project`，在该克隆中执行 `git update-index --add --cacheinfo 160000,<40 位 hex>,hp22-gitlink/` 与 `git write-tree`，再以 `git commit-tree -p HEAD -m ` 生成提交，用 `push --no-thin origin <该提交>:refs/heads/main` 经 `/project` 以 N = 1 推送，注册 `:/project/hp22-gitlink`；断言：`-c protocol.version=0` 与默认 v2 的 clone 都退出 0（输入确认：两个克隆中 `git ls-tree -r HEAD` 都含该 `160000` 条目，说明视图确含 gitlink）。开工时（ER-02）先在未改代码的树上确认 trunk 接受这一推送；只有当 trunk 因 gitlink 条目本身拒绝时（路径策略的拒绝不算），才改由测试进程按 ADR-HP-06 直接写库构造该根提交，并按 ER-03 修订本卡。`integration_git_cli_view_pack_http_clone_ready_empty`（AC 6 的 HTTP 部分）：注册根历史中从未出现的 `:/project/hp22-absent` 并等到就绪；宿主 curl 带 `Git-Protocol: version=2` 与 `Content-Type: application/x-git-upload-pack-request` 两个头，POST 到 `git_cli::mega2_host_http_url(port, "<视图路径>/git-upload-pack")`（容器中的 clone 仍用 `git_cli::mega2_http_url`），请求体为 `0014command=ls-refs\n0001000csymrefs\n0000`，状态码 200，响应体恰为 `0000`；`-c protocol.version=0` 与默认 v2 的 clone 都退出 0，stderr 含 `You appear to have cloned an empty repository`，`git for-each-ref` 输出为空。`integration_git_cli_view_pack_http_clone_depth_filter_400`（AC 8）：视图为 `:/project/hp22-depth`，种子步骤的写法同 `seed_project_foo`（`:1784-1809`）：clone `/project`，写入 `hp22-depth/seed.txt` 并提交，经 `/project` 以 N = 1 推送；对该视图，`-c protocol.version=0 clone --depth 1`、默认 v2 `clone --depth 1`、默认 v2 `clone --filter=blob:none` 都退出非零，stderr 含 `RPC failed; HTTP 400`。输入确认/对照（证明 400 来自 depth / filter 参数）：同一 URL 不带这两个参数的 v0 与 v2 clone 都退出 0） -- [ ] `source .env.test && docker compose -p mega2-it -f docker/docker-compose.test.yml --profile git up -d --wait && cargo test -p mega2 --test integration_git_ssh integration_git_ssh_view_pack_ssh_ -- --test-threads=1`（new；前缀过滤，`test result` 中 passed ≥ 2，输出中下列两个用例都为 `ok`。服务以 `service multi http ssh` 启动，用 HP-18 在该文件中新增的、多接收一组环境变量的 `boot_service_multi` 同形变体（既有 `boot_service_multi`（`tests/integration_git_ssh.rs:1727-1733`）只接受 `(env, enforcement)`，传不进视图与 trunk 的环境变量）。开工时（ER-02）核对该变体存在；只有核对不到时，本卡才在该文件中新增一个：端口分配、命令行（`--host` 取 `git_cli::service_listen_host()`）与等待监听同 `boot_service_multi_with_session`（`:1736-1784`），额外环境变量逐个 `command.env` 写入，写法同 `boot_service_ssh_with_env`（`:1689-1720`），既有 helper 的签名与行为不变。用例以 `enforcement = "off"` 调用该变体：先启动一次再以 SIGINT 关停（迁移与 SSH host key），再正式启动，两次启动的写法同 `boot_storage_only_ssh`（`:523-541`）。两次启动传入同一组环境变量：`MEGA_VIEWS__ENABLED=true`、`MEGA_VIEWS__ALLOW_ANONYMOUS_REGISTER=true`、`MEGA_MONOREPO__PUSH_POLICY=trunk`、`MEGA_GIT__PUSH_AUTH=none`、`MEGA_GIT__SSH_RECEIVE_PACK=false`、`MEGA_GIT__ANONYMOUS_ACCESS=true`。夹具推送与注册走 HTTP 腿，在宿主执行：推送用宿主 git 访问 `http://127.0.0.1:/project`（经 `git_host_ssh`（`:480-501`）执行，HTTP URL 不受 `GIT_SSH_COMMAND` 影响），注册由宿主发 HTTP 请求。视图侧的 git 经 SSH 腿在容器 runner 中匿名执行，写法同 `prepare_authenticated_ssh_multi_with_session`（`:1808-1861`）的 SSH 客户端部分，但不登记公钥：启动之前以 `git_cli::generate_client_ed25519` 在 `env.ssh_dir` 下生成客户端密钥；正式启动之后以 `git_cli::write_known_hosts_via_keyscan(&env.ssh_dir.join("known_hosts"), ssh_port)` 在容器内取得 known_hosts；`GIT_SSH_COMMAND` 取 `git_cli::git_ssh_command(&env.case_dir, ssh_port)`（`tests/common/git_cli.rs:1245-1262`）；git 由 `git_cli::git_cli_ssh`（`:1269-1279`）执行；`git_cli::mega2_ssh_repo_url(ssh_port, "git")` 的返回值以 `/` 结尾（`tests/common/git_cli.rs:340-342`），URL 由它直接接上不带前导 `/` 的视图路径（`.filter/.git` 或 `.view/@.git`）组成，形如 `ssh://git@:/.filter/.git`，不出现 `//`。storage-only 下服务端接受 none 认证、拒绝公钥认证（`src/contract/git_protocol/ssh.rs:249-266`、`:297-306`），OpenSSH 客户端先发 none 认证请求，连接因此是匿名的。`integration_git_ssh_view_pack_ssh_josh_proxy`（AC 5 的 SSH 部分）：三个 Josh case 各用独立路径 `/project/hp22-`。种子步骤经 HTTP 腿执行：clone `/project`，把 `.t` 第一个提交的文件放到 `hp22-/` 下，以 `.t` 中该提交的 message 提交，经 `/project` 以 N = 1 推送（不从空历史直接推到新路径，也不开通路径）；`.t` 之后的提交按顺序逐个经 HTTP 腿以 N = 1 推送，经 `/project` 的克隆或 `/project/hp22-.git` 的克隆均可。注册把 Josh URL 中的过滤器套在该路径上的视图，对照关系与 HTTP 用例相同（写在 `tests/fixtures/views/josh_proxy/` 的 README 中）。每个 case 经 SSH 建两个克隆。v0 克隆以 `git -c protocol.version=0 clone ` 取得，此后在它里面的每次 fetch 都写作 `git -C -c protocol.version=0 fetch`：子命令之前的 `-c` 只对当次命令生效，不写进克隆的配置，所以每次 fetch 都显式带上；这些 fetch 带 have，经 v0 的 have 分支（`src/ceres/protocol/smart.rs:272-305`）到达 `incremental_pack`。v2 克隆的 clone 与 fetch 都用默认协议。按 `.t` 的 fetch 段，每次经 HTTP 腿推送并按前置「投影追平」规则轮询之后，两个克隆各经 SSH fetch 一次，再在本地以 `git -C merge --ff-only @{upstream}` 前进 HEAD（不经网络）。输入确认：每次 clone 与 fetch（含 v0 克隆中的 fetch）退出码都为 0。断言：clone 之后，以及每次 fetch 并前进 HEAD 之后，都经 `git_cli_ssh` 在容器中执行 `git log --graph --pretty=%s` 与 `git ls-tree -r --name-only HEAD`，两个克隆的输出都与 fixture 逐字节相等。`integration_git_ssh_view_pack_ssh_ready_empty`（AC 6 的 SSH 部分）：注册根历史中从未出现的 `:/project/hp22-absent` 并等到就绪，经 SSH 的 `-c protocol.version=0` 与默认 v2 clone 都退出 0，stderr 含 `You appear to have cloned an empty repository`，在容器中执行的 `git for-each-ref` 输出为空） -- [ ] `source .env.test && bash -c 'chk(){ if rg -q -- "$2" "$1"; then echo "OK: $1: $2"; else rc=$?; echo "FAIL: $1: $2 (rg exit $rc)"; exit 1; fi; }; for f in docs/user-guide.zh.md docs/user-guide.md docs/architecture.zh.md docs/architecture.md; do chk "$f" "/\.view/@\.git"; chk "$f" "/\.filter/\.git"; done; chk docs/user-guide.zh.md "视图不是读安全边界"; chk docs/user-guide.md "[Vv]iews are not a read security boundary"; for f in docs/user-guide.zh.md docs/user-guide.md; do chk "$f" "ERR view unavailable"; done; p=docs/refactoring/protocol.md; chk "$p" "^\| 视图 clone \| cargo:integration_git_cli_view_pack_http_josh_proxy \| cargo:integration_git_ssh_view_pack_ssh_josh_proxy \|"; chk "$p" "^\| 视图 fetch \| cargo:integration_git_cli_view_pack_http_fetch_after_trunk_push \| cargo:integration_git_ssh_view_pack_ssh_josh_proxy \|"; chk "$p" "^\| 已就绪的空视图 \| cargo:integration_git_cli_view_pack_http_clone_ready_empty \| cargo:integration_git_ssh_view_pack_ssh_ready_empty \|"; chk "$p" "^\| 视图 clone \|.*\| cargo:integration_git_cli_view_pack_http_clone_gitlink \|$"; chk "$p" "^\| .shallow. \|.*ViewRepo"; chk "$p" "^\| .fetch=shallow filter. \|.*ViewRepo"; chk docs/architecture.zh.md "^\| 视图 API \| .*/api/v1/views.*\| .*\[views\]\.enabled"; chk docs/architecture.md "^\| Views API \| .*/api/v1/views.*\| .*\[views\]\.enabled"'`（文档同步 → Docs and compatibility impact。开工前全部不命中；`rg` 执行失败同样判 FAIL） +- [x] `source .env.test && cargo test -p mega2 --lib 'ceres::pack::view_repo::tests::full_pack_closure_one_to_one' -- --exact`（new；AC 1。夹具：线性根历史 8 个根提交；Subdir、Prefix、Exclude、不相交 Compose 四类视图各一个，另有一个闭包含 `160000` 条目的 Subdir 视图，该条目的目标 id 不在任何表与对象存储中。每个视图以 tip 与一个较早的视图提交为 want 各调用一次 `full_pack`，收齐流后用 `Pack::new_with_hash_kind` 解码，写法同 `src/ceres/pack/monorepo.rs:3779-3809`。断言：解码成功；条目数等于 pack 头的对象数；按条目字节重算的 id 无重复，集合等于测试按夹具在内存中独立算出的期望闭包；Prefix 视图的 pack 含 `mega_view_object` 中的脊柱 tree；gitlink 视图的 pack 不含该目标 id） +- [x] `source .env.test && cargo test -p mega2 --lib 'ceres::pack::view_repo::tests::incremental_pack_have_exclusion' -- --exact`（new；AC 2，逐项断言设计 §6.3「打包」的 have 侧排除规则与 gitlink 规则、方法表 `incremental_pack` 行。夹具：线性根历史 12 个根提交，视图 `:/a` 下有每个提交都不改动的子树 `a/big/`（1 500 个 blob，跨越分块边界），其余提交改动 `a/x/` 下的文件，其中一个提交把文件从 `a/x/m` 移到 `a/y/m`，一个提交加入 `160000` 条目、下一个提交改变它的目标。① have 为链上 seq = k 的视图提交 c_k 时，pack 恰含 (k, s_w] 内的视图提交，加上对其中每个视图提交逐路径比较它与链上前一个视图提交的 tree 得到的新增或改变的 tree 与 blob（同名同 id 的条目整棵跳过），每个对象一次；k 取 0 之外的三个值；② have 含链上视图提交时（① 与 ⑤），pack 不含 `a/big/` 下的对象；③ 测试在内存中把 pack 中的对象与 c_k 的完整闭包（c_k 及其全部祖先视图提交，以及这些提交的 tree 闭包）合并，合并结果包含 want 的完整闭包（seq ≤ s_w 的全部视图提交及其 tree 闭包，不含 `160000` 条目指向的对象）；④ have 全不在视图链上（`mega_commit` 中存在的根提交、另一个过滤器的视图提交、不存在的 40 位 hex）时，pack 与 have 为空时 `full_pack` 的结果一一对应；⑤ have 同时含链上 seq = 2、seq = 5 的视图提交与链外提交时，按 seq = 5 计算；⑥ 任何 pack 都不含 gitlink 的新旧目标） +- [x] `source .env.test && cargo test -p mega2 --lib 'ceres::pack::view_repo::tests::pack_statement_count_independent_of_chain' -- --exact`（new；AC 4。用 `test_db_config` 取得 `(db_config, _schema)`，守卫持有到用例结束；经 `database_connection(&db_config)` 自建连接，以 `Storage::new_with_connection(config, Arc::new(conn), mock_object_storage())` 组装存储，在这条连接上设 `set_metric_callback` 计数；不在 `test_db_connection` 返回的连接上替换回调（GC-HP-10）。夹具：视图 `:/a` 有 200 个视图提交，视图 `:/b` 有 10 个视图提交，两者每个提交都改动深度为 3 的一个文件，每层待读 id 都少于 1000。每次调用前计数清零，收齐 pack 流后读取计数。断言：两个视图以 tip 为 want 的 `full_pack` 语句数相等；`:/a` 上 have 为 tip 之前 1 个与之前 150 个视图提交的 `incremental_pack` 语句数相等） +- [x] `source .env.test && docker compose -p mega2-it -f docker/docker-compose.test.yml --profile git up -d --wait && cargo test -p mega2 --test integration_git_cli integration_git_cli_view_pack_http_josh_proxy -- --exact --test-threads=1`（new；AC 5 的 HTTP 部分。服务由 `boot_service_http_with_env`（`tests/integration_git_cli.rs:308-349`）启动，extra_env 为 `trunk_boot_env()`（`:1658-1664`）的三项，加上 `MEGA_VIEWS__ENABLED=true` 与 `MEGA_VIEWS__ALLOW_ANONYMOUS_REGISTER=true`。每个 Josh case 用独立路径 `/project/hp22-`。种子步骤的写法同 `seed_project_foo`（`tests/integration_git_cli.rs:1784-1809`，宿主 git 经 `trunk_host_git`）：clone `/project`，把 `.t` 第一个提交的文件放到 `hp22-/` 下，以 `.t` 中该提交的 message 提交，经 `/project` 以 N = 1 推送。不从空历史直接推到 `/project/hp22-.git`（trunk 以 `MONO_PATH_UNINITIALIZED` 拒绝），也不开通路径（开通写入的 `.gitkeep` 提交会破坏与 fixture 的逐字节一致）。`.t` 之后的提交按顺序逐个经宿主 git 以 N = 1 推送，经 `/project` 的克隆或 `/project/hp22-.git` 的克隆均可。注册把 Josh URL 中的过滤器套在该路径上的视图，对照关系写在 fixture 的 README 中。视图侧的 git 都经 `git_cli::git_cli_no_auth`（`tests/common/git_cli.rs:603-643`）在容器 runner 中执行，URL 取 `git_cli::mega2_http_url(port, "<视图路径>")`。每个 case 建两个克隆。v0 克隆以 `git -c protocol.version=0 clone ` 取得，此后在它里面的每次 fetch 都写作 `git -C -c protocol.version=0 fetch`：子命令之前的 `-c` 只对当次命令生效，不写进克隆的配置，所以每次 fetch 都显式带上；这些 fetch 带 have，经 v0 的 have 分支（`src/ceres/protocol/smart.rs:272-305`）到达 `incremental_pack`。v2 克隆的 clone 与 fetch 都用默认协议。按 ER-03 登记的 `*.fetch.*` 扩展，每次经宿主 git 推送并按前置「投影追平」规则轮询之后，两个克隆各 fetch 一次，再在本地以 `git -C merge --ff-only @{upstream}` 前进 HEAD（不经网络）。输入确认：每次 clone 与 fetch（含 v0 克隆中的 fetch）退出码都为 0。断言：clone 之后，以及每次 fetch 并前进 HEAD 之后，都在容器中执行 `git log --graph --pretty=%s` 与 `git ls-tree -r --name-only HEAD`，两个克隆的输出都与 fixture 逐字节相等。三个 case 中至少一个用 `/.filter/.git`，至少一个用 `/.view/@.git`） +- [x] `source .env.test && docker compose -p mega2-it -f docker/docker-compose.test.yml --profile git up -d --wait && cargo test -p mega2 --test integration_git_cli integration_git_cli_view_pack_http_fetch_after_trunk_push -- --exact --test-threads=1`（new；AC 7。服务由 `boot_service_http_with_env`（`tests/integration_git_cli.rs:308-349`）启动，extra_env 为 `trunk_boot_env()`（`:1658-1664`）的三项加上 `MEGA_VIEWS__ENABLED=true` 与 `MEGA_VIEWS__ALLOW_ANONYMOUS_REGISTER=true`。视图为 `:/project/hp22-fetch`。种子步骤的写法同 `seed_project_foo`（`tests/integration_git_cli.rs:1784-1809`，宿主 git 经 `trunk_host_git`）：clone `/project`，写入 `hp22-fetch/seed.txt` 并提交，经 `/project` 以 N = 1 推送；注册视图后，以 `git_cli::git_cli_no_auth` 在容器 runner 中默认 v2 clone 视图，URL 取 `git_cli::mega2_http_url`。之后的推送都在 `/project/hp22-fetch.git` 的宿主克隆中提交，经 `trunk_host_git` 推送。带 trace 的 `git fetch` 由本用例文件中新增的用例内 helper 在容器中执行：`docker exec -e GIT_TRACE_PACKET=1`，其余 `-w` 与 `-e` 参数同 `git_cli_no_auth` 的容器分支（`tests/common/git_cli.rs:615-643`）；容器 id 用 `docker ps -q --filter label=com.docker.compose.project=mega2-it --filter label=com.docker.compose.service=git-cli --filter status=running` 取得，与该文件私有 helper（`:349-378`）写法相同，不走 Compose 的 `exec` 子命令；工作目录为 `/work/` 加克隆目录相对 `git_cli::git_cli_workdir()` 的路径，与私有 `container_path_for_host`（`:513-525`）相同。`git rev-list` 经 `git_cli_no_auth` 在容器中执行。N = 1：推送一个改动视图内文件的提交，按前置「投影追平」规则轮询到 `lag_commits` 为 0 之后，执行带 trace 的 `git fetch`（输入确认：fetch 退出码 0；trace 中的 fetch 请求含 `have` 行，说明本次 fetch 带 have、经 `incremental_pack` 发包）；断言：`git rev-list --count <旧 tip>..origin/main` 为 1。N > 1：一次推送两个都改动视图内文件的提交，同样先轮询，再执行带 trace 的 `git fetch`（输入确认同 N = 1）；断言：`git rev-list --count <旧 tip>..origin/main` 为 1。`<旧 tip>` 为该次 fetch 之前 `origin/main` 指向的提交。git clone 与 git fetch 自带对所收对象的连通性检查，本行不另跑 `git fsck`） +- [x] `source .env.test && docker compose -p mega2-it -f docker/docker-compose.test.yml --profile git up -d --wait && cargo test -p mega2 --test integration_git_cli integration_git_cli_view_pack_http_clone_ -- --test-threads=1`（new；前缀过滤，`test result` 中 passed ≥ 3，输出中下列三个用例都为 `ok`。每个用例的服务都由 `boot_service_http_with_env`（`tests/integration_git_cli.rs:308-349`）启动，extra_env 为 `trunk_boot_env()`（`:1658-1664`）的三项加上 `MEGA_VIEWS__ENABLED=true` 与 `MEGA_VIEWS__ALLOW_ANONYMOUS_REGISTER=true`；种子推送与 gitlink 夹具的 `update-index`、`write-tree`、`commit-tree` 用宿主 git（`trunk_host_git`，`:1669-1697`）；视图侧的 clone、`git ls-tree`、`git for-each-ref` 都经 `git_cli::git_cli_no_auth` 在容器 runner 中执行，URL 取 `git_cli::mega2_http_url`。`integration_git_cli_view_pack_http_clone_gitlink`（AC 3）：clone `/project`，在该克隆中执行 `git update-index --add --cacheinfo 160000,<40 位 hex>,hp22-gitlink/` 与 `git write-tree`，再以 `git commit-tree -p HEAD -m ` 生成提交，用 `push --no-thin origin <该提交>:refs/heads/main` 经 `/project` 以 N = 1 推送，注册 `:/project/hp22-gitlink`；断言：`-c protocol.version=0` 与默认 v2 的 clone 都退出 0（输入确认：两个克隆中 `git ls-tree -r HEAD` 都含该 `160000` 条目，说明视图确含 gitlink）。开工时（ER-02）先在未改代码的树上确认 trunk 接受这一推送；只有当 trunk 因 gitlink 条目本身拒绝时（路径策略的拒绝不算），才改由测试进程按 ADR-HP-06 直接写库构造该根提交，并按 ER-03 修订本卡。`integration_git_cli_view_pack_http_clone_ready_empty`（AC 6 的 HTTP 部分）：注册根历史中从未出现的 `:/project/hp22-absent` 并等到就绪；宿主 curl 带 `Git-Protocol: version=2` 与 `Content-Type: application/x-git-upload-pack-request` 两个头，POST 到 `git_cli::mega2_host_http_url(port, "<视图路径>/git-upload-pack")`（容器中的 clone 仍用 `git_cli::mega2_http_url`），请求体为 `0014command=ls-refs\n0001000csymrefs\n0000`，状态码 200，响应体恰为 `0000`；`-c protocol.version=0` 与默认 v2 的 clone 都退出 0，stderr 含 `You appear to have cloned an empty repository`，`git for-each-ref` 输出为空。`integration_git_cli_view_pack_http_clone_depth_filter_400`（AC 8）：视图为 `:/project/hp22-depth`，种子步骤的写法同 `seed_project_foo`（`:1784-1809`）：clone `/project`，写入 `hp22-depth/seed.txt` 并提交，经 `/project` 以 N = 1 推送；对该视图，`-c protocol.version=0 clone --depth 1`、默认 v2 `clone --depth 1`、默认 v2 `clone --filter=blob:none` 都退出非零，stderr 含 `RPC failed; HTTP 400`。输入确认/对照（证明 400 来自 depth / filter 参数）：同一 URL 不带这两个参数的 v0 与 v2 clone 都退出 0） +- [x] `source .env.test && docker compose -p mega2-it -f docker/docker-compose.test.yml --profile git up -d --wait && cargo test -p mega2 --test integration_git_ssh integration_git_ssh_view_pack_ssh_ -- --test-threads=1`（new；前缀过滤，`test result` 中 passed ≥ 2，输出中下列两个用例都为 `ok`。服务以 `service multi http ssh` 启动，用 HP-18 在该文件中新增的、多接收一组环境变量的 `boot_service_multi` 同形变体（既有 `boot_service_multi`（`tests/integration_git_ssh.rs:1727-1733`）只接受 `(env, enforcement)`，传不进视图与 trunk 的环境变量）。开工时（ER-02）核对该变体存在；只有核对不到时，本卡才在该文件中新增一个：端口分配、命令行（`--host` 取 `git_cli::service_listen_host()`）与等待监听同 `boot_service_multi_with_session`（`:1736-1784`），额外环境变量逐个 `command.env` 写入，写法同 `boot_service_ssh_with_env`（`:1689-1720`），既有 helper 的签名与行为不变。用例以 `enforcement = "off"` 调用该变体：先启动一次再以 SIGINT 关停（迁移与 SSH host key），再正式启动，两次启动的写法同 `boot_storage_only_ssh`（`:523-541`）。两次启动传入同一组环境变量：`MEGA_VIEWS__ENABLED=true`、`MEGA_VIEWS__ALLOW_ANONYMOUS_REGISTER=true`、`MEGA_MONOREPO__PUSH_POLICY=trunk`、`MEGA_GIT__PUSH_AUTH=none`、`MEGA_GIT__SSH_RECEIVE_PACK=false`、`MEGA_GIT__ANONYMOUS_ACCESS=true`。夹具推送与注册走 HTTP 腿，在宿主执行：推送用宿主 git 访问 `http://127.0.0.1:/project`（经 `git_host_ssh`（`:480-501`）执行，HTTP URL 不受 `GIT_SSH_COMMAND` 影响），注册由宿主发 HTTP 请求。视图侧的 git 经 SSH 腿在容器 runner 中匿名执行，写法同 `prepare_authenticated_ssh_multi_with_session`（`:1808-1861`）的 SSH 客户端部分，但不登记公钥：启动之前以 `git_cli::generate_client_ed25519` 在 `env.ssh_dir` 下生成客户端密钥；正式启动之后以 `git_cli::write_known_hosts_via_keyscan(&env.ssh_dir.join("known_hosts"), ssh_port)` 在容器内取得 known_hosts；`GIT_SSH_COMMAND` 取 `git_cli::git_ssh_command(&env.case_dir, ssh_port)`（`tests/common/git_cli.rs:1245-1262`）；git 由 `git_cli::git_cli_ssh`（`:1269-1279`）执行；`git_cli::mega2_ssh_repo_url(ssh_port, "git")` 的返回值以 `/` 结尾（`tests/common/git_cli.rs:340-342`），URL 由它直接接上不带前导 `/` 的视图路径（`.filter/.git` 或 `.view/@.git`）组成，形如 `ssh://git@:/.filter/.git`，不出现 `//`。storage-only 下服务端接受 none 认证、拒绝公钥认证（`src/contract/git_protocol/ssh.rs:249-266`、`:297-306`），OpenSSH 客户端先发 none 认证请求，连接因此是匿名的。`integration_git_ssh_view_pack_ssh_josh_proxy`（AC 5 的 SSH 部分）：三个 Josh case 各用独立路径 `/project/hp22-`。种子步骤经 HTTP 腿执行：clone `/project`，把 `.t` 第一个提交的文件放到 `hp22-/` 下，以 `.t` 中该提交的 message 提交，经 `/project` 以 N = 1 推送（不从空历史直接推到新路径，也不开通路径）；`.t` 之后的提交按顺序逐个经 HTTP 腿以 N = 1 推送，经 `/project` 的克隆或 `/project/hp22-.git` 的克隆均可。注册把 Josh URL 中的过滤器套在该路径上的视图，对照关系与 HTTP 用例相同（写在 `tests/fixtures/views/josh_proxy/` 的 README 中）。每个 case 经 SSH 建两个克隆。v0 克隆以 `git -c protocol.version=0 clone ` 取得，此后在它里面的每次 fetch 都写作 `git -C -c protocol.version=0 fetch`：子命令之前的 `-c` 只对当次命令生效，不写进克隆的配置，所以每次 fetch 都显式带上；这些 fetch 带 have，经 v0 的 have 分支（`src/ceres/protocol/smart.rs:272-305`）到达 `incremental_pack`。v2 克隆的 clone 与 fetch 都用默认协议。按 ER-03 登记的 `*.fetch.*` 扩展，每次经 HTTP 腿推送并按前置「投影追平」规则轮询之后，两个克隆各经 SSH fetch 一次，再在本地以 `git -C merge --ff-only @{upstream}` 前进 HEAD（不经网络）。输入确认：每次 clone 与 fetch（含 v0 克隆中的 fetch）退出码都为 0。断言：clone 之后，以及每次 fetch 并前进 HEAD 之后，都经 `git_cli_ssh` 在容器中执行 `git log --graph --pretty=%s` 与 `git ls-tree -r --name-only HEAD`，两个克隆的输出都与 fixture 逐字节相等。`integration_git_ssh_view_pack_ssh_ready_empty`（AC 6 的 SSH 部分）：注册根历史中从未出现的 `:/project/hp22-absent` 并等到就绪，经 SSH 的 `-c protocol.version=0` 与默认 v2 clone 都退出 0，stderr 含 `You appear to have cloned an empty repository`，在容器中执行的 `git for-each-ref` 输出为空） +- [x] `source .env.test && bash -c 'chk(){ if rg -q -- "$2" "$1"; then echo "OK: $1: $2"; else rc=$?; echo "FAIL: $1: $2 (rg exit $rc)"; exit 1; fi; }; for f in docs/user-guide.zh.md docs/user-guide.md docs/architecture.zh.md docs/architecture.md; do chk "$f" "/\.view/@\.git"; chk "$f" "/\.filter/\.git"; done; chk docs/user-guide.zh.md "视图不是读安全边界"; chk docs/user-guide.md "[Vv]iews are not a read security boundary"; for f in docs/user-guide.zh.md docs/user-guide.md; do chk "$f" "ERR view unavailable"; done; p=docs/refactoring/protocol.md; chk "$p" "^\| 视图 clone \| cargo:integration_git_cli_view_pack_http_josh_proxy \| cargo:integration_git_ssh_view_pack_ssh_josh_proxy \|"; chk "$p" "^\| 视图 fetch \| cargo:integration_git_cli_view_pack_http_fetch_after_trunk_push \| cargo:integration_git_ssh_view_pack_ssh_josh_proxy \|"; chk "$p" "^\| 已就绪的空视图 \| cargo:integration_git_cli_view_pack_http_clone_ready_empty \| cargo:integration_git_ssh_view_pack_ssh_ready_empty \|"; chk "$p" "^\| 视图 clone \|.*\| cargo:integration_git_cli_view_pack_http_clone_gitlink \|$"; chk "$p" "^\| .shallow. \|.*ViewRepo"; chk "$p" "^\| .fetch=shallow filter. \|.*ViewRepo"; chk docs/architecture.zh.md "^\| 视图 API \| .*/api/v1/views.*\| .*\[views\]\.enabled"; chk docs/architecture.md "^\| Views API \| .*/api/v1/views.*\| .*\[views\]\.enabled"'`（文档同步 → Docs and compatibility impact。开工前全部不命中；`rg` 执行失败同样判 FAIL） **Dependencies:** `HP-20`（消费：`resolve_view_target` 与 `SmartSession.view`，`repo_handler_with_commands` 据此构造 ViewRepo，视图 URL 经 HTTP 与 SSH 到达打包方法；经其上游链传递可用的视图注册接口 `POST /api/v1/views?wait=true`、不带凭据即可查询的 `GET /api/v1/views/{filter_id}`（应答含 `lag_commits`，供 Verification 的「投影追平」轮询）与 `[views].enabled` 下的 `/api/v1/views` 挂载；经其上游链传递可用的、`tests/integration_git_ssh.rs` 中多接收一组环境变量的 `boot_service_multi` 同形变体；写集 `tests/integration_git_cli.rs`、`tests/integration_git_ssh.rs`、`docs/refactoring/protocol.md` 串行；经其上游链传递串行的写集 `docs/architecture.md` / `.zh.md` 与 `docs/user-guide.md` / `.zh.md`）、`HP-19`（消费：`ViewRepo` 及其构造参数（含请求快照与其中的 hash kind）、`check_commit_exist`、经数据库 `TreeSource` 的 `get_trees_by_hashes`、按设计 §6.3 方法表实现的 `get_blobs_by_hashes` 与 `get_blob_metadata_by_hashes`；经其上游链传递可用的数据库 `TreeSource` 及其 `prefetch(ids)`、`read_tree`（经共用解析函数，含同名条目的 tree 判为「无法解析」）与 `MissingObject` 的三种原因；写集 `src/ceres/pack/view_repo.rs`、`src/jupiter/storage/view_projection_storage.rs` 串行）、`DEP-HP-03`（Josh `tests/proxy/` 中 `clone_subtree.t`、`clone_subsubtree.t`、`clone_prefix.t` 在 `e6dfb95e` 的期望输出；取不到时按该登记行的回落处理） @@ -5030,7 +5033,7 @@ ER-04 强制门（不计入 VER）：本卡改 `src/contract/git_protocol/mod.rs **Implementation write set:** - 代码：`src/ceres/pack/view_repo.rs`、`src/jupiter/storage/view_projection_storage.rs` - 测试：`src/ceres/pack/view_repo.rs` 的 `tests` 模块；`tests/integration_git_cli.rs`；`tests/integration_git_ssh.rs`（服务启动复用 HP-18 在该文件中新增的、多接收一组环境变量的 `boot_service_multi` 同形变体；只有开工时 ER-02 核对不到该变体，才在该文件中新增一个）；测试数据 `tests/fixtures/views/josh_proxy/`（新建，含 README：上游 revision `e6dfb95e`、MIT 许可、视图过滤器的对照关系、tree 输出的转写规则，以及与上游的逐条差异） -- 文档：`docs/user-guide.md`、`docs/user-guide.zh.md`、`docs/architecture.md`、`docs/architecture.zh.md`、`docs/refactoring/protocol.md` +- 文档：`docs/user-guide.md`、`docs/user-guide.zh.md`、`docs/architecture.md`、`docs/architecture.zh.md`、`docs/refactoring/protocol.md`、`docs/deploy-trunk.md`（ER-03 限定为 §9 一句事实更新） `src/ceres/pack/view_repo.rs` 的 `tests` 模块中，既有用例 `remaining_methods_match_design` 对 `full_pack` / `incremental_pack` 返回 `view pack not implemented` 的断言随本卡失效；本卡把它改为断言两者不再返回该错误，其余断言不变。本卡与上游卡在 `docs/architecture.md` / `.zh.md`（HP-07 已在第 7 节「热加载」条目的白名单中加入 `views`）、`docs/user-guide.md` / `.zh.md`（HP-08 已在「允许的根」与 `MONO_PATH_INVALID` 条目中写明保留名 `.view`、`.filter`）上相交，由传递依赖串行化（`HP-07 -> HP-18 -> HP-20 -> HP-22`、`HP-08 -> HP-18 -> HP-20 -> HP-22`），本卡在后，与计划「共享文件」表一致；本卡只改架构文档第 6 节的协议面表与用户指南 §2 末尾的新增小节，不改第 7 节「热加载」条目，也不改「允许的根」与 `MONO_PATH_INVALID` 条目。不改 `src/ceres/pack/mod.rs`、`src/ceres/protocol/**`、`src/jupiter/storage/view_tree_source.rs`、`src/jupiter/tests.rs`、`tests/common/` 与 `docs/refactoring/integration.md`。 @@ -6099,6 +6102,10 @@ Result 只允许 `PASS` 或 `FAIL`。`FAIL` 必须列出 P0/P1 条目，并在 | HP21-R2 | HP-21 首轮修订，Claude CLI 独立只读复审 | FAIL | P0 0；P1 0；P2：首次注册仍在旧 worker 周期与名额配置下运行；R3 修订关闭 | P3 的单条 WARN、共享测试环境和运维措辞已修订 | `/tmp/mega2-hp21-claude-r2.txt` | | HP21-R3 | HP-21 最终差异与八项验收，Claude CLI 独立只读复审 | PASS | P0 0；P1 0；P2 0 | R1/R2 已关闭；四种准入结果、重启恢复、认证开关、热加载和运维文档逐项核对 | `/tmp/mega2-hp21-claude-r3.txt`；`VERDICT: PASS` | | HP21-R4 | HP-21 代码、测试、架构、安全、性能及本地门禁，Codex 复核 | PASS | P0 0；P1 0；P2 0 | 8/8 精确验证、ER-04 A 组只读 smoke 8/8、nightly 格式、Clippy、完整测试 19 个 target 全 0 失败；两个原样构建退出 0，专用链接参数下均 0 警告；改动仅在回收态增加准入短事务 | 当前差异、`/tmp/mega2-hp21-ver-final-1.txt` 至 `-8.txt`、`/tmp/mega2-hp21-test-all.txt`、`/tmp/mega2-hp21-build-clean.txt`、`/tmp/mega2-hp21-build-tests-clean.txt` | +| HP22-R1 | HP-22 初始打包实现、黑盒夹具与文档，Claude CLI 独立只读复审 | FAIL | 精确闭包、gitlink 与 Josh 来源说明等证据缺口；R2/R3 前修订关闭 | 测试加强为 pack 对象集合逐项比较，夹具来源与本项目扩展分开记录 | `/tmp/mega2-hp22-claude-r1.txt` | +| HP22-R2 | HP-22 修订后的打包实现与验收，Claude CLI 独立只读复审 | PASS | P0 0；P1 0；低优先级建议 3 项，R3 前关闭 | 容器 fetch 的工作目录及环境、gitlink 精确闭包与 ER-03 措辞补齐 | `/tmp/mega2-hp22-claude-r2.txt`；`VERDICT: PASS` | +| HP22-R3 | HP-22 最终差异与八项验收，Claude CLI 独立只读复审 | PASS | P0 0；P1 0；P2 0 | R2 三项建议均关闭；HTTP/SSH v0/v2、单次 SQL 定位、tree 分层与 pack 闭包逐项核对 | `/tmp/mega2-hp22-claude-r3.txt`；`VERDICT: PASS` | +| HP22-R4 | HP-22 代码、测试、文档及本地门禁，Codex 复核 | PASS | P0 0；P1 0；P2 0 | 8/8 精确验证：lib 3 条各 1/1，HTTP Josh 与增量各 1/1、clone 3/3，SSH 2/2，文档 20 项 OK；19 个目标的完整测试退出 0；nightly 格式与全目标全特性 Clippy 退出 0；两个构建命令退出 0，专用 macOS 链接参数下 0 警告；`libra diff --check` 通过 | 当前差异、`/tmp/mega2-hp22-ver-final-1.txt` 至 `-8.txt`、`/tmp/mega2-hp22-test-all-r2.txt`、`/tmp/mega2-hp22-build-nowarn.txt`、`/tmp/mega2-hp22-build-tests-nowarn.txt` | ## 非目标与延后项 diff --git a/docs/refactoring/protocol.md b/docs/refactoring/protocol.md index 46fda164..b8252638 100644 --- a/docs/refactoring/protocol.md +++ b/docs/refactoring/protocol.md @@ -687,6 +687,9 @@ SSH 中 `git-lfs-transfer` 返回明确 unsupported failure，`git-lfs-authentic | 视图 LFS 404 | cargo:layer2_lfs_not_found | cargo:integration_git_ssh_view_layer2_ssh_lfs_rejected | cargo:layer2_lfs_not_found | N/A+视图前缀不进入 LFS 命名空间 | | 未知或已禁用的视图 | cargo:layer2_disabled_not_found、cargo:integration_git_cli_view_layer3_http_unknown | cargo:integration_git_ssh_view_layer2_ssh_not_found、cargo:integration_git_ssh_view_layer3_unknown | cargo:layer2_disabled_not_found、cargo:layer3_before_resolve_auth | cargo:layer2_enabled_unregistered_not_found | | 视图 ls-remote | cargo:integration_git_cli_view_layer3_http_ls_remote | cargo:integration_git_ssh_view_layer3_ls_remote | cargo:layer3_before_resolve_auth | N/A+视图只有 ViewRepo 一种形态 | +| 视图 clone | cargo:integration_git_cli_view_pack_http_josh_proxy | cargo:integration_git_ssh_view_pack_ssh_josh_proxy | N/A+读授权与路径仓库相同 | cargo:integration_git_cli_view_pack_http_clone_gitlink | +| 视图 fetch | cargo:integration_git_cli_view_pack_http_fetch_after_trunk_push | cargo:integration_git_ssh_view_pack_ssh_josh_proxy | N/A+读授权与路径仓库相同 | N/A+视图只有 ViewRepo 一种形态 | +| 已就绪的空视图 | cargo:integration_git_cli_view_pack_http_clone_ready_empty | cargo:integration_git_ssh_view_pack_ssh_ready_empty | N/A+读授权与路径仓库相同 | N/A+视图只有 ViewRepo 一种形态 | | 未就绪的视图 | cargo:integration_git_cli_view_layer3_http_unready | cargo:integration_git_ssh_view_layer3_unready | cargo:layer3_before_resolve_auth | N/A+视图只有 ViewRepo 一种形态 | #### GAP 映射（plan-20260803） @@ -818,9 +821,9 @@ LFS: | `ofs-delta` | ✅ both | ✅ | ✅ pack decode 委托 `git-internal`（支持 offset delta 编解码，见 `git-internal::internal::pack::decode`） | ✅ advertise/parse：`parse_capabilities_recognizes_ofs_delta` + advertise 断言（pack decode 由 `git-internal` 自测） | OFS_DELTA pack 编解码由 `git-internal` 实现并自测；mega2 侧仅覆盖 advertise/parse | | `multi_ack_detailed` | ✅ upload-pack | ✅ | ✅ negotiation ACK 逻辑 | ✅ `parse_capabilities` 单测 | upload-pack negotiation | | `no-done` | ✅ upload-pack | ✅ | ✅ 与 multi_ack_detailed 联动 | ✅ negotiation 单测 | 允许在 multi_ack_detailed 下提前发 pack | -| `shallow` | ✅ upload-pack | ✅ | ✅ `deepen` / `deepen-relative` 生成 shallow pack 和 `shallow` response | ✅ capability parse/advertise 单测 + shallow traversal 单测 | protocol v1 shallow clone 基础语义 | +| `shallow` | ✅ upload-pack | ✅ | ✅ `deepen` / `deepen-relative` 生成 shallow pack 和 `shallow` response；ViewRepo（P0）不覆写 `supports_shallow_fetch`，HTTP v0/v2 depth 返回 400；SSH 同样拒绝但写裸文本 `error: …`（DEFER-HP-17） | ✅ capability parse/advertise 单测 + shallow traversal 单测 | protocol v1 shallow clone 基础语义 | | `ls-refs` | ✅ protocol v2 | ✅ | ✅ 处理 `ref-prefix`、`symrefs`、`peel` | ✅ v2 capability / command parse 单测 | protocol v2 refs discovery | -| `fetch=shallow filter` | ✅ protocol v2 | ✅ | ✅ v2 fetch 支持 `want`/`have`/`done`、`deepen`、`filter blob:none`；**（2026-06-30）非 Monorepo handler 现在对 shallow/filter 请求返回明确协议错误而非静默 fallback** | ✅ v2 command parse 单测 + pack generation gates + capability honesty 单测 | v2 fetch；`filter blob:none` 只发送 commit/tree objects；`RepoHandler::supports_shallow_fetch`/`supports_filtered_fetch` 门控 | +| `fetch=shallow filter` | ✅ protocol v2 | ✅ | ✅ v2 fetch 支持 `want`/`have`/`done`、`deepen`、`filter blob:none`；ViewRepo（P0）不覆写 `supports_filtered_fetch`，HTTP v2 depth/filter 返回 400，SSH 同样拒绝但写裸文本 `error: …`（DEFER-HP-17）；v0 不宣告 filter | ✅ v2 command parse 单测 + pack generation gates + capability honesty 单测 | v2 fetch；`filter blob:none` 只发送 commit/tree objects；`RepoHandler::supports_shallow_fetch`/`supports_filtered_fetch` 门控 | | `agent=mega/0.1.0` | ✅ both | ❌ | ❌ | ❌ | 信息性，不影响协议行为 | | `atomic` | ❌ 已移除 | ✅ | ❌ | N/A | 未实现原子 ref 更新，已从 advertise 移除 | | `report-status-v2` | ❌ 已移除 | ✅ | ❌ | N/A | 未实现 v2 语义，已从 advertise 移除 | diff --git a/docs/user-guide.md b/docs/user-guide.md index 14d4954b..54700492 100644 --- a/docs/user-guide.md +++ b/docs/user-guide.md @@ -138,6 +138,16 @@ mega2 --config /etc/mega2/config.toml import-repo remove --path /third-party/.git`, `/.view/.git`, and `/.view/@.git`. The unversioned named URL is for human browsing; agents and automation must use `@version`. Views are not a read security boundary; read authorization is the same as for `/.git`. + +View history is projected from the repository beginning. Commits have different hashes from `/.git` even when file contents match, so do not mix these URLs in one remote. Creating a new version replaces the history served by the unversioned named URL. Existing clones may see a non-fast-forward fetch and should be cloned again. + +View URLs are read-only: HTTP push returns 403; SSH push returns one line, `ERR view URLs are read-only`. Over HTTP, `--depth` with v0 or v2, and `--filter` with v2, return 400 and git reports `RPC failed; HTTP 400`. V0 does not advertise filter, so git ignores `--filter` and performs a full clone. SSH rejects depth and v2 filter too, but currently writes plain `error: …` text, so git may only report a protocol error. + +When a view is not ready, HTTP returns 503 with `Retry-After`; SSH returns `ERR view unavailable: ` and exits with code 75, while git stderr contains `remote error: view unavailable`. Retry later. + ## 3. Large files: LFS - **Git LFS (standard)**: endpoints `/info/lfs` and `/api/v1/lfs` work with standard `git-lfs` clients. LFS writes use the same `git.push_auth` setting as Git receive-pack; see the [Deployment Guide](./deployment.md) for authentication modes. diff --git a/docs/user-guide.zh.md b/docs/user-guide.zh.md index 441b747f..9a6eb435 100644 --- a/docs/user-guide.zh.md +++ b/docs/user-guide.zh.md @@ -138,6 +138,16 @@ mega2 --config /etc/mega2/config.toml import-repo remove --path /third-party/.git`、`/.view/.git`、`/.view/@.git` clone 和 fetch；不带版本的命名 URL 只供人工浏览，Agent 和自动化必须使用 `@version`。视图不是读安全边界，读授权规则与 `/.git` 相同。 + +视图历史从仓库起点开始投影；即使文件内容相同，视图提交与 `/.git` 提交的哈希也不同。同一个 remote 不得混用这两种 URL。创建新版本后，不带版本的命名 URL 会整体切换历史，已有克隆 fetch 可能遇到非快进，应重新 clone。 + +视图 URL 只读：HTTP 推送返回 403，SSH 推送返回一行 `ERR view URLs are read-only`。经 HTTP 以 v0 或 v2 使用 `--depth`，或以 v2 使用 `--filter`，服务返回 400，git 报 `RPC failed; HTTP 400`；v0 不宣告 filter，git 会忽略 `--filter` 并完整 clone。SSH 下 depth 和 v2 filter 也被拒，但当前应答是裸文本 `error: …`，git 可能只报告协议错误。 + +未就绪时，HTTP 返回 503 与 `Retry-After`；SSH 返回一行 `ERR view unavailable: ` 并以退出码 75 结束，git stderr 包含 `remote error: view unavailable`。稍后重试即可。 + ## 3. 大文件:LFS - **Git LFS(标准)**:端点 `/info/lfs` 与 `/api/v1/lfs`,stock `git-lfs` 客户端直接可用;LFS 写授权与 Git receive-pack 共用 `git.push_auth`(矩阵见 [`deploy-trunk.md`](./deploy-trunk.md) 第 6 节)。 diff --git a/src/ceres/pack/view_repo.rs b/src/ceres/pack/view_repo.rs index 7a7a3d20..0f5a7a6f 100644 --- a/src/ceres/pack/view_repo.rs +++ b/src/ceres/pack/view_repo.rs @@ -1,22 +1,31 @@ -use std::{collections::HashMap, sync::Arc}; +use std::{ + collections::{BTreeSet, HashMap, HashSet}, + sync::Arc, +}; use async_trait::async_trait; +use futures::TryStreamExt; use git_internal::{ errors::GitError, - hash::HashKind, + hash::{HashKind, ObjectHash}, internal::{ metadata::{EntryMeta, MetaAttached}, - object::tree::Tree, - pack::entry::Entry, + object::{ + ObjectTrait, + tree::{Tree, TreeItemMode}, + types::ObjectType, + }, + pack::{encode::PackEncoder, entry::Entry}, }, }; +use tokio::sync::mpsc; use tokio_stream::wrappers::ReceiverStream; use crate::{ ceres::{ pack::RepoHandler, protocol::import_refs::{RefCommand, Refs}, - view::tree_source::TreeSource, + view::tree_source::{TreeSource, empty_tree_id, read_tree}, }, common::{ errors::{MegaError, ViewUnavailableReason}, @@ -26,8 +35,11 @@ use crate::{ jupiter::{ service::view_projection_service::{CatchUpOutcome, ViewProjectionService}, storage::{ - Storage, base_storage::StorageConnector, view_projection_storage::ViewReaderState, - view_root_chain::RootChainOutcome, view_storage::ViewLockMode, + Storage, + base_storage::StorageConnector, + view_projection_storage::{ViewPackCommit, ViewReaderState}, + view_root_chain::RootChainOutcome, + view_storage::ViewLockMode, view_tree_source::ViewTreeSource, }, }, @@ -43,7 +55,230 @@ pub(crate) struct ViewRepo { filter_id: String, } +struct ViewPackPlan { + commits: Vec, + tree_levels: Vec>, + blobs: Vec, + object_count: usize, +} + impl ViewRepo { + async fn count_pack( + &self, + kind: HashKind, + wants: &[String], + haves: &[String], + ) -> Result { + let view_storage = self.storage.view_storage(); + let bounds = view_storage + .view_pack_bounds(self.filter_pk, wants, haves) + .await + .map_err(GitError::from)?; + let commits = view_storage + .view_pack_commits(self.filter_pk, bounds.have_seq, bounds.want_seq) + .await + .map_err(GitError::from)?; + let empty = empty_tree_id(kind)?.to_string(); + let mut previous = bounds.have_tree.unwrap_or_else(|| empty.clone()); + let mut frontier = BTreeSet::new(); + for commit in &commits { + if commit.tree_id != previous { + frontier.insert((previous.clone(), commit.tree_id.clone())); + } + previous = commit.tree_id.clone(); + } + + let mut seen_trees = HashSet::new(); + let mut seen_blobs = BTreeSet::new(); + let mut tree_levels = Vec::new(); + while !frontier.is_empty() { + let ids = frontier + .iter() + .flat_map(|(old, new)| [old.clone(), new.clone()]) + .collect::>() + .into_iter() + .collect::>(); + let mono = self.storage.mono_storage(); + let mut source = ViewTreeSource::new(&mono, view_storage.get_connection(), kind) + .map_err(GitError::from)?; + source.prefetch(&ids).await.map_err(GitError::from)?; + let trees = ids + .iter() + .map(|id| { + read_tree(kind, &source, id) + .map(|tree| (id.clone(), tree)) + .map_err(|missing| GitError::CustomError(format!("view tree {missing:?}"))) + }) + .collect::, _>>()?; + let mut next = BTreeSet::new(); + let mut level = Vec::new(); + for (old_id, new_id) in frontier { + if seen_trees.insert(new_id.clone()) { + level.push(new_id.clone()); + } + let old = trees.get(&old_id).ok_or_else(|| { + GitError::CustomError(format!("prefetched view tree missing: {old_id}")) + })?; + let new = trees.get(&new_id).ok_or_else(|| { + GitError::CustomError(format!("prefetched view tree missing: {new_id}")) + })?; + let old_items = old + .tree_items + .iter() + .map(|item| (item.name.as_str(), item)) + .collect::>(); + for item in &new.tree_items { + let prior = old_items.get(item.name.as_str()); + if prior.is_some_and(|old_item| old_item.id == item.id) { + continue; + } + match item.mode { + TreeItemMode::Tree => { + let old_child = prior + .filter(|old_item| old_item.mode == TreeItemMode::Tree) + .map_or_else(|| empty.clone(), |old_item| old_item.id.to_string()); + next.insert((old_child, item.id.to_string())); + } + TreeItemMode::Commit => {} + TreeItemMode::Blob | TreeItemMode::BlobExecutable | TreeItemMode::Link => { + seen_blobs.insert(item.id.to_string()); + } + } + } + } + tree_levels.push(level); + frontier = next; + } + let blobs = seen_blobs.into_iter().collect::>(); + let object_count = commits.len() + seen_trees.len() + blobs.len(); + Ok(ViewPackPlan { + commits, + tree_levels, + blobs, + object_count, + }) + } + + async fn send_pack_entry( + sender: &mpsc::Sender>, + kind: HashKind, + object_id: &str, + obj_type: ObjectType, + data: Vec, + meta: EntryMeta, + ) -> Result<(), GitError> { + let hash = ObjectHash::from_hex_for_kind(kind, object_id)?; + sender + .send(MetaAttached { + inner: Entry { + obj_type, + data, + hash, + chain_len: 0, + }, + meta, + }) + .await + .map_err(|error| GitError::CustomError(format!("view pack send failed: {error}"))) + } + + async fn encode_pack( + &self, + plan: ViewPackPlan, + kind: HashKind, + ) -> Result>, GitError> { + let capacity = self.config.pack.channel_message_size; + let (entry_tx, entry_rx) = mpsc::channel(capacity); + let (stream_tx, stream_rx) = mpsc::channel(capacity); + PackEncoder::new_with_hash_kind(kind, plan.object_count, 0, stream_tx) + .encode_async(entry_rx) + .await?; + + for commit in plan.commits { + Self::send_pack_entry( + &entry_tx, + kind, + &commit.object_id, + ObjectType::Commit, + commit.data, + EntryMeta::new(), + ) + .await?; + } + let view_storage = self.storage.view_storage(); + let mono = self.storage.mono_storage(); + for level in plan.tree_levels { + for chunk in level.chunks(1000) { + let mut source = ViewTreeSource::new(&mono, view_storage.get_connection(), kind) + .map_err(GitError::from)?; + source.prefetch(chunk).await.map_err(GitError::from)?; + for id in chunk { + let tree = read_tree(kind, &source, id).map_err(|missing| { + GitError::CustomError(format!("view tree {missing:?}")) + })?; + Self::send_pack_entry( + &entry_tx, + kind, + id, + ObjectType::Tree, + tree.to_data()?, + EntryMeta::new(), + ) + .await?; + } + } + } + for chunk in plan.blobs.chunks(1000) { + let mut metadata = self + .get_blob_metadata_by_hashes(chunk.to_vec()) + .await + .map_err(GitError::from)?; + let mut stream = self + .get_blobs_by_hashes(chunk.to_vec()) + .await + .map_err(GitError::from)?; + let expected = chunk.iter().map(String::as_str).collect::>(); + let mut received = HashSet::new(); + while let Some((key, mut body, _)) = stream + .try_next() + .await + .map_err(MegaError::from) + .map_err(GitError::from)? + { + if !expected.contains(key.key.as_str()) || !received.insert(key.key.clone()) { + return Err(GitError::CustomError(format!( + "unexpected view blob: {}", + key.key + ))); + } + let mut data = Vec::new(); + while let Some(bytes) = body.try_next().await.map_err(|error| { + GitError::CustomError(format!("view blob read failed: {error}")) + })? { + data.extend_from_slice(&bytes); + } + let actual = + ObjectHash::from_type_and_data_for_kind(kind, ObjectType::Blob, &data)?; + if actual.to_string() != key.key { + return Err(GitError::CustomError(format!( + "view blob hash mismatch: {}", + key.key + ))); + } + let meta = metadata.remove(&key.key).unwrap_or_else(EntryMeta::new); + Self::send_pack_entry(&entry_tx, kind, &key.key, ObjectType::Blob, data, meta) + .await?; + } + if received.len() != expected.len() { + return Err(GitError::CustomError( + "view blob missing from object storage".to_owned(), + )); + } + } + drop(entry_tx); + Ok(ReceiverStream::new(stream_rx)) + } + pub(crate) fn new( storage: Storage, config: Arc, @@ -207,20 +442,20 @@ impl RepoHandler for ViewRepo { Err(read_only_git_error()) } - async fn full_pack(&self, _want: Vec) -> Result>, GitError> { - Err(GitError::CustomError( - "view pack not implemented".to_owned(), - )) + async fn full_pack(&self, want: Vec) -> Result>, GitError> { + let kind = self.object_hash_kind().map_err(GitError::from)?; + let plan = self.count_pack(kind, &want, &[]).await?; + self.encode_pack(plan, kind).await } async fn incremental_pack( &self, - _want: Vec, - _have: Vec, + want: Vec, + have: Vec, ) -> Result>, GitError> { - Err(GitError::CustomError( - "view pack not implemented".to_owned(), - )) + let kind = self.object_hash_kind().map_err(GitError::from)?; + let plan = self.count_pack(kind, &want, &have).await?; + self.encode_pack(plan, kind).await } async fn get_trees_by_hashes(&self, hashes: Vec) -> Result, MegaError> { @@ -302,13 +537,27 @@ impl RepoHandler for ViewRepo { #[cfg(test)] mod tests { use std::{ - sync::Arc, + collections::HashSet, + sync::{ + Arc, Mutex, + atomic::{AtomicUsize, Ordering}, + }, time::{Duration, Instant}, }; + use bytes::Bytes; use chrono::Utc; - use futures::TryStreamExt; - use git_internal::{hash::HashKind, internal::object::ObjectTrait}; + use futures::{StreamExt, TryStreamExt}; + use git_internal::{ + hash::{HashKind, ObjectHash}, + internal::{ + object::{ + ObjectTrait, + tree::{TreeItem, TreeItemMode}, + }, + pack::Pack, + }, + }; use sea_orm::{ ActiveModelTrait, ColumnTrait, ConnectionTrait, DbBackend, EntityTrait, IntoActiveModel, PaginatorTrait, QueryFilter, Set, Statement, TransactionTrait, @@ -324,13 +573,15 @@ mod tests { config::testing::isolated_config, jupiter::{ storage::{ + init::database_connection, + object_storage::mock_object_storage, view_storage::{VIEW_LOCK_TIMEOUT, ViewLock, ViewLockMode, acquire_view_lock}, view_test_fixtures::{ - RootCommitFixture, cas_fixture_main, root_tree_from_paths, + RootCommitFixture, RootTreeFixture, cas_fixture_main, root_tree_from_paths, seed_linear_root_history_with_trees, seed_single_parent_root_commit_with_tree, }, }, - tests::test_storage_with_config, + tests::{test_db_config, test_storage_with_config}, }, }; @@ -348,12 +599,6 @@ mod tests { } async fn with_limits(root_count: usize, spec: &str, sync: u64, walk: u64) -> Self { - let temp = tempfile::tempdir().unwrap(); - let mut config = isolated_config(temp.path().join("config")); - config.views.batch_size = 100; - config.views.sync_catch_up_commits = sync; - config.views.max_append_walk = walk; - let storage = test_storage_with_config(temp.path(), config).await; let trees = (0..root_count) .map(|index| { root_tree_from_paths( @@ -371,6 +616,16 @@ mod tests { ) }) .collect(); + Self::from_trees(trees, spec, sync, walk).await + } + + async fn from_trees(trees: Vec, spec: &str, sync: u64, walk: u64) -> Self { + let temp = tempfile::tempdir().unwrap(); + let mut config = isolated_config(temp.path().join("config")); + config.views.batch_size = 100; + config.views.sync_catch_up_commits = sync; + config.views.max_append_walk = walk; + let storage = test_storage_with_config(temp.path(), config).await; let roots = seed_linear_root_history_with_trees( storage.view_storage().get_connection(), HashKind::Sha1, @@ -444,6 +699,22 @@ mod tests { } } + async fn save_blobs(&self) { + let mut saved = HashSet::new(); + for root in &self.roots { + for blob in &root.blobs { + if !saved.insert(blob.id) { + continue; + } + self.storage + .git_service + .save_object_from_raw(Bytes::copy_from_slice(&blob.data)) + .await + .unwrap(); + } + } + } + async fn append(&mut self, count: usize) { let db = self.storage.view_storage(); for index in 0..count { @@ -489,6 +760,557 @@ mod tests { } } + async fn pack_ids(stream: ReceiverStream>, temp: &TempDir) -> (usize, HashSet) { + let bytes = stream.concat().await; + assert!(bytes.len() >= 12); + let declared = u32::from_be_bytes(bytes[8..12].try_into().unwrap()) as usize; + let mut pack = Pack::new_with_hash_kind( + HashKind::Sha1, + Some(1), + Some(64 * 1024 * 1024), + Some(temp.path().to_path_buf()), + true, + ); + let ids = Arc::new(Mutex::new(Vec::new())); + let sink = ids.clone(); + pack.decode( + &mut std::io::Cursor::new(bytes), + move |entry| { + let expected = ObjectHash::from_type_and_data_for_kind( + HashKind::Sha1, + entry.inner.obj_type, + &entry.inner.data, + ) + .unwrap(); + assert_eq!(entry.inner.hash, expected); + sink.lock().unwrap().push(expected.to_string()); + }, + None::, + ) + .unwrap(); + let ids = ids.lock().unwrap(); + let unique = ids.iter().cloned().collect::>(); + assert_eq!(declared, ids.len()); + assert_eq!(unique.len(), ids.len()); + (declared, unique) + } + + fn fixture_view_closure(root: &RootCommitFixture, view_tree: &str) -> HashSet { + let mut ids = HashSet::new(); + let mut pending = vec![view_tree.to_owned()]; + while let Some(tree_id) = pending.pop() { + if !ids.insert(tree_id.clone()) { + continue; + } + let tree = root + .trees + .iter() + .find(|tree| tree.id.to_string() == tree_id) + .unwrap(); + for item in &tree.tree_items { + match item.mode { + TreeItemMode::Tree => pending.push(item.id.to_string()), + TreeItemMode::Commit => {} + TreeItemMode::Blob | TreeItemMode::BlobExecutable | TreeItemMode::Link => { + ids.insert(item.id.to_string()); + } + } + } + } + ids + } + + #[tokio::test] + async fn full_pack_closure_one_to_one() { + for (spec, root_path, other_path) in [ + (":/repo", "file.txt", None), + ( + ":prefix=pre", + "pre/repo/file.txt", + Some("pre/other/file.txt"), + ), + (":exclude[::other/]", "repo/file.txt", None), + (":/repo:prefix=pre", "pre/file.txt", None), + ( + ":[:/repo:prefix=r,:/other:prefix=o]", + "r/file.txt", + Some("o/file.txt"), + ), + ] { + let fixture = Fixture::new(8, spec).await; + fixture.save_blobs().await; + fixture.catch_up(1).await; + let repo = fixture.current_repo(); + let tip = fixture.state().await.view_tip.unwrap(); + let commits = fixture + .storage + .view_storage() + .view_pack_commits(fixture.filter_pk, 0, 8) + .await + .unwrap(); + assert_eq!(commits.len(), 8, "{spec}"); + for index in [2, 7] { + let want = commits[index].object_id.clone(); + let (count, ids) = + pack_ids(repo.full_pack(vec![want]).await.unwrap(), &fixture._temp).await; + let mut expected = HashSet::new(); + for (source_index, commit) in commits.iter().take(index + 1).enumerate() { + expected.insert(commit.object_id.clone()); + let mut paths = vec![( + root_path.to_owned(), + format!("root-{source_index}").into_bytes(), + )]; + if let Some(other_path) = other_path { + paths.push(( + other_path.to_owned(), + format!("other-{source_index}").into_bytes(), + )); + } + let tree = root_tree_from_paths(HashKind::Sha1, &paths); + expected.extend(tree.trees.iter().map(|tree| tree.id.to_string())); + expected.extend(tree.blobs.iter().map(|blob| blob.id.to_string())); + assert_eq!(commit.tree_id, tree.root.id.to_string(), "{spec}"); + } + assert_eq!(count, expected.len(), "{spec}"); + assert_eq!(ids, expected, "{spec}"); + } + if spec == ":prefix=pre" { + let expected = root_tree_from_paths( + HashKind::Sha1, + &[ + ("pre/repo/file.txt".to_owned(), b"root-7".to_vec()), + ("pre/other/file.txt".to_owned(), b"other-7".to_vec()), + ], + ); + let spine_id = expected.root.id.to_string(); + let spine = mega_view_object::Entity::find_by_id(&spine_id) + .one(fixture.storage.view_storage().get_connection()) + .await + .unwrap() + .unwrap(); + assert_eq!(spine.kind, 2); + } + assert_eq!(tip, commits[7].object_id); + } + let target = "1".repeat(40); + let target_id = ObjectHash::from_hex_for_kind(HashKind::Sha1, &target).unwrap(); + let trees = (0..8) + .map(|index| { + let mut fixture = root_tree_from_paths( + HashKind::Sha1, + &[ + ( + "repo/file.txt".to_owned(), + format!("root-{index}").into_bytes(), + ), + ( + "other/file.txt".to_owned(), + format!("other-{index}").into_bytes(), + ), + ], + ); + let old_root_id = fixture.root.id; + let old_repo_id = fixture + .root + .tree_items + .iter() + .find(|item| item.name == "repo") + .unwrap() + .id; + let old_repo = fixture + .trees + .iter() + .find(|tree| tree.id == old_repo_id) + .unwrap(); + let mut repo_items = old_repo.tree_items.clone(); + repo_items.push(TreeItem::new( + TreeItemMode::Commit, + target_id, + "submodule".to_owned(), + )); + let new_repo = Tree::from_tree_items_with_kind(HashKind::Sha1, repo_items).unwrap(); + let mut root_items = fixture.root.tree_items.clone(); + root_items + .iter_mut() + .find(|item| item.name == "repo") + .unwrap() + .id = new_repo.id; + let new_root = Tree::from_tree_items_with_kind(HashKind::Sha1, root_items).unwrap(); + fixture + .trees + .retain(|tree| tree.id != old_repo_id && tree.id != old_root_id); + fixture.trees.extend([new_repo, new_root.clone()]); + fixture.root = new_root; + fixture + }) + .collect::>(); + let temp = tempfile::tempdir().unwrap(); + let storage = + test_storage_with_config(temp.path(), isolated_config(temp.path().join("config"))) + .await; + let roots = seed_linear_root_history_with_trees( + storage.view_storage().get_connection(), + HashKind::Sha1, + trees, + ) + .await; + assert_eq!( + storage + .view_storage() + .extend_root_chain(None, 100, ViewLockMode::Try) + .await + .unwrap(), + RootChainOutcome::CaughtUp + ); + let filter_pk = 5001; + let filter_id = insert_filter(&storage, filter_pk, ":/repo").await; + let fixture = Fixture { + _temp: temp, + storage, + roots, + filter_pk, + filter_id, + }; + fixture.save_blobs().await; + fixture.catch_up(1).await; + let repo = fixture.current_repo(); + let commits = fixture + .storage + .view_storage() + .view_pack_commits(filter_pk, 0, 8) + .await + .unwrap(); + assert_eq!(commits.len(), 8); + for index in [2, 7] { + let (_, ids) = pack_ids( + repo.full_pack(vec![commits[index].object_id.clone()]) + .await + .unwrap(), + &fixture._temp, + ) + .await; + assert!(!ids.contains(&target)); + let mut expected = HashSet::new(); + for (root, commit) in fixture.roots.iter().zip(&commits).take(index + 1) { + expected.insert(commit.object_id.clone()); + expected.extend(fixture_view_closure(root, &commit.tree_id)); + } + assert_eq!(ids, expected); + } + } + + #[tokio::test] + async fn incremental_pack_have_exclusion() { + let stable = (0..1500) + .map(|index| { + ( + format!("a/big/file{index:04}"), + format!("stable-{index}").into_bytes(), + ) + }) + .collect::>(); + let old_target = "1".repeat(40); + let new_target = "2".repeat(40); + let trees = (0..12) + .map(|index| { + let mut paths = stable.clone(); + let moving = if index < 4 { "a/x/m" } else { "a/y/m" }; + let content_index = if index == 4 { 3 } else { index }; + paths.push(( + moving.to_owned(), + format!("moving-{content_index}").into_bytes(), + )); + paths.push(("b/file".to_owned(), format!("other-{index}").into_bytes())); + let mut fixture = root_tree_from_paths(HashKind::Sha1, &paths); + if index >= 6 { + let target = if index == 6 { &old_target } else { &new_target }; + let target = ObjectHash::from_hex_for_kind(HashKind::Sha1, target).unwrap(); + let old_root_id = fixture.root.id; + let old_a_id = fixture + .root + .tree_items + .iter() + .find(|item| item.name == "a") + .unwrap() + .id; + let old_a = fixture + .trees + .iter() + .find(|tree| tree.id == old_a_id) + .unwrap(); + let mut a_items = old_a.tree_items.clone(); + a_items.push(TreeItem::new( + TreeItemMode::Commit, + target, + "gitlink".to_owned(), + )); + crate::jupiter::utils::converter::sort_git_tree_items(&mut a_items); + let new_a = Tree::from_tree_items_with_kind(HashKind::Sha1, a_items).unwrap(); + let mut root_items = fixture.root.tree_items.clone(); + root_items + .iter_mut() + .find(|item| item.name == "a") + .unwrap() + .id = new_a.id; + let new_root = + Tree::from_tree_items_with_kind(HashKind::Sha1, root_items).unwrap(); + fixture + .trees + .retain(|tree| tree.id != old_a_id && tree.id != old_root_id); + fixture.trees.extend([new_a, new_root.clone()]); + fixture.root = new_root; + } + fixture + }) + .collect::>(); + let fixture = Fixture::from_trees(trees, ":/a", 2, 100).await; + fixture.save_blobs().await; + fixture.catch_up(1).await; + let repo = fixture.current_repo(); + let commits = fixture + .storage + .view_storage() + .view_pack_commits(fixture.filter_pk, 0, 12) + .await + .unwrap(); + assert_eq!(commits.len(), 12); + let big_id = fixture.roots[0] + .trees + .iter() + .find(|tree| tree.tree_items.len() == 1500) + .unwrap() + .id + .to_string(); + let big_objects = fixture_view_closure(&fixture.roots[0], &big_id); + assert_eq!(big_objects.len(), 1501); + let want = commits[11].object_id.clone(); + for have_index in [1, 4, 8] { + let (_, ids) = pack_ids( + repo.incremental_pack( + vec![want.clone()], + vec![commits[have_index].object_id.clone()], + ) + .await + .unwrap(), + &fixture._temp, + ) + .await; + let mut expected = HashSet::new(); + for (root, commit) in fixture.roots.iter().zip(&commits).skip(have_index + 1) { + expected.insert(commit.object_id.clone()); + expected.insert(commit.tree_id.clone()); + let view_tree = root + .trees + .iter() + .find(|tree| tree.id.to_string() == commit.tree_id) + .unwrap(); + let moving = view_tree + .tree_items + .iter() + .find(|item| item.name == "x" || item.name == "y") + .unwrap(); + expected.insert(moving.id.to_string()); + let moving_tree = root.trees.iter().find(|tree| tree.id == moving.id).unwrap(); + expected.insert(moving_tree.tree_items[0].id.to_string()); + } + assert_eq!(ids, expected); + assert!(ids.is_disjoint(&big_objects)); + assert!(!ids.contains(&old_target) && !ids.contains(&new_target)); + let mut have_closure = HashSet::new(); + let mut want_closure = HashSet::new(); + for (index, (root, commit)) in fixture.roots.iter().zip(&commits).enumerate() { + let closure = fixture_view_closure(root, &commit.tree_id); + want_closure.insert(commit.object_id.clone()); + want_closure.extend(closure.iter().cloned()); + if index <= have_index { + have_closure.insert(commit.object_id.clone()); + have_closure.extend(closure); + } + } + have_closure.extend(ids); + assert!(have_closure.is_superset(&want_closure)); + } + let (_, full) = pack_ids( + repo.full_pack(vec![want.clone()]).await.unwrap(), + &fixture._temp, + ) + .await; + assert!(!full.contains(&old_target) && !full.contains(&new_target)); + let (_, one_have) = pack_ids( + repo.incremental_pack(vec![want.clone()], vec![commits[4].object_id.clone()]) + .await + .unwrap(), + &fixture._temp, + ) + .await; + let (_, multiple_haves) = pack_ids( + repo.incremental_pack( + vec![want.clone()], + vec![ + commits[1].object_id.clone(), + commits[4].object_id.clone(), + fixture.roots[0].commit.id.to_string(), + ], + ) + .await + .unwrap(), + &fixture._temp, + ) + .await; + assert_eq!(one_have, multiple_haves); + let other_pk = 5002; + insert_filter(&fixture.storage, other_pk, ":/b").await; + ViewProjectionService::new(fixture.storage.clone(), fixture.storage.view_metrics()) + .catch_up_one_batch(other_pk, &fixture.storage.config(), 100) + .await + .unwrap(); + let other_tip = fixture + .storage + .view_storage() + .view_reader_state(other_pk) + .await + .unwrap() + .unwrap() + .view_tip + .unwrap(); + let (_, off_chain) = pack_ids( + repo.incremental_pack( + vec![want], + vec![ + fixture.roots[0].commit.id.to_string(), + other_tip, + "f".repeat(40), + ], + ) + .await + .unwrap(), + &fixture._temp, + ) + .await; + assert_eq!(full, off_chain); + } + + #[tokio::test] + async fn pack_statement_count_independent_of_chain() { + let temp = tempfile::tempdir().unwrap(); + let (db_config, _schema) = test_db_config(temp.path()).await; + let mut config = isolated_config(temp.path().join("config")); + config.database = db_config.clone(); + config.views.batch_size = 250; + let counter = Arc::new(AtomicUsize::new(0)); + let metric_counter = counter.clone(); + let mut connection = database_connection(&db_config).await.unwrap(); + connection.set_metric_callback(move |_| { + metric_counter.fetch_add(1, Ordering::Relaxed); + }); + let storage = Storage::new_with_connection( + Arc::new(config), + Arc::new(connection), + mock_object_storage(), + ) + .await + .unwrap(); + let trees = (0..200) + .map(|index| { + root_tree_from_paths( + HashKind::Sha1, + &[ + ("a/x/y/file".to_owned(), format!("a-{index}").into_bytes()), + ( + "b/x/y/file".to_owned(), + format!("b-{}", index.min(9)).into_bytes(), + ), + ], + ) + }) + .collect::>(); + let roots = seed_linear_root_history_with_trees( + storage.view_storage().get_connection(), + HashKind::Sha1, + trees, + ) + .await; + for root in &roots { + for blob in &root.blobs { + storage + .git_service + .save_object_from_raw(Bytes::copy_from_slice(&blob.data)) + .await + .unwrap(); + } + } + assert_eq!( + storage + .view_storage() + .extend_root_chain(None, 250, ViewLockMode::Try) + .await + .unwrap(), + RootChainOutcome::CaughtUp + ); + let a = insert_filter(&storage, 5001, ":/a").await; + let b = insert_filter(&storage, 5002, ":/b").await; + let service = ViewProjectionService::new(storage.clone(), storage.view_metrics()); + for filter_pk in [5001, 5002] { + service + .catch_up_one_batch(filter_pk, &storage.config(), 250) + .await + .unwrap(); + } + let a_repo = ViewRepo::new(storage.clone(), storage.config(), service.clone(), 5001, a); + let b_repo = ViewRepo::new(storage.clone(), storage.config(), service, 5002, b); + let a_commits = storage + .view_storage() + .view_pack_commits(5001, 0, 200) + .await + .unwrap(); + let b_commits = storage + .view_storage() + .view_pack_commits(5002, 0, 200) + .await + .unwrap(); + assert_eq!(a_commits.len(), 200); + assert_eq!(b_commits.len(), 10); + counter.store(0, Ordering::Relaxed); + a_repo + .full_pack(vec![a_commits[199].object_id.clone()]) + .await + .unwrap() + .concat() + .await; + let full_a = counter.load(Ordering::Relaxed); + counter.store(0, Ordering::Relaxed); + b_repo + .full_pack(vec![b_commits[9].object_id.clone()]) + .await + .unwrap() + .concat() + .await; + let full_b = counter.load(Ordering::Relaxed); + assert_eq!(full_a, full_b); + counter.store(0, Ordering::Relaxed); + a_repo + .incremental_pack( + vec![a_commits[199].object_id.clone()], + vec![a_commits[198].object_id.clone()], + ) + .await + .unwrap() + .concat() + .await; + let short = counter.load(Ordering::Relaxed); + counter.store(0, Ordering::Relaxed); + a_repo + .incremental_pack( + vec![a_commits[199].object_id.clone()], + vec![a_commits[49].object_id.clone()], + ) + .await + .unwrap() + .concat() + .await; + assert_eq!(short, counter.load(Ordering::Relaxed)); + } + async fn insert_filter(storage: &Storage, filter_pk: i64, spec: &str) -> String { let canonical = parse_for_registration(spec).unwrap(); let filter_id = canonical.filter_id.clone(); @@ -958,10 +1780,10 @@ mod tests { .contains("view URLs are read-only") ); assert!( - matches!(repo.full_pack(vec![own.clone()]).await.err(), Some(GitError::CustomError(message)) if message.contains("view pack not implemented")) + !matches!(repo.full_pack(vec![own.clone()]).await.err(), Some(GitError::CustomError(message)) if message.contains("view pack not implemented")) ); assert!( - matches!(repo.incremental_pack(vec![own.clone()], vec![]).await.err(), Some(GitError::CustomError(message)) if message.contains("view pack not implemented")) + !matches!(repo.incremental_pack(vec![own.clone()], vec![]).await.err(), Some(GitError::CustomError(message)) if message.contains("view pack not implemented")) ); db.get_connection() diff --git a/src/jupiter/storage/view_projection_storage.rs b/src/jupiter/storage/view_projection_storage.rs index 6e1d186a..df38c170 100644 --- a/src/jupiter/storage/view_projection_storage.rs +++ b/src/jupiter/storage/view_projection_storage.rs @@ -59,7 +59,99 @@ pub(crate) struct ViewWantState { pub(crate) wants: Vec<(String, Option)>, } +pub(crate) struct ViewPackBounds { + pub(crate) want_seq: i64, + pub(crate) have_seq: i64, + pub(crate) have_tree: Option, +} + +pub(crate) struct ViewPackCommit { + pub(crate) object_id: String, + pub(crate) tree_id: String, + pub(crate) data: Vec, +} + impl ViewStorage { + pub(crate) async fn view_pack_bounds( + &self, + filter_pk: i64, + wants: &[String], + haves: &[String], + ) -> Result { + let row = self + .get_connection() + .query_one_raw(Statement::from_sql_and_values( + sea_orm::DbBackend::Postgres, + "WITH requested AS ( \ + SELECT 'want' AS role, unnest($2::text[]) AS oid \ + UNION ALL SELECT 'have', unnest($3::text[]) \ + ), matched AS ( \ + SELECT r.role, m.seq_from FROM requested r \ + JOIN mega_view_commit_map m ON m.filter_pk = $1 AND m.view_commit = r.oid \ + ), bounds AS ( \ + SELECT max(seq_from) FILTER (WHERE role = 'want') AS want_seq, \ + coalesce(max(seq_from) FILTER (WHERE role = 'have'), 0) AS have_seq \ + FROM matched \ + ) \ + SELECT b.want_seq, b.have_seq, m.view_tree AS have_tree \ + FROM bounds b LEFT JOIN mega_view_commit_map m \ + ON m.filter_pk = $1 AND m.seq_from = b.have_seq", + [ + Value::from(filter_pk), + Value::from(wants.to_vec()), + Value::from(haves.to_vec()), + ], + )) + .await? + .ok_or_else(|| MegaError::Other("view pack bounds returned no row".to_owned()))?; + let want_seq: Option = row.try_get("", "want_seq")?; + Ok(ViewPackBounds { + want_seq: want_seq.ok_or_else(|| { + MegaError::Other("view pack want is outside the view chain".to_owned()) + })?, + have_seq: row.try_get("", "have_seq")?, + have_tree: row.try_get("", "have_tree")?, + }) + } + + pub(crate) async fn view_pack_commits( + &self, + filter_pk: i64, + have_seq: i64, + want_seq: i64, + ) -> Result, MegaError> { + let rows = self + .get_connection() + .query_all_raw(Statement::from_sql_and_values( + sea_orm::DbBackend::Postgres, + "SELECT m.view_commit, m.view_tree, o.data \ + FROM mega_view_commit_map m \ + LEFT JOIN mega_view_object o ON o.object_id = m.view_commit AND o.kind = 1 \ + WHERE m.filter_pk = $1 AND m.seq_from > $2 AND m.seq_from <= $3 \ + AND m.view_commit IS NOT NULL \ + ORDER BY m.seq_from", + [ + Value::from(filter_pk), + Value::from(have_seq), + Value::from(want_seq), + ], + )) + .await?; + rows.into_iter() + .map(|row| { + let object_id: String = row.try_get("", "view_commit")?; + let data: Option> = row.try_get("", "data")?; + Ok(ViewPackCommit { + tree_id: row.try_get("", "view_tree")?, + data: data.ok_or_else(|| { + MegaError::Other(format!("view commit object missing: {object_id}")) + })?, + object_id, + }) + }) + .collect() + } + pub(crate) async fn view_reader_state( &self, filter_pk: i64, diff --git a/tests/fixtures/views/josh_proxy/README.md b/tests/fixtures/views/josh_proxy/README.md new file mode 100644 index 00000000..0bd5a727 --- /dev/null +++ b/tests/fixtures/views/josh_proxy/README.md @@ -0,0 +1,11 @@ +# Josh proxy clone expectations + +Source: [`josh-project/josh` at `e6dfb95e`](https://github.com/josh-project/josh/tree/e6dfb95e/tests/proxy), MIT license. These fixtures transcribe only expected `git log --graph --pretty=%s` and tree output from `clone_subtree.t`, `clone_subsubtree.t`, and `clone_prefix.t`; no Josh implementation code is copied. + +| Upstream proxy filter | Mega2 view filter | Fixture | +| --- | --- | --- | +| `:/sub1` | `:/project/hp22-subtree/sub1` | `subtree.*` | +| `:/sub1/subsub` | `:/project/hp22-subsubtree/sub1/subsub` | `subsubtree.*` | +| `:prefix=pre` | `:prefix=pre` composed with `:/project/hp22-prefix` | `prefix.*` | + +The upstream `git-tree-pretty` output includes box drawing characters and blob contents. The `.tree` files instead record its path portion as `git ls-tree -r --name-only HEAD` emits it, preserving path names and ordering byte for byte. The `git log` lines are copied verbatim. Mega2 seeds each case under `/project/hp22-` through trunk and projects it from the root, so commit IDs, remote URL, branch name (`main` instead of `master`), and tree drawing are intentionally excluded from the comparison. These three pinned `.t` files contain clone output but no later fetch segment. The `.fetch.*` files are a Mega2-only extension: one additional `add file3` commit in the selected path, followed by fetch and fast-forward. They are not claimed to be upstream expected output. diff --git a/tests/fixtures/views/josh_proxy/prefix.fetch.log b/tests/fixtures/views/josh_proxy/prefix.fetch.log new file mode 100644 index 00000000..4f2852f2 --- /dev/null +++ b/tests/fixtures/views/josh_proxy/prefix.fetch.log @@ -0,0 +1,3 @@ +* add file3 +* add file2 +* add file1 diff --git a/tests/fixtures/views/josh_proxy/prefix.fetch.tree b/tests/fixtures/views/josh_proxy/prefix.fetch.tree new file mode 100644 index 00000000..574ea90b --- /dev/null +++ b/tests/fixtures/views/josh_proxy/prefix.fetch.tree @@ -0,0 +1,3 @@ +pre/sub1/file1 +pre/sub1/file3 +pre/sub2/file2 diff --git a/tests/fixtures/views/josh_proxy/prefix.log b/tests/fixtures/views/josh_proxy/prefix.log new file mode 100644 index 00000000..4b0cc267 --- /dev/null +++ b/tests/fixtures/views/josh_proxy/prefix.log @@ -0,0 +1,2 @@ +* add file2 +* add file1 diff --git a/tests/fixtures/views/josh_proxy/prefix.tree b/tests/fixtures/views/josh_proxy/prefix.tree new file mode 100644 index 00000000..73e6aacd --- /dev/null +++ b/tests/fixtures/views/josh_proxy/prefix.tree @@ -0,0 +1,2 @@ +pre/sub1/file1 +pre/sub2/file2 diff --git a/tests/fixtures/views/josh_proxy/subsubtree.fetch.log b/tests/fixtures/views/josh_proxy/subsubtree.fetch.log new file mode 100644 index 00000000..7cab1146 --- /dev/null +++ b/tests/fixtures/views/josh_proxy/subsubtree.fetch.log @@ -0,0 +1,2 @@ +* add file3 +* add file1 diff --git a/tests/fixtures/views/josh_proxy/subsubtree.fetch.tree b/tests/fixtures/views/josh_proxy/subsubtree.fetch.tree new file mode 100644 index 00000000..f6cdd87a --- /dev/null +++ b/tests/fixtures/views/josh_proxy/subsubtree.fetch.tree @@ -0,0 +1,2 @@ +file1 +file3 diff --git a/tests/fixtures/views/josh_proxy/subsubtree.log b/tests/fixtures/views/josh_proxy/subsubtree.log new file mode 100644 index 00000000..44b1175c --- /dev/null +++ b/tests/fixtures/views/josh_proxy/subsubtree.log @@ -0,0 +1 @@ +* add file1 diff --git a/tests/fixtures/views/josh_proxy/subsubtree.tree b/tests/fixtures/views/josh_proxy/subsubtree.tree new file mode 100644 index 00000000..e2129701 --- /dev/null +++ b/tests/fixtures/views/josh_proxy/subsubtree.tree @@ -0,0 +1 @@ +file1 diff --git a/tests/fixtures/views/josh_proxy/subtree.fetch.log b/tests/fixtures/views/josh_proxy/subtree.fetch.log new file mode 100644 index 00000000..7cab1146 --- /dev/null +++ b/tests/fixtures/views/josh_proxy/subtree.fetch.log @@ -0,0 +1,2 @@ +* add file3 +* add file1 diff --git a/tests/fixtures/views/josh_proxy/subtree.fetch.tree b/tests/fixtures/views/josh_proxy/subtree.fetch.tree new file mode 100644 index 00000000..f6cdd87a --- /dev/null +++ b/tests/fixtures/views/josh_proxy/subtree.fetch.tree @@ -0,0 +1,2 @@ +file1 +file3 diff --git a/tests/fixtures/views/josh_proxy/subtree.log b/tests/fixtures/views/josh_proxy/subtree.log new file mode 100644 index 00000000..44b1175c --- /dev/null +++ b/tests/fixtures/views/josh_proxy/subtree.log @@ -0,0 +1 @@ +* add file1 diff --git a/tests/fixtures/views/josh_proxy/subtree.tree b/tests/fixtures/views/josh_proxy/subtree.tree new file mode 100644 index 00000000..e2129701 --- /dev/null +++ b/tests/fixtures/views/josh_proxy/subtree.tree @@ -0,0 +1 @@ +file1 diff --git a/tests/integration_git_cli.rs b/tests/integration_git_cli.rs index 5a5a3d57..a17a06ff 100644 --- a/tests/integration_git_cli.rs +++ b/tests/integration_git_cli.rs @@ -5938,6 +5938,518 @@ fn layer3_http_boot() -> (GitCliEnv, ServiceProcess, u16, PathBuf) { (env, service, port, err) } +fn hp22_http_boot() -> (GitCliEnv, ServiceProcess, u16, PathBuf) { + assert!(!git_cli::git_cli_skip_requested()); + git_cli::require_git_cli_runner(); + let env = GitCliEnv::new(); + let mut extra = trunk_boot_env().to_vec(); + extra.extend([ + ("MEGA_VIEWS__ENABLED", "true"), + ("MEGA_VIEWS__ALLOW_ANONYMOUS_REGISTER", "true"), + ]); + let (service, port, _, err) = boot_service_http_with_env(&env, Some("off"), None, &extra); + (env, service, port, err) +} + +fn hp22_register(port: u16, name: &str, spec: &str) -> String { + let response = reqwest::blocking::Client::builder() + .no_proxy() + .build() + .unwrap() + .post(git_cli::mega2_host_http_url( + port, + "/api/v1/views?wait=true", + )) + .json(&serde_json::json!({"name": name, "filter_spec": spec})) + .send() + .unwrap(); + let status = response.status(); + let body: serde_json::Value = response.json().unwrap(); + assert!(status.is_success(), "register {spec}: {status} {body}"); + assert_eq!(body["data"]["ready"], true, "{body}"); + body["data"]["filter_id"].as_str().unwrap().to_owned() +} + +fn hp22_wait_lag_zero(port: u16, filter_id: &str) { + let client = reqwest::blocking::Client::builder() + .no_proxy() + .build() + .unwrap(); + let path = format!("/api/v1/views/{filter_id}"); + let deadline = Instant::now() + Duration::from_secs(60); + loop { + let response = client + .get(git_cli::mega2_host_http_url(port, &path)) + .send() + .unwrap(); + if response.status().is_success() { + let body: serde_json::Value = response.json().unwrap(); + if body["data"]["lag_commits"] == 0 { + return; + } + } + assert!( + Instant::now() < deadline, + "view {filter_id} did not catch up" + ); + sleep(Duration::from_millis(250)); + } +} + +fn hp22_source_commit(case_dir: &Path, relative_path: &str, message: &str) { + let path = case_dir.join("hp22-source").join(relative_path); + fs::create_dir_all(path.parent().unwrap()).unwrap(); + fs::write(&path, b"contents1\n").unwrap(); + git_ok_no_auth(case_dir, &["-C", "hp22-source", "add", relative_path]); + git_ok_no_auth(case_dir, &["-C", "hp22-source", "commit", "-m", message]); + git_cli::assert_git_success(&trunk_push(case_dir, "hp22-source"), "HP-22 trunk push"); +} + +fn hp22_fetch_with_trace(case_dir: &Path, clone: &str, v0: bool) { + let old = git_cli::git_cli_no_auth(case_dir, &["-C", clone, "rev-parse", "origin/main"]); + git_cli::assert_git_success(&old, "HP-22 old view tip"); + let old = String::from_utf8(old.stdout).unwrap().trim().to_owned(); + let discovered = Command::new("docker") + .args([ + "ps", + "-q", + "--filter", + "label=com.docker.compose.project=mega2-it", + "--filter", + "label=com.docker.compose.service=git-cli", + "--filter", + "status=running", + ]) + .output() + .unwrap(); + assert!(discovered.status.success(), "{discovered:?}"); + let container = String::from_utf8(discovered.stdout).unwrap(); + let container = container.trim(); + assert!(!container.is_empty() && !container.contains('\n')); + let relative = case_dir.strip_prefix(git_cli::git_cli_workdir()).unwrap(); + let workdir = format!("/work/{}", relative.display()); + let mut command = Command::new("timeout"); + command.args([ + "-k", + "5", + "105", + "docker", + "exec", + "-w", + &workdir, + "-e", + "GIT_TRACE_PACKET=1", + "-e", + "GIT_TERMINAL_PROMPT=0", + "-e", + "GIT_ASKPASS=true", + "-e", + "GIT_CONFIG_COUNT=1", + "-e", + "GIT_CONFIG_KEY_0=credential.helper", + "-e", + "GIT_CONFIG_VALUE_0=", + container, + "git", + "-C", + clone, + ]); + if v0 { + command.args(["-c", "protocol.version=0"]); + } + let output = command.arg("fetch").output().unwrap(); + git_cli::assert_git_success(&output, "HP-22 traced fetch"); + let trace = String::from_utf8_lossy(&output.stderr); + assert!( + trace.lines().any(|line| line.contains("> have ")), + "fetch must send have: {trace}" + ); + let range = format!("{old}..origin/main"); + let advanced = + git_cli::git_cli_no_auth(case_dir, &["-C", clone, "rev-list", "--count", &range]); + git_cli::assert_git_success(&advanced, "HP-22 view tip advancement"); + assert_eq!(advanced.stdout, b"1\n"); +} + +fn hp22_assert_view_clone(case_dir: &Path, port: u16, filter_id: &str, case: &str) { + let path = if case == "prefix" { + "/.view/hp22-prefix@1.git".to_owned() + } else { + format!("/.filter/{filter_id}.git") + }; + let url = git_cli::mega2_http_url(port, &path); + let log_expected = + fs::read_to_string(format!("tests/fixtures/views/josh_proxy/{case}.log")).unwrap(); + let tree_expected = + fs::read_to_string(format!("tests/fixtures/views/josh_proxy/{case}.tree")).unwrap(); + for (label, v0) in [("v0", true), ("v2", false)] { + let clone = format!("hp22-{case}-{label}"); + let mut args = Vec::new(); + if v0 { + args.extend(["-c", "protocol.version=0"]); + } + args.extend(["clone", &url, &clone]); + git_cli::assert_git_success( + &git_cli::git_cli_no_auth(case_dir, &args), + "HP-22 view clone", + ); + let log = + git_cli::git_cli_no_auth(case_dir, &["-C", &clone, "log", "--graph", "--pretty=%s"]); + git_cli::assert_git_success(&log, "HP-22 view log"); + assert_eq!(log.stdout, log_expected.as_bytes()); + let tree = git_cli::git_cli_no_auth( + case_dir, + &["-C", &clone, "ls-tree", "-r", "--name-only", "HEAD"], + ); + git_cli::assert_git_success(&tree, "HP-22 view tree"); + assert_eq!(tree.stdout, tree_expected.as_bytes()); + git_cli::assert_git_success( + &git_cli::git_cli_no_auth(case_dir, &["-C", &clone, "fsck", "--strict"]), + "HP-22 view fsck", + ); + } +} + +#[test] +fn integration_git_cli_view_pack_http_josh_proxy() { + let (env, mut service, port, err) = hp22_http_boot(); + let case_dir = env.case_dir.as_path(); + git_ok_no_auth( + case_dir, + &["clone", &trunk_subpath_url(port, "/project"), "hp22-source"], + ); + configure_git_identity_no_auth(case_dir, "hp22-source"); + for (case, first) in [ + ("subtree", "sub1/file1"), + ("subsubtree", "sub1/subsub/file1"), + ("prefix", "sub1/file1"), + ] { + hp22_source_commit(case_dir, &format!("hp22-{case}/{first}"), "add file1"); + hp22_source_commit(case_dir, &format!("hp22-{case}/sub2/file2"), "add file2"); + let spec = match case { + "subtree" => ":/project/hp22-subtree/sub1", + "subsubtree" => ":/project/hp22-subsubtree/sub1/subsub", + _ => ":/project/hp22-prefix:prefix=pre", + }; + let filter_id = hp22_register(port, &format!("hp22-{case}"), spec); + hp22_wait_lag_zero(port, &filter_id); + hp22_assert_view_clone(case_dir, port, &filter_id, case); + let third = if case == "subsubtree" { + "sub1/subsub/file3" + } else { + "sub1/file3" + }; + hp22_source_commit(case_dir, &format!("hp22-{case}/{third}"), "add file3"); + hp22_wait_lag_zero(port, &filter_id); + let expected_log = + fs::read(format!("tests/fixtures/views/josh_proxy/{case}.fetch.log")).unwrap(); + let expected_tree = + fs::read(format!("tests/fixtures/views/josh_proxy/{case}.fetch.tree")).unwrap(); + for (label, v0) in [("v0", true), ("v2", false)] { + let clone = format!("hp22-{case}-{label}"); + let mut args = vec!["-C", clone.as_str()]; + if v0 { + args.extend(["-c", "protocol.version=0"]); + } + args.push("fetch"); + git_cli::assert_git_success( + &git_cli::git_cli_no_auth(case_dir, &args), + "HP-22 Josh fetch", + ); + git_cli::assert_git_success( + &git_cli::git_cli_no_auth( + case_dir, + &["-C", &clone, "merge", "--ff-only", "@{upstream}"], + ), + "HP-22 Josh fast forward", + ); + let log = git_cli::git_cli_no_auth( + case_dir, + &["-C", &clone, "log", "--graph", "--pretty=%s"], + ); + git_cli::assert_git_success(&log, "HP-22 Josh fetched log"); + assert_eq!(log.stdout, expected_log); + let tree = git_cli::git_cli_no_auth( + case_dir, + &["-C", &clone, "ls-tree", "-r", "--name-only", "HEAD"], + ); + git_cli::assert_git_success(&tree, "HP-22 Josh fetched tree"); + assert_eq!(tree.stdout, expected_tree); + git_cli::assert_git_success( + &git_cli::git_cli_no_auth(case_dir, &["-C", &clone, "fsck", "--strict"]), + "HP-22 Josh fetched fsck", + ); + } + } + assert!( + service + .shutdown_via_sigint(Duration::from_secs(10)) + .success(), + "{}", + read_log(&err) + ); +} + +#[test] +fn integration_git_cli_view_pack_http_fetch_after_trunk_push() { + let (env, mut service, port, err) = hp22_http_boot(); + let case_dir = env.case_dir.as_path(); + git_ok_no_auth( + case_dir, + &["clone", &trunk_subpath_url(port, "/project"), "hp22-source"], + ); + configure_git_identity_no_auth(case_dir, "hp22-source"); + hp22_source_commit(case_dir, "hp22-fetch/file1", "add file1"); + let id = hp22_register(port, "hp22-fetch", ":/project/hp22-fetch"); + hp22_wait_lag_zero(port, &id); + let url = git_cli::mega2_http_url(port, &format!("/.filter/{id}.git")); + for (clone, v0) in [("hp22-fetch-v0", true), ("hp22-fetch-v2", false)] { + let mut args = Vec::new(); + if v0 { + args.extend(["-c", "protocol.version=0"]); + } + args.extend(["clone", &url, clone]); + git_cli::assert_git_success( + &git_cli::git_cli_no_auth(case_dir, &args), + "HP-22 initial clone", + ); + } + git_ok_no_auth( + case_dir, + &[ + "clone", + &git_cli::mega2_host_http_url(port, "/project/hp22-fetch.git"), + "hp22-subpath", + ], + ); + configure_git_identity_no_auth(case_dir, "hp22-subpath"); + fs::write(case_dir.join("hp22-subpath/file2"), b"contents2\n").unwrap(); + git_ok_no_auth(case_dir, &["-C", "hp22-subpath", "add", "file2"]); + git_ok_no_auth( + case_dir, + &["-C", "hp22-subpath", "commit", "-m", "add file2"], + ); + git_cli::assert_git_success( + &trunk_push(case_dir, "hp22-subpath"), + "HP-22 N=1 subpath push", + ); + hp22_wait_lag_zero(port, &id); + for (clone, v0) in [("hp22-fetch-v0", true), ("hp22-fetch-v2", false)] { + hp22_fetch_with_trace(case_dir, clone, v0); + git_cli::assert_git_success( + &git_cli::git_cli_no_auth( + case_dir, + &["-C", clone, "merge", "--ff-only", "@{upstream}"], + ), + "HP-22 fast forward", + ); + let tree = git_cli::git_cli_no_auth( + case_dir, + &["-C", clone, "ls-tree", "-r", "--name-only", "HEAD"], + ); + git_cli::assert_git_success(&tree, "HP-22 fetched tree"); + assert_eq!(tree.stdout, b"file1\nfile2\n"); + } + for file in ["file3", "file4"] { + let path = case_dir.join("hp22-subpath").join(file); + fs::write(path, format!("{file}\n")).unwrap(); + git_ok_no_auth(case_dir, &["-C", "hp22-subpath", "add", file]); + git_ok_no_auth( + case_dir, + &["-C", "hp22-subpath", "commit", "-m", &format!("add {file}")], + ); + } + git_cli::assert_git_success( + &trunk_push(case_dir, "hp22-subpath"), + "HP-22 N>1 trunk push", + ); + hp22_wait_lag_zero(port, &id); + for (clone, v0) in [("hp22-fetch-v0", true), ("hp22-fetch-v2", false)] { + hp22_fetch_with_trace(case_dir, clone, v0); + } + assert!( + service + .shutdown_via_sigint(Duration::from_secs(10)) + .success(), + "{}", + read_log(&err) + ); +} + +#[test] +fn integration_git_cli_view_pack_http_clone_ready_empty() { + let (_env, mut service, port, err) = hp22_http_boot(); + let id = hp22_register(port, "hp22-empty", ":/project/hp22-absent"); + hp22_wait_lag_zero(port, &id); + let path = format!("/.filter/{id}.git"); + let response = reqwest::blocking::Client::builder() + .no_proxy() + .build() + .unwrap() + .post(git_cli::mega2_host_http_url( + port, + &format!("{path}/git-upload-pack"), + )) + .header("Git-Protocol", "version=2") + .header("Content-Type", "application/x-git-upload-pack-request") + .body("0014command=ls-refs\n0001000csymrefs\n0000") + .send() + .unwrap(); + assert_eq!(response.status(), reqwest::StatusCode::OK); + assert_eq!(response.bytes().unwrap().as_ref(), b"0000"); + let url = git_cli::mega2_http_url(port, &path); + let env = _env; + for (clone, v0) in [("hp22-empty-v0", true), ("hp22-empty-v2", false)] { + let mut args = Vec::new(); + if v0 { + args.extend(["-c", "protocol.version=0"]); + } + args.extend(["clone", &url, clone]); + let result = git_cli::git_cli_no_auth(&env.case_dir, &args); + git_cli::assert_git_success(&result, "HP-22 empty view clone"); + assert!( + String::from_utf8_lossy(&result.stderr) + .contains("You appear to have cloned an empty repository") + ); + let refs = git_cli::git_cli_no_auth(&env.case_dir, &["-C", clone, "for-each-ref"]); + git_cli::assert_git_success(&refs, "HP-22 empty view refs"); + assert!(refs.stdout.is_empty()); + } + assert!( + service + .shutdown_via_sigint(Duration::from_secs(10)) + .success(), + "{}", + read_log(&err) + ); +} + +#[test] +fn integration_git_cli_view_pack_http_clone_depth_filter_400() { + let (env, mut service, port, err) = hp22_http_boot(); + let case_dir = env.case_dir.as_path(); + git_ok_no_auth( + case_dir, + &["clone", &trunk_subpath_url(port, "/project"), "hp22-source"], + ); + configure_git_identity_no_auth(case_dir, "hp22-source"); + hp22_source_commit(case_dir, "hp22-depth/seed.txt", "seed depth"); + let id = hp22_register(port, "hp22-depth", ":/project/hp22-depth"); + hp22_wait_lag_zero(port, &id); + let url = git_cli::mega2_http_url(port, &format!("/.filter/{id}.git")); + for (name, options) in [ + ( + "v0-depth", + vec!["-c", "protocol.version=0", "clone", "--depth", "1"], + ), + ("v2-depth", vec!["clone", "--depth", "1"]), + ("v2-filter", vec!["clone", "--filter=blob:none"]), + ] { + let mut args = options; + args.extend([url.as_str(), name]); + let result = git_cli::git_cli_no_auth(case_dir, &args); + assert!(!result.status.success(), "{result:?}"); + assert!( + String::from_utf8_lossy(&result.stderr).contains("RPC failed; HTTP 400"), + "{result:?}" + ); + } + for (name, v0) in [("v0-control", true), ("v2-control", false)] { + let mut args = Vec::new(); + if v0 { + args.extend(["-c", "protocol.version=0"]); + } + args.extend(["clone", &url, name]); + git_cli::assert_git_success( + &git_cli::git_cli_no_auth(case_dir, &args), + "HP-22 control clone", + ); + } + assert!( + service + .shutdown_via_sigint(Duration::from_secs(10)) + .success(), + "{}", + read_log(&err) + ); +} + +#[test] +fn integration_git_cli_view_pack_http_clone_gitlink() { + let (env, mut service, port, err) = hp22_http_boot(); + let case_dir = env.case_dir.as_path(); + git_ok_no_auth( + case_dir, + &["clone", &trunk_subpath_url(port, "/project"), "hp22-source"], + ); + configure_git_identity_no_auth(case_dir, "hp22-source"); + let gitlink = "1".repeat(40); + git_ok_no_auth( + case_dir, + &[ + "-C", + "hp22-source", + "update-index", + "--add", + "--cacheinfo", + &format!("160000,{gitlink},hp22-gitlink/submodule"), + ], + ); + let tree = git_stdout_no_auth(case_dir, &["-C", "hp22-source", "write-tree"]); + let parent = git_stdout_no_auth(case_dir, &["-C", "hp22-source", "rev-parse", "HEAD"]); + let commit = git_stdout_no_auth( + case_dir, + &[ + "-C", + "hp22-source", + "commit-tree", + &tree, + "-p", + &parent, + "-m", + "add gitlink", + ], + ); + git_ok_no_auth( + case_dir, + &[ + "-C", + "hp22-source", + "push", + "--no-thin", + "origin", + &format!("{commit}:refs/heads/main"), + ], + ); + let id = hp22_register(port, "hp22-gitlink", ":/project/hp22-gitlink"); + hp22_wait_lag_zero(port, &id); + let url = git_cli::mega2_http_url(port, &format!("/.filter/{id}.git")); + for (clone, v0) in [("hp22-gitlink-v0", true), ("hp22-gitlink-v2", false)] { + let mut args = Vec::new(); + if v0 { + args.extend(["-c", "protocol.version=0"]); + } + args.extend(["clone", &url, clone]); + git_cli::assert_git_success( + &git_cli::git_cli_no_auth(case_dir, &args), + "HP-22 gitlink clone", + ); + let tree = git_cli::git_cli_no_auth(case_dir, &["-C", clone, "ls-tree", "-r", "HEAD"]); + git_cli::assert_git_success(&tree, "HP-22 gitlink tree"); + assert_eq!( + tree.stdout, + format!("160000 commit {gitlink}\tsubmodule\n").as_bytes() + ); + } + assert!( + service + .shutdown_via_sigint(Duration::from_secs(10)) + .success(), + "{}", + read_log(&err) + ); +} + fn layer3_http_execute(env: &GitCliEnv, statement: Statement) { with_runtime(async { let db = Database::connect(&env.database.db_url).await.unwrap(); diff --git a/tests/integration_git_ssh.rs b/tests/integration_git_ssh.rs index 842af848..98df5ded 100644 --- a/tests/integration_git_ssh.rs +++ b/tests/integration_git_ssh.rs @@ -3619,6 +3619,265 @@ fn integration_git_ssh_view_layer3_ls_remote() { ); } +fn hp22_ssh_boot() -> (GitSshEnv, ServiceProcess, u16, u16, PathBuf) { + assert!(!git_cli::git_cli_skip_requested()); + git_cli::require_git_cli_runner(); + let env = GitSshEnv::new(); + git_cli::generate_client_ed25519(&env.ssh_dir.join("client_ed25519")); + let extra = [ + ("MEGA_VIEWS__ENABLED", "true"), + ("MEGA_VIEWS__ALLOW_ANONYMOUS_REGISTER", "true"), + ("MEGA_MONOREPO__PUSH_POLICY", "trunk"), + ("MEGA_GIT__PUSH_AUTH", "none"), + ("MEGA_GIT__SSH_RECEIVE_PACK", "false"), + ("MEGA_GIT__ANONYMOUS_ACCESS", "true"), + ]; + let (mut first, _, _, _, first_err) = boot_service_multi_with_extra(&env, "off", None, &extra); + assert!( + first.shutdown_via_sigint(Duration::from_secs(10)).success(), + "{}", + read_log(&first_err) + ); + drop(first); + let (service, http_port, ssh_port, _, err) = + boot_service_multi_with_extra(&env, "off", None, &extra); + git_cli::write_known_hosts_via_keyscan(&env.ssh_dir.join("known_hosts"), ssh_port); + (env, service, http_port, ssh_port, err) +} + +fn hp22_ssh_register(port: u16, name: &str, spec: &str) -> String { + let response = reqwest::blocking::Client::builder() + .no_proxy() + .build() + .unwrap() + .post(git_cli::mega2_host_http_url( + port, + "/api/v1/views?wait=true", + )) + .json(&serde_json::json!({"name": name, "filter_spec": spec})) + .send() + .unwrap(); + let status = response.status(); + let body: serde_json::Value = response.json().unwrap(); + assert!(status.is_success(), "register {spec}: {status} {body}"); + assert_eq!(body["data"]["ready"], true, "{body}"); + body["data"]["filter_id"].as_str().unwrap().to_owned() +} + +fn hp22_ssh_wait_lag_zero(port: u16, filter_id: &str) { + let client = reqwest::blocking::Client::builder() + .no_proxy() + .build() + .unwrap(); + let deadline = Instant::now() + Duration::from_secs(60); + loop { + let path = format!("/api/v1/views/{filter_id}"); + let response = client + .get(git_cli::mega2_host_http_url(port, &path)) + .send() + .unwrap(); + if response.status().is_success() { + let body: serde_json::Value = response.json().unwrap(); + if body["data"]["lag_commits"] == 0 { + return; + } + } + assert!( + Instant::now() < deadline, + "view {filter_id} did not catch up" + ); + sleep(Duration::from_millis(250)); + } +} + +fn hp22_ssh_host_git(case_dir: &Path, args: &[&str]) -> String { + let output = git_host_ssh(case_dir, "", args); + git_cli::assert_git_success(&output, "HP-22 host HTTP git"); + String::from_utf8(output.stdout).unwrap() +} + +fn hp22_ssh_source_commit(case_dir: &Path, relative_path: &str, message: &str) { + let path = case_dir.join("hp22-source").join(relative_path); + fs::create_dir_all(path.parent().unwrap()).unwrap(); + fs::write(&path, b"contents1\n").unwrap(); + hp22_ssh_host_git(case_dir, &["-C", "hp22-source", "add", relative_path]); + hp22_ssh_host_git(case_dir, &["-C", "hp22-source", "commit", "-m", message]); + hp22_ssh_host_git( + case_dir, + &[ + "-C", + "hp22-source", + "push", + "--no-thin", + "origin", + "HEAD:refs/heads/main", + ], + ); +} + +#[test] +fn integration_git_ssh_view_pack_ssh_josh_proxy() { + let (env, mut service, http_port, ssh_port, err) = hp22_ssh_boot(); + let case_dir = env.case_dir.as_path(); + let project_url = git_cli::mega2_host_http_url(http_port, "/project"); + hp22_ssh_host_git(case_dir, &["clone", &project_url, "hp22-source"]); + hp22_ssh_host_git( + case_dir, + &["-C", "hp22-source", "config", "user.name", "HP22"], + ); + hp22_ssh_host_git( + case_dir, + &[ + "-C", + "hp22-source", + "config", + "user.email", + "hp22@example.invalid", + ], + ); + let ssh_command = git_cli::git_ssh_command(case_dir, ssh_port); + for (case, first) in [ + ("subtree", "sub1/file1"), + ("subsubtree", "sub1/subsub/file1"), + ("prefix", "sub1/file1"), + ] { + hp22_ssh_source_commit(case_dir, &format!("hp22-{case}/{first}"), "add file1"); + hp22_ssh_source_commit(case_dir, &format!("hp22-{case}/sub2/file2"), "add file2"); + let spec = match case { + "subtree" => ":/project/hp22-subtree/sub1", + "subsubtree" => ":/project/hp22-subsubtree/sub1/subsub", + _ => ":/project/hp22-prefix:prefix=pre", + }; + let id = hp22_ssh_register(http_port, &format!("hp22-{case}"), spec); + hp22_ssh_wait_lag_zero(http_port, &id); + let path = if case == "prefix" { + ".view/hp22-prefix@1.git".to_owned() + } else { + format!(".filter/{id}.git") + }; + let url = format!("{}{}", git_cli::mega2_ssh_repo_url(ssh_port, "git"), path); + let expected_log = fs::read(format!("tests/fixtures/views/josh_proxy/{case}.log")).unwrap(); + let expected_tree = + fs::read(format!("tests/fixtures/views/josh_proxy/{case}.tree")).unwrap(); + for (label, v0) in [("v0", true), ("v2", false)] { + let clone = format!("hp22-{case}-{label}"); + let mut args = Vec::new(); + if v0 { + args.extend(["-c", "protocol.version=0"]); + } + args.extend(["clone", &url, &clone]); + let output = git_cli::git_cli_ssh(case_dir, &ssh_command, &args); + git_cli::assert_git_success(&output, "HP-22 SSH clone"); + let log = git_cli::git_cli_ssh( + case_dir, + &ssh_command, + &["-C", &clone, "log", "--graph", "--pretty=%s"], + ); + git_cli::assert_git_success(&log, "HP-22 SSH log"); + assert_eq!(log.stdout, expected_log); + let tree = git_cli::git_cli_ssh( + case_dir, + &ssh_command, + &["-C", &clone, "ls-tree", "-r", "--name-only", "HEAD"], + ); + git_cli::assert_git_success(&tree, "HP-22 SSH tree"); + assert_eq!(tree.stdout, expected_tree); + git_cli::assert_git_success( + &git_cli::git_cli_ssh(case_dir, &ssh_command, &["-C", &clone, "fsck", "--strict"]), + "HP-22 SSH fsck", + ); + } + let third = if case == "subsubtree" { + "sub1/subsub/file3" + } else { + "sub1/file3" + }; + hp22_ssh_source_commit(case_dir, &format!("hp22-{case}/{third}"), "add file3"); + hp22_ssh_wait_lag_zero(http_port, &id); + let expected_log = + fs::read(format!("tests/fixtures/views/josh_proxy/{case}.fetch.log")).unwrap(); + let expected_tree = + fs::read(format!("tests/fixtures/views/josh_proxy/{case}.fetch.tree")).unwrap(); + for (label, v0) in [("v0", true), ("v2", false)] { + let clone = format!("hp22-{case}-{label}"); + let mut args = vec!["-C", clone.as_str()]; + if v0 { + args.extend(["-c", "protocol.version=0"]); + } + args.push("fetch"); + let fetch = git_cli::git_cli_ssh(case_dir, &ssh_command, &args); + git_cli::assert_git_success(&fetch, "HP-22 SSH Josh fetch"); + let merge = git_cli::git_cli_ssh( + case_dir, + &ssh_command, + &["-C", &clone, "merge", "--ff-only", "@{upstream}"], + ); + git_cli::assert_git_success(&merge, "HP-22 SSH Josh fast forward"); + let log = git_cli::git_cli_ssh( + case_dir, + &ssh_command, + &["-C", &clone, "log", "--graph", "--pretty=%s"], + ); + git_cli::assert_git_success(&log, "HP-22 SSH Josh fetched log"); + assert_eq!(log.stdout, expected_log); + let tree = git_cli::git_cli_ssh( + case_dir, + &ssh_command, + &["-C", &clone, "ls-tree", "-r", "--name-only", "HEAD"], + ); + git_cli::assert_git_success(&tree, "HP-22 SSH Josh fetched tree"); + assert_eq!(tree.stdout, expected_tree); + git_cli::assert_git_success( + &git_cli::git_cli_ssh(case_dir, &ssh_command, &["-C", &clone, "fsck", "--strict"]), + "HP-22 SSH Josh fetched fsck", + ); + } + } + assert!( + service + .shutdown_via_sigint(Duration::from_secs(10)) + .success(), + "{}", + read_log(&err) + ); +} + +#[test] +fn integration_git_ssh_view_pack_ssh_ready_empty() { + let (env, mut service, http_port, ssh_port, err) = hp22_ssh_boot(); + let id = hp22_ssh_register(http_port, "hp22-empty", ":/project/hp22-absent"); + hp22_ssh_wait_lag_zero(http_port, &id); + let url = format!( + "{}.filter/{id}.git", + git_cli::mega2_ssh_repo_url(ssh_port, "git") + ); + let ssh_command = git_cli::git_ssh_command(&env.case_dir, ssh_port); + for (clone, v0) in [("hp22-empty-v0", true), ("hp22-empty-v2", false)] { + let mut args = Vec::new(); + if v0 { + args.extend(["-c", "protocol.version=0"]); + } + args.extend(["clone", &url, clone]); + let output = git_cli::git_cli_ssh(&env.case_dir, &ssh_command, &args); + git_cli::assert_git_success(&output, "HP-22 SSH empty clone"); + assert!( + String::from_utf8_lossy(&output.stderr) + .contains("You appear to have cloned an empty repository") + ); + let refs = + git_cli::git_cli_ssh(&env.case_dir, &ssh_command, &["-C", clone, "for-each-ref"]); + git_cli::assert_git_success(&refs, "HP-22 SSH empty refs"); + assert!(refs.stdout.is_empty()); + } + assert!( + service + .shutdown_via_sigint(Duration::from_secs(10)) + .success(), + "{}", + read_log(&err) + ); +} + fn layer3_pkt(payload: &str) -> Vec { format!("{:04x}{payload}", payload.len() + 4).into_bytes() } From 08bde899f5c08b6c28578dffce02aa0bdcdf8b99 Mon Sep 17 00:00:00 2001 From: Eli Ma Date: Thu, 8 Oct 2026 01:26:13 +0800 Subject: [PATCH 2/5] v0.42.22: HP-23 view pack preflight Signed-off-by: Eli Ma --- Cargo.lock | 2 +- Cargo.toml | 2 +- docs/plan/plan-20261002.md | 38 +- docs/refactoring/protocol.md | 8 + src/ceres/pack/view_repo.rs | 807 ++++++++++++++++-- src/jupiter/service/view_metrics.rs | 10 + .../storage/view_projection_storage.rs | 131 ++- tests/integration_git_cli.rs | 237 +++++ tests/integration_git_ssh.rs | 457 ++++++++++ tests/integration_views.rs | 486 +++++++++++ 10 files changed, 2055 insertions(+), 123 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index 4853c9bd..55b411c8 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4172,7 +4172,7 @@ checksum = "7ebb8d8732c6a6df3d8f032a82911cfc747e00efb95cc46e8d0acd5b5b88570c" [[package]] name = "mega2" -version = "0.42.21" +version = "0.42.22" dependencies = [ "anyhow", "async-recursion", diff --git a/Cargo.toml b/Cargo.toml index 85e176d0..3484527d 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "mega2" -version = "0.42.21" +version = "0.42.22" edition = "2024" license = "MIT OR Apache-2.0" description = "Monorepo engine and Git hosting server for AI agents: trunk-based storage, Git Smart HTTP, Git LFS, OCI registry, and Agent Session Capture" diff --git a/docs/plan/plan-20261002.md b/docs/plan/plan-20261002.md index b32ee375..6deda447 100644 --- a/docs/plan/plan-20261002.md +++ b/docs/plan/plan-20261002.md @@ -2,7 +2,7 @@ **模板版本：** v2.1；依据 [`plan-template.md`](plan-template.md)。源码核对日期 2026-10-02；核对时 tip 为 `c9ae6df`（`v0.41.34`）。成稿时另一会话已发布 `fb5ac71`（`v0.41.35: BB-43 ART input validation case`），只改了 `Cargo.toml`、`Cargo.lock`、`docs/deploy-trunk.md`、`docs/plan/plan-20261001.md`、`scripts/artifacts_smoke_storage_only.sh`，不影响本文的源码锚点。工作区仍有该会话的未提交改动，本计划不触碰；每张卡开工时按 ER-01、ER-12 重新读取权威版本。 -**计划状态：** **执行中**。主链已完成 HP-32 与 HP-34；HP-30 承载发布为 `v0.42.7`；HP-13 承载发布为 `v0.42.8`；HP-31 承载发布为 `v0.42.9`；HP-14 由 `v0.42.10` 承载；HP-16 由 `v0.42.11` 承载；HP-17 由 `v0.42.12` 承载；HP-34 由 `v0.42.13` 承载；HP-32 由 `v0.42.14` 承载；HP-15 由 `v0.42.15` 承载；HP-33 由 `v0.42.16` 承载；HP-18 由 `v0.42.17` 承载；HP-19 由 `v0.42.18` 承载；HP-20 由 `v0.42.19` 承载；HP-21 由 `v0.42.20` 承载；HP-22 由 `v0.42.21` 承载。按剩余依赖顺序从 HP-23 继续。Docker Build / Actions 按发布者指令不等待、不查询。 +**计划状态：** **执行中**。主链已完成 HP-32 与 HP-34；HP-30 承载发布为 `v0.42.7`；HP-13 承载发布为 `v0.42.8`；HP-31 承载发布为 `v0.42.9`；HP-14 由 `v0.42.10` 承载；HP-16 由 `v0.42.11` 承载；HP-17 由 `v0.42.12` 承载；HP-34 由 `v0.42.13` 承载；HP-32 由 `v0.42.14` 承载；HP-15 由 `v0.42.15` 承载；HP-33 由 `v0.42.16` 承载；HP-18 由 `v0.42.17` 承载；HP-19 由 `v0.42.18` 承载；HP-20 由 `v0.42.19` 承载；HP-21 由 `v0.42.20` 承载；HP-22 由 `v0.42.21` 承载；HP-23 由 `v0.42.22` 承载。按剩余依赖顺序从 HP-24 继续。Docker Build / Actions 按发布者指令不等待、不查询。 **设计契约：** [`docs/refactoring/history-projection.md`](../refactoring/history-projection.md)（下称「设计」，v0.2，经 Codex 五轮审计，见设计附录 F）。本文用「设计 §x.y」引用其条款；设计节号冻结（ADR-HP-01）。 @@ -5071,7 +5071,7 @@ ER-04 强制门（不计入 VER）：本卡改 `src/contract/git_protocol/mod.rs **Task type:** `implementation` -**Lifecycle / Acceptance:** `pending` / 空 +**Lifecycle / Acceptance:** `done` / `complete`（7/7 精确 Verification、8/8 AC、ER-04 A 组只读 smoke；Claude HP23-R3 `VERDICT: PASS` 与 Codex HP23-R4 PASS；nightly 格式、Clippy、完整测试 19 个目标及两个构建门通过；承载发布 `v0.42.22`；Docker Build / Actions 按发布者指令不等待、不查询） **Description:** 实现 `ViewRepo::prepare_pack`（设计 §6.3 方法表 `prepare_pack` 行），并以黑盒锁定打包预检在视图 URL 上的线上应答（设计 §6.1 错误契约表「打包预检失败」行；设计 §7.4 错误契约第 5、6 项）。唯一的行为轴是「发包前的 fail-closed 预检及其线上应答」。本卡接收原属 HP-20 的「want 不属于本视图」黑盒：该错误由 HP-19 的 `check_wants_and_ready` 产生，自 HP-20 起经视图 URL 可达。范围： - 单快照复查：`prepare_pack` 先用一条 SQL 读出设计 §4.6「读者的单快照规则」第 4 项所列各项（就绪状态与水位、`root_chain_halted`、各 want 是否仍命中 `(filter_pk, view_commit)`、seq 区间内的视图提交 id），查询写在 `view_projection_storage.rs`。以下情形返回 `MegaError::ViewUnavailable`，不报 `not our ref`： @@ -5128,26 +5128,26 @@ ER-04 强制门（不计入 VER）：本卡改 `src/contract/git_protocol/mod.rs **Acceptance criteria:** -- [ ] 三种打包预检失败的服务端原始应答，在 HTTP 与 SSH、v0 与 v2 上都与设计 §6.1 错误契约表「打包预检失败」行一致（ERR 消息取设计 §6.3 为该触发规定的文本），对以下触发都成立：want 不属于本视图（设计 §7.4 错误契约第 5 项的五种请求形态）、闭包含 L0 不自洽 tree（第 6 项的四种请求形态）、闭包缺 tree 对象。 -- [ ] git 客户端的诊断与设计 §6.1 错误契约表「打包预检失败」行的「git 的表现」列一致（以 `remote error: ` 退出），对以下请求都成立：经 HTTP 与 SSH 以 v2 `git fetch <视图 URL> ` 请求不属于本视图的提交、经 HTTP 与 SSH 以 v0 与 v2 `git clone` 闭包含 L0 不自洽 tree 的视图。 -- [ ] 对 want 合法的视图（包括闭包含 L0 不自洽 tree 的视图），v2 不带 `done`、have 全不在视图链上的 NAK 轮照常返回以 flush 结束的 `acknowledgments` 加 `NAK`（设计 §7.4 错误契约第 5 项对照组、第 6 项末句）。 -- [ ] `view_pack_tree_mismatch_total` 的增量恰等于 L0 tree 不自洽导致的拒绝次数（经 `GET /api/v1/views/metrics` 观察）。 -- [ ] L0 tree 不自洽或闭包缺 tree 对象导致的拒绝之后，视图保持就绪（设计 §6.3「L0 tree 校验」、方法表 `get_trees_by_hashes` 行）。 -- [ ] L0 tree 不自洽或闭包缺 tree 对象导致的每次拒绝，恰发出一条字段符合 Description「计数」一条对该情形所列事件规格的 ERROR 级 `tracing` 事件（ADR-HP-05）。 -- [ ] 同一 ViewRepo 实例上 `check_wants_and_ready` 通过之后，`prepare_pack` 的单一快照读到视图未就绪、根链停追或任一 want 已不在映射中时，返回 `MegaError::ViewUnavailable`（不是 `ViewPackRejected`），其 reason 取设计 §6.1 错误契约表「暂不可用」行规定的值（依据设计 §4.6「读者的单快照规则」第 4 项）。 -- [ ] 一次发包轮（`prepare_pack` 加一个打包方法，读尽 pack 流）的 SQL 语句符合单快照规则的语句口径（设计 §4.6「读者的单快照规则」第 4 项「用一条 SQL 读出」「之后的打包只读对象，不再读 commit_map」、§6.3「发包与回收交错」、方法表 `full_pack` 行）。 +- [x] 三种打包预检失败的服务端原始应答，在 HTTP 与 SSH、v0 与 v2 上都与设计 §6.1 错误契约表「打包预检失败」行一致（ERR 消息取设计 §6.3 为该触发规定的文本），对以下触发都成立：want 不属于本视图（设计 §7.4 错误契约第 5 项的五种请求形态）、闭包含 L0 不自洽 tree（第 6 项的四种请求形态）、闭包缺 tree 对象。 +- [x] git 客户端的诊断与设计 §6.1 错误契约表「打包预检失败」行的「git 的表现」列一致（以 `remote error: ` 退出），对以下请求都成立：经 HTTP 与 SSH 以 v2 `git fetch <视图 URL> ` 请求不属于本视图的提交、经 HTTP 与 SSH 以 v0 与 v2 `git clone` 闭包含 L0 不自洽 tree 的视图。 +- [x] 对 want 合法的视图（包括闭包含 L0 不自洽 tree 的视图），v2 不带 `done`、have 全不在视图链上的 NAK 轮照常返回以 flush 结束的 `acknowledgments` 加 `NAK`（设计 §7.4 错误契约第 5 项对照组、第 6 项末句）。 +- [x] `view_pack_tree_mismatch_total` 的增量恰等于 L0 tree 不自洽导致的拒绝次数（经 `GET /api/v1/views/metrics` 观察）。 +- [x] L0 tree 不自洽或闭包缺 tree 对象导致的拒绝之后，视图保持就绪（设计 §6.3「L0 tree 校验」、方法表 `get_trees_by_hashes` 行）。 +- [x] L0 tree 不自洽或闭包缺 tree 对象导致的每次拒绝，恰发出一条字段符合 Description「计数」一条对该情形所列事件规格的 ERROR 级 `tracing` 事件（ADR-HP-05）。 +- [x] 同一 ViewRepo 实例上 `check_wants_and_ready` 通过之后，`prepare_pack` 的单一快照读到视图未就绪、根链停追或任一 want 已不在映射中时，返回 `MegaError::ViewUnavailable`（不是 `ViewPackRejected`），其 reason 取设计 §6.1 错误契约表「暂不可用」行规定的值（依据设计 §4.6「读者的单快照规则」第 4 项）。 +- [x] 一次发包轮（`prepare_pack` 加一个打包方法，读尽 pack 流）的 SQL 语句符合单快照规则的语句口径（设计 §4.6「读者的单快照规则」第 4 项「用一条 SQL 读出」「之后的打包只读对象，不再读 commit_map」、§6.3「发包与回收交错」、方法表 `full_pack` 行）。 **Verification:** 第 1–3 行是 lib 用例，第 4 行是不需要 git-cli runner 的进程级用例，都需要测试栈（ER-01）。第 5、6 行另需 git-cli runner，命令内先拉起 compose 的 `git` profile。这两行的新用例开头按 `tests/integration_git_ssh.rs:545-549` 的写法断言 `MEGA2_IT_SKIP_GIT_CLI` 不为 1，再调用 `git_cli::require_git_cli_runner()`；不使用 `tests/integration_git_cli.rs:1871-1874` 那种打印 SKIP 后返回的分支，跳过不算通过。执行位置与设计 §7.4 一致（`docs/refactoring/history-projection.md:1316`、`:1374`）：第 5 行的 SSH 原始请求在 git-cli 容器内执行 `ssh`；第 5、6 行的 git 客户端都在 git-cli 容器 runner 中执行，经 SSH 用 `git_cli::git_cli_ssh`，经 HTTP 用 `git_cli::git_cli_no_auth`。本卡不采用 `integration_git_ssh` 既有 storage-only 用例的宿主回环写法（宿主 `ssh` 与宿主 git 经 `127.0.0.1`）。容器 runner 的 git 版本固定为 `PINNED_GIT_CLI_VERSION`（`tests/common/git_cli.rs:29`、`:416-450`），验收时不设 `MEGA2_IT_ALLOW_HOST_GIT`。宿主 git 只用于构造夹具（`git mktree`、`git commit-tree` 与 trunk 推送），不参与比对；宿主需要 `git` 与 `timeout`。ER-04 A 组（不计入 VER）：按计划的按卡映射跑 本计划 ER-04 A 组表中该卡那一行（`scripts/git_protocol_smoke_storage_only.sh`，compose 项目 `mega2-trunk`）。 -- [ ] `source .env.test && cargo test -p mega2 --lib 'ceres::pack::view_repo::tests::prepare_pack_rejects_l0_and_missing_trees' -- --exact`（new；AC 1 的消息部分、AC 5、AC 6。用例为同步 `#[test]`：整个用例（建库、夹具、清表重建与预检调用）都在捕获闭包内构建的同一个 current-thread runtime 中执行，写法同 `src/jupiter/service/storage_event_emitter.rs:1277-1285`；捕获 helper 按 `:1204-1253` 的写法新增在 `view_repo.rs` 的 `tests` 模块中，捕获期间保持 `_pin_registry` 注册，订阅器带 `.with_ansi(false)`，不照搬 `un25_freeze.rs` 的简版 helper。`test_db_connection` + `apply_migrations` 建库，schema 由该连接的 metric 回调持有（`src/jupiter/tests.rs:85-87`），不另绑守卫。用 `view_test_fixtures` 写线性根历史，以 `extend_root_chain` 建好根链，四个过滤器行写为冷启动状态（`projected_seq = 0`，`ready_seq` 为 NULL，`warming_since` 非空）；夹具 helper 写在 `view_repo.rs` 的 `tests` 模块中。两个 L0 tree 按 receive-pack 的真实路径写入：原始 tree 字节构造成 `Entry`（`obj_type` 为 Tree，`hash` 为原始字节的 sha1），经 `process_entry`（`src/jupiter/utils/converter.rs:133-143`）与 `convert_to_mega_model(EntryMeta::default())`（`:106-117`）得到 `MegaObjectModel::Tree(model)`，把 `model.commit_id` 设为引入它的夹具根提交的 id，再以 `vec![model.into_active_model()]` 经 `StorageConnector::batch_save_model`（`src/jupiter/storage/base_storage.rs:165`）写入，与 `MonoService::save_entry` 的 tree 分支相同（`src/jupiter/service/mono_service.rs:175-190`、`:227-229`）。T_mode 含一个 `100664` 条目，T_gbk 含一个 GBK 编码的文件名（字节 `C4 E3 BA C3`，即「你好」）；先断言两行都满足 `hash(Tree, sub_trees) ≠ tree_id`。T_mode 位于 `a/mode`，是视图 `:/a/mode` 的结果 tree；T_gbk 位于 `b/gbk`，是视图 `:/b/gbk:prefix=p` 中被 Prefix 包住的 T。视图 `:/c` 的子目录 `c/d` 只含一个文件名独有的 blob `hp23-only-in-c-d.txt`；对照视图 `:/e` 的闭包只含自洽 tree。夹具前提（设计 §7.4「L0 tree 不自洽」测试项断言中的「投影照常推进，清表重建后哈希不变」，L0 tree 的读取由 HP-29、`catch_up` 由 HP-11 承接；前提不成立按 Out of scope 对应条处理）：四个视图都 `catch_up` 到就绪；记下各视图 commit_map 中按 seq 排列的视图提交 id；然后清表重建：删除四个过滤器的 `mega_view_commit_map` 行与 `mega_view_object_ref` 行，以及本 schema 的全部 `mega_view_object` 行，状态列复位为冷启动，再 `catch_up` 到就绪；各视图的视图提交 id 序列与重建前逐字节相同。预检调用：每个视图新建一个 ViewRepo，want = [tip]、have 为空，依次调用 `check_wants_and_ready` 与 `prepare_pack`。捕获的日志按行计数。先依次检查 `:/a/mode`、`:/b/gbk:prefix=p`、`:/e`；随后断言夹具写入的所有 tree 中 id 等于 `c/d` 的只有路径 `c/d` 这一处，再删去 `c/d` 的 `mega_tree` 行，最后检查 `:/c`。断言：`:/a/mode` 与 `:/b/gbk:prefix=p` 返回 `ViewPackRejected`，消息逐字节等于 `view pack aborted: tree does not match its stored entries`；`:/c` 返回 `ViewPackRejected`，消息逐字节等于 `view pack aborted: tree is missing`；每次拒绝，级别为 ERROR 且同时含 `filter_id=<该 filter_id>` 与 `tree_id=<该 tree_id>` 两个字段的日志行恰为 1 行，其它级别的行不计；`:/a/mode` 与 `:/b/gbk:prefix=p` 的这一行另含 `metric=view_pack_tree_mismatch_total`；`:/c` 的这一行不含 `metric=`；`:/e` 返回 `Ok(())`，不出现含 `filter_id=<:/e 的 filter_id>` 的 ERROR 级行；每次调用前后 `mega_view_filter` 中该视图的行逐列相等） -- [ ] `source .env.test && cargo test -p mega2 --lib 'ceres::pack::view_repo::tests::prepare_pack_rechecks_single_snapshot' -- --exact`（new；AC 7。四种情形与对照组各用一个独立的 `test_db_connection` schema，写同一夹具：视图 `:/a` 已就绪，有三个视图提交 C1、C2、C3。新建两个 ViewRepo 实例 R1、R2，want 都为 [C3]，各调用一次 `check_wants_and_ready`，断言 `Ok(())`；随后用自动提交的语句改库；然后 R1 以 have 为空、R2 以 have = [C1] 调用 `prepare_pack`。四种改库：① 回收：`UPDATE mega_view_filter SET ready_seq = NULL, projected_seq = 0, warming_since = NULL WHERE id = `，再删除该过滤器的 `mega_view_commit_map` 行与 `mega_view_object_ref` 行（设计 §4.6「视图删除即回收」第 2、3 步）；② 只把 `ready_seq` 置为 NULL；③ 只删除 C3 所在的那一行 `mega_view_commit_map`；④ 用 `view_test_fixtures` 把 `main@/` CAS 为无关历史，调用 `extend_root_chain` 判出不连续，使 `root_chain_halted` 为真。四种情形下两次调用都返回 `MegaError::ViewUnavailable`，不是 `ViewPackRejected`，filter_id 为本视图；reason 在 ①、②、③ 中为 `warming up`，在 ④ 中为 `root chain halted`。对照：不改库时两次调用都返回 `Ok(())`） -- [ ] `source .env.test && cargo test -p mega2 --lib 'ceres::pack::view_repo::tests::commit_map_read_only_by_prepare_pack' -- --exact`（new；AC 8。用 `test_db_config` 取得 `(db_config, _schema)`，守卫持有到用例结束；经 `database_connection(&db_config)`（`src/jupiter/storage/init.rs:19-26`，建连时执行迁移）自建连接，在这条连接上设 `set_metric_callback`，记录每条语句的 SQL 文本；以 `Storage::new_with_connection(config, Arc::new(conn), mock_object_storage())` 组装 ViewRepo 所用的存储，夹具的 blob 写入这份存储的对象存储；不在 `test_db_connection` 返回的连接上替换回调（GC-HP-10）。视图 `:/a` 已就绪，有三个视图提交 C1、C2、C3。夹具（含 `catch_up`）写完之后，每轮与对照组开始前都清空记录，只统计从该轮第一个调用起的语句；各轮都不调用 `check_wants_and_ready`。两轮各用一个新实例：want = [C3]、have 为空，调用 `prepare_pack` 后调用 `full_pack`；want = [C3]、have = [C1]，调用 `prepare_pack` 后调用 `incremental_pack`。每轮断言两项单快照口径：从 `prepare_pack` 开始到它返回，SQL 文本含 `mega_view_filter` 的语句恰为 1 条，它同时含 `mega_view_commit_map`，即就绪状态与 commit_map 由同一条语句读出（设计 §4.6「读者的单快照规则」第 4 项）；读尽 pack 流之后，该轮 SQL 文本含 `mega_view_commit_map` 的语句仍恰为这 1 条。对照组：另用一个新实例不调用 `prepare_pack`，直接调用 `full_pack`，不记录任何含 `mega_view_commit_map` 的语句；该调用的返回值不在本行断言） -- [ ] `source .env.test && cargo test -p mega2 --test integration_views view_precheck_http_ -- --test-threads=1`（new；前缀过滤，`test result` 中 passed ≥ 3；AC 1 的 HTTP 部分、AC 3、AC 4、AC 5。服务以 `service http` 启动：trunk、sha1、`MEGA_VIEWS__ENABLED=true`、`MEGA_VIEWS__ALLOW_ANONYMOUS_REGISTER=true`、`push_auth = none`、`anonymous_access = true`。夹具按 ADR-HP-06 由测试进程直接写服务所用的测试库：用 git-internal 的 `*_with_kind` API 构造对象，写入 `mega_tree`、`mega_commit` 后 CAS `main@/`，再经 `POST /api/v1/views?wait=true` 逐个注册视图，每个等到就绪再注册下一个（`max_concurrent_cold_starts` 缺省为 2）。根历史在服务初始化得到的 `main@/` 之后追加两个提交。夹具写入之前，先经测试库读出 `main@/` 的提交 id，记为 R0。R1 的父提交为 R0，在 `hp23/mode`、`hp23/gbk`、`hp23/ok`、`hp23/miss/d` 放自洽 tree，其中 `hp23/miss/d` 只含一个文件名独有的 blob `hp23-only-in-miss-d.txt`。R2 的父提交为 R1，把 `hp23/mode`、`hp23/gbk` 分别换成含 `100664` 条目的 T_mode 与含 GBK 文件名的 T_gbk。CAS 以 R0 为期望值，把 `main@/` 置为 R2（R1 若无父，而根链表已接入 R0，按设计 §3.3 第 3 步「c 无父、根链表非空」判为不连续，视图一直 503）。L0 tree 的行按 receive-pack 的落库字节写入：`tree_id` 为原始字节的 sha1，`sub_trees` 为 git-internal `Tree::from_bytes(原始字节, id)` 再 `to_data()` 的结果，与 `process_entry`、`into_mega_model` 落库的字节相同（`src/jupiter/utils/converter.rs:133-143`、`:260-271`）。原始请求用 `reqwest` blocking 发到 `/.filter/.git/git-upload-pack`，v2 带 `Git-Protocol: version=2`，取状态码、`Content-Type` 与响应体字节；v0 have 非空的请求在首个 want 行带 `multi_ack_detailed`。`view_precheck_http_want_not_ours`：视图 `:/hp23/ok`，oid 取 R2，即 `main@/` 的提交 id（在 `mega_commit` 中，不在本视图的 commit_map 中）；五种请求形态为 v0 have 为空、v0 have = [视图 tip]、v2 带 `done`、v2 不带 `done` 且 have 为 R1（不在视图链上）、v2 不带 `done` 且 have = [视图 tip]；每个都返回 200，`Content-Type` 为 `application/x-git-upload-pack-result`，响应体恰为一个载荷为 `ERR upload-pack: not our ref \n` 的 pkt-line，不含 `acknowledgments`、`NAK`、`ACK`、`packfile` 与 `PACK`；对照组 want = [视图 tip]、have 为 R1、不带 `done` 的 v2 请求返回 `0014acknowledgments\n0008NAK\n0000`；用例前后 `view_pack_tree_mismatch_total` 相等。`view_precheck_http_l0_mismatch`：视图 `:/hp23/mode` 与 `:/hp23/gbk:prefix=p` 各有视图提交 V1、V2；对每个视图依次发送四种请求形态（v0 have 为空；v0 have = [V1]；v2 带 `done`；v2 不带 `done`、have = [V1]），want 都为 [V2]，每个都返回 200、同上 `Content-Type`，响应体恰为一个载荷为 `ERR view pack aborted: tree does not match its stored entries\n` 的 pkt-line，不含上述五个标记；每个请求前后各取一次 `GET /api/v1/views/metrics`，响应 `data` 中的 `view_pack_tree_mismatch_total` 恰加 1；每次拒绝之后 v0 与 v2 的 `GET info/refs?service=git-upload-pack` 返回 200；NAK 轮：want = [V2]、have 为 R1、不带 `done`，返回 `0014acknowledgments\n0008NAK\n0000`，计数不变，随后同一 want 带 `done` 的请求得到上述 ERR，计数加 1。`view_precheck_http_missing_tree`：视图 `:/hp23/miss` 就绪后，先断言夹具写入的所有 tree 中 id 等于 `hp23/miss/d` 的只有这一处路径，再删去该 tree 的 `mega_tree` 行；v0 have 为空与 v2 带 `done` 各一次，返回 200、同上 `Content-Type`，响应体恰为一个载荷为 `ERR view pack aborted: tree is missing\n` 的 pkt-line，不含上述五个标记；`view_pack_tree_mismatch_total` 不变；之后 v0 与 v2 的 `GET info/refs?service=git-upload-pack` 返回 200） -- [ ] `source .env.test && docker compose -p mega2-it -f docker/docker-compose.test.yml --profile git up -d --wait && cargo test -p mega2 --test integration_git_ssh integration_git_ssh_view_precheck_ -- --test-threads=1`（new；前缀过滤，`test result` 中 passed ≥ 3；AC 1、2、3 的 SSH 部分。三个用例各自建库启动服务：`service multi http ssh`，用 `boot_service_multi`（`tests/integration_git_ssh.rs:1727-1784`）的同形变体启动，该变体多接收一组环境变量，监听地址取 `git_cli::service_listen_host()`；先迁移启动再正式启动，写法同 `boot_storage_only_ssh`（`:523-541`），但 known_hosts 不用 `write_known_hosts_via_host_keyscan`。配置为 trunk、sha1、`MEGA_CEDAR__ENFORCEMENT=off`、`MEGA_VIEWS__ENABLED=true`、`MEGA_VIEWS__ALLOW_ANONYMOUS_REGISTER=true`、`push_auth = none`、`anonymous_access = true`、`ssh_receive_pack = false`。SSH 客户端在 git-cli 容器内：启动之前以 `git_cli::generate_client_ed25519` 在 `env.ssh_dir` 下生成 `client_ed25519`，它只为满足 `git_cli::git_ssh_command` 中的 `-i`，不写入数据库（storage-only 下经 none 认证）；正式启动之后以 `git_cli::write_known_hosts_via_keyscan(&env.ssh_dir.join("known_hosts"), ssh_port)` 在容器内写 known_hosts，该 helper 先为容器的运行 UID 补好 passwd 项（`tests/common/git_cli.rs:1194`），容器内的 `ssh` 依赖这一步；SSH 命令串取 `git_cli::git_ssh_command(&env.case_dir, ssh_port)`，下称 ``；SSH 视图 URL 为 `git_cli::mega2_ssh_repo_url(ssh_port, "git")` 加 `.filter/.git`。夹具经 HTTP 腿的 trunk 推送写入，写法同 `seed_project_foo`（`tests/integration_git_cli.rs:1784-1809`）：宿主 git 经 `git_host_ssh`（`:480-501`）执行，访问 `http://127.0.0.1:/project`（HTTP URL 不受 `GIT_SSH_COMMAND` 影响）；克隆 `/project`，先推一个提交建立 `hp23-mode`、`hp23-gbk`、`hp23-ok`、`hp23-miss/d`（都是自洽 tree，`hp23-miss/d` 只含一个文件名独有的 blob `hp23-only-in-miss-d.txt`）；再用 `git mktree` 生成含 `100664` 条目的 T_mode，以及以 C 风格引号路径 `"\304\343\272\303.txt"`（GBK 编码的「你好」）生成含 GBK 文件名的 T_gbk，用 `git ls-tree HEAD` 的输出替换这两个条目后经 `git mktree` 生成新的 `/project` 树，`git commit-tree -p HEAD` 后推送。推送之后、注册视图之前，经该用例的测试库连接（`TestDatabase`，`:37-60`）读出 T_mode、T_gbk 两行 `mega_tree`，先断言夹具前提：两行都满足 `ObjectHash::from_type_and_data_for_kind(HashKind::Sha1, ObjectType::Tree, sub_trees) ≠ tree_id`；这一断言早于任何 ERR 断言。视图 `:/project/hp23-mode`、`:/project/hp23-gbk:prefix=p`、`:/project/hp23-ok`、`:/project/hp23-miss` 经 `POST /api/v1/views?wait=true` 逐个注册并就绪。开工核对（ER-02）若发现推送被拒，或落库后 T_mode / T_gbk 的行不满足 `hash(Tree, sub_trees) ≠ tree_id`，改为按 ADR-HP-06 直接写库（写法同第 4 行），并按 ER-03 修订本卡。`integration_git_ssh_view_precheck_raw`：原始请求在 git-cli 容器内执行。`` 由用例内的 `docker ps -q --filter label=com.docker.compose.project=mega2-it --filter label=com.docker.compose.service=git-cli --filter status=running` 取得（与 `tests/common/git_cli.rs:349-378` 的私有 helper 写法相同，不走 Compose 的 `exec` 子命令）。每个请求在宿主执行 `timeout -k 5 45 docker exec -i sh -c <脚本>`，脚本由 ``、v2 时追加的 `-o SetEnv=GIT_PROTOCOL=version=2`、`git@` 与远端命令 `git-upload-pack '/.filter/.git'` 组成；stdin 为预先构造的请求字节，经 `docker exec -i` 送入容器；取 stdout 字节与退出码。顺序固定：先测 want 不属于本视图（视图 `:/project/hp23-ok`，oid 取 `main@/` 的提交 id）的五种请求形态，再测两个 L0 视图各自的四种请求形态（形态定义同第 4 行），再测对照组；这些断言全部完成之后，用 `git ls-tree -r -t` 对两个推送提交断言 `hp23-miss/d` 的 tree id 只出现在这一处路径，再经该用例的测试库连接删去这棵 tree 的 `mega_tree` 行，最后对 `:/project/hp23-miss` 发 v0 have 为空与 v2 带 `done` 两种请求。三种触发的 stdout 在宣告之后恰为一个 ERR pkt-line，载荷与第 4 行对应的触发相同，退出码 1。对照组：want 合法、have 不在视图链上、不带 `done` 的 v2 请求（对 `:/project/hp23-ok` 与 `:/project/hp23-mode` 各一次），stdout 在宣告之后恰为 `0014acknowledgments\n0008NAK\n0000`，退出码 0。`integration_git_ssh_view_precheck_git_client_not_our_ref`：经 `git_cli::git_cli_ssh(&env.case_dir, &, …)` 在容器内先 `git init` 一个空仓库，再在其中执行 `git -c protocol.version=2 fetch `，退出非零，stderr 含 `remote error: upload-pack: not our ref `。`integration_git_ssh_view_precheck_git_client_l0_clone`：经 `git_cli::git_cli_ssh` 在容器内对两个 L0 视图各执行 `git -c protocol.version=0 clone` 与 `git -c protocol.version=2 clone`，都退出非零，stderr 含 `remote error:` 与该 tree_id，不含 `did not send all necessary objects`；对照组：`:/project/hp23-ok` 的两种 clone 都成功） -- [ ] `source .env.test && docker compose -p mega2-it -f docker/docker-compose.test.yml --profile git up -d --wait && cargo test -p mega2 --test integration_git_cli integration_git_cli_view_precheck_ -- --test-threads=1`（new；前缀过滤，`test result` 中 passed ≥ 2；AC 2 的 HTTP 部分。`boot_service_http_with_env`（`tests/integration_git_cli.rs:308`）以 `trunk_boot_env()`（`:1658-1664`）加 `MEGA_VIEWS__ENABLED=true`、`MEGA_VIEWS__ALLOW_ANONYMOUS_REGISTER=true` 启动；夹具与四个视图的构造同第 5 行，推送用 `trunk_host_git`（`:1669-1697`）访问 `http://127.0.0.1:/project`；夹具前提（T_mode、T_gbk 两行都满足 `hash(Tree, sub_trees) ≠ tree_id`）同样在推送之后、注册视图之前断言，早于任何 ERR 断言，测试库连接取 `env.database.db_url`；开工核对的回落条件与做法同第 5 行；本行不删除任何行。客户端命令在 git-cli runner 中经 `git_cli::git_cli_no_auth`（`tests/common/git_cli.rs:603`）执行，视图 URL 用 `git_cli::mega2_http_url`（`:310`）拼出。`integration_git_cli_view_precheck_not_our_ref`：在空仓库中执行 `git -c protocol.version=2 fetch `，退出非零，stderr 含 `remote error: upload-pack: not our ref `。v0 不做客户端用例：v0 宣告不含 `allow-tip-sha1-in-want` 与 `allow-reachable-sha1-in-want`（`src/ceres/protocol/smart.rs:54-61`），git 在本地拒绝请求未宣告的对象（`git/git@v2.49.1 fetch-pack.c:661-662`、`:2266`），请求到不了服务端；v0 的服务端应答由第 4、5 行覆盖。`integration_git_cli_view_precheck_l0_clone`：对两个 L0 视图各执行 `git -c protocol.version=0 clone` 与 `git -c protocol.version=2 clone`，都退出非零，stderr 含 `remote error:` 与该 tree_id，不含 `did not send all necessary objects`；对照组：`:/project/hp23-ok` 的两种 clone 都成功） -- [ ] `source .env.test && bash -c 'p=docs/refactoring/protocol.md; s=target/tmp/hp23/precheck-section.md; chk(){ if rg -q -- "$2" "$1"; then echo "OK: $2"; else rc=$?; echo "FAIL: $2 (rg exit $rc)"; exit 1; fi; }; chkf(){ if rg -q -F -- "$2" "$1"; then echo "OK: $2"; else rc=$?; echo "FAIL: $2 (rg exit $rc)"; exit 1; fi; }; chk "$p" "^### 视图打包预检（HP-23）$"; chk "$p" "^\| 视图 fetch 非本视图提交（not our ref） \| cargo:integration_git_cli_view_precheck_not_our_ref \| cargo:integration_git_ssh_view_precheck_git_client_not_our_ref \|"; chk "$p" "^\| 视图 clone 遇 L0 tree 不自洽 \| cargo:integration_git_cli_view_precheck_l0_clone \| cargo:integration_git_ssh_view_precheck_git_client_l0_clone \|"; mkdir -p target/tmp/hp23 || exit 1; awk "/^### 视图打包预检（HP-23）\$/{f=1;next} /^##/{f=0} f" "$p" > "$s" || { echo "FAIL: awk"; exit 1; }; test -s "$s" || { echo "FAIL: empty section"; exit 1; }; for k in "warming up" "root chain halted" "does not match its stored entries" "is missing" view_pack_tree_mismatch_total NAK mega_view_commit_map history-projection.md; do chkf "$s" "$k"; done'`（文档同步：输出全部为 `OK`，退出码 0。先核对小节标题与场景覆盖表的两行，两行的 HTTP、SSH 两列都是本卡的 `cargo:` 单元格；再按标题切出该小节（到下一个 `##` 或 `###` 标题为止），小节非空，并逐个以 `rg -F` 核对 Docs and compatibility impact 要求写入的八个关键词：两个 reason、两条拒绝消息的措辞、计数名、`NAK`（NAK 轮不预检）、`mega_view_commit_map`（打包方法不再读取）与指向设计 §6.1 的链接文件名。开工前小节标题与两行都零命中；任一项不命中或 `rg` 执行失败都判 FAIL） +- [x] `source .env.test && cargo test -p mega2 --lib 'ceres::pack::view_repo::tests::prepare_pack_rejects_l0_and_missing_trees' -- --exact`（new；AC 1 的消息部分、AC 5、AC 6。用例为同步 `#[test]`：整个用例（建库、夹具、清表重建与预检调用）都在捕获闭包内构建的同一个 current-thread runtime 中执行，写法同 `src/jupiter/service/storage_event_emitter.rs:1277-1285`；捕获 helper 按 `:1204-1253` 的写法新增在 `view_repo.rs` 的 `tests` 模块中，捕获期间保持 `_pin_registry` 注册，订阅器带 `.with_ansi(false)`，不照搬 `un25_freeze.rs` 的简版 helper。`test_db_connection` + `apply_migrations` 建库，schema 由该连接的 metric 回调持有（`src/jupiter/tests.rs:85-87`），不另绑守卫。用 `view_test_fixtures` 写线性根历史，以 `extend_root_chain` 建好根链，四个过滤器行写为冷启动状态（`projected_seq = 0`，`ready_seq` 为 NULL，`warming_since` 非空）；夹具 helper 写在 `view_repo.rs` 的 `tests` 模块中。两个 L0 tree 按 receive-pack 的真实路径写入：原始 tree 字节构造成 `Entry`（`obj_type` 为 Tree，`hash` 为原始字节的 sha1），经 `process_entry`（`src/jupiter/utils/converter.rs:133-143`）与 `convert_to_mega_model(EntryMeta::default())`（`:106-117`）得到 `MegaObjectModel::Tree(model)`，把 `model.commit_id` 设为引入它的夹具根提交的 id，再以 `vec![model.into_active_model()]` 经 `StorageConnector::batch_save_model`（`src/jupiter/storage/base_storage.rs:165`）写入，与 `MonoService::save_entry` 的 tree 分支相同（`src/jupiter/service/mono_service.rs:175-190`、`:227-229`）。T_mode 含一个 `100664` 条目，T_gbk 含一个 GBK 编码的文件名（字节 `C4 E3 BA C3`，即「你好」）；先断言两行都满足 `hash(Tree, sub_trees) ≠ tree_id`。T_mode 位于 `a/mode`，是视图 `:/a/mode` 的结果 tree；T_gbk 位于 `b/gbk`，是视图 `:/b/gbk:prefix=p` 中被 Prefix 包住的 T。视图 `:/c` 的子目录 `c/d` 只含一个文件名独有的 blob `hp23-only-in-c-d.txt`；对照视图 `:/e` 的闭包只含自洽 tree。夹具前提（设计 §7.4「L0 tree 不自洽」测试项断言中的「投影照常推进，清表重建后哈希不变」，L0 tree 的读取由 HP-29、`catch_up` 由 HP-11 承接；前提不成立按 Out of scope 对应条处理）：四个视图都 `catch_up` 到就绪；记下各视图 commit_map 中按 seq 排列的视图提交 id；然后清表重建：删除四个过滤器的 `mega_view_commit_map` 行与 `mega_view_object_ref` 行，以及本 schema 的全部 `mega_view_object` 行，状态列复位为冷启动，再 `catch_up` 到就绪；各视图的视图提交 id 序列与重建前逐字节相同。预检调用：每个视图新建一个 ViewRepo，want = [tip]、have 为空，依次调用 `check_wants_and_ready` 与 `prepare_pack`。捕获的日志按行计数。先依次检查 `:/a/mode`、`:/b/gbk:prefix=p`、`:/e`；随后断言夹具写入的所有 tree 中 id 等于 `c/d` 的只有路径 `c/d` 这一处，再删去 `c/d` 的 `mega_tree` 行，最后检查 `:/c`。断言：`:/a/mode` 与 `:/b/gbk:prefix=p` 返回 `ViewPackRejected`，消息逐字节等于 `view pack aborted: tree does not match its stored entries`；`:/c` 返回 `ViewPackRejected`，消息逐字节等于 `view pack aborted: tree is missing`；每次拒绝，级别为 ERROR 且同时含 `filter_id=<该 filter_id>` 与 `tree_id=<该 tree_id>` 两个字段的日志行恰为 1 行，其它级别的行不计；`:/a/mode` 与 `:/b/gbk:prefix=p` 的这一行另含 `metric=view_pack_tree_mismatch_total`；`:/c` 的这一行不含 `metric=`；`:/e` 返回 `Ok(())`，不出现含 `filter_id=<:/e 的 filter_id>` 的 ERROR 级行；每次调用前后 `mega_view_filter` 中该视图的行逐列相等） +- [x] `source .env.test && cargo test -p mega2 --lib 'ceres::pack::view_repo::tests::prepare_pack_rechecks_single_snapshot' -- --exact`（new；AC 7。四种情形与对照组各用一个独立的 `test_db_connection` schema，写同一夹具：视图 `:/a` 已就绪，有三个视图提交 C1、C2、C3。新建两个 ViewRepo 实例 R1、R2，want 都为 [C3]，各调用一次 `check_wants_and_ready`，断言 `Ok(())`；随后用自动提交的语句改库；然后 R1 以 have 为空、R2 以 have = [C1] 调用 `prepare_pack`。四种改库：① 回收：`UPDATE mega_view_filter SET ready_seq = NULL, projected_seq = 0, warming_since = NULL WHERE id = `，再删除该过滤器的 `mega_view_commit_map` 行与 `mega_view_object_ref` 行（设计 §4.6「视图删除即回收」第 2、3 步）；② 只把 `ready_seq` 置为 NULL；③ 只删除 C3 所在的那一行 `mega_view_commit_map`；④ 用 `view_test_fixtures` 把 `main@/` CAS 为无关历史，调用 `extend_root_chain` 判出不连续，使 `root_chain_halted` 为真。四种情形下两次调用都返回 `MegaError::ViewUnavailable`，不是 `ViewPackRejected`，filter_id 为本视图；reason 在 ①、②、③ 中为 `warming up`，在 ④ 中为 `root chain halted`。对照：不改库时两次调用都返回 `Ok(())`） +- [x] `source .env.test && cargo test -p mega2 --lib 'ceres::pack::view_repo::tests::commit_map_read_only_by_prepare_pack' -- --exact`（new；AC 8。用 `test_db_config` 取得 `(db_config, _schema)`，守卫持有到用例结束；经 `database_connection(&db_config)`（`src/jupiter/storage/init.rs:19-26`，建连时执行迁移）自建连接，在这条连接上设 `set_metric_callback`，记录每条语句的 SQL 文本；以 `Storage::new_with_connection(config, Arc::new(conn), mock_object_storage())` 组装 ViewRepo 所用的存储，夹具的 blob 写入这份存储的对象存储；不在 `test_db_connection` 返回的连接上替换回调（GC-HP-10）。视图 `:/a` 已就绪，有三个视图提交 C1、C2、C3。夹具（含 `catch_up`）写完之后，每轮与对照组开始前都清空记录，只统计从该轮第一个调用起的语句；各轮都不调用 `check_wants_and_ready`。两轮各用一个新实例：want = [C3]、have 为空，调用 `prepare_pack` 后调用 `full_pack`；want = [C3]、have = [C1]，调用 `prepare_pack` 后调用 `incremental_pack`。每轮断言两项单快照口径：从 `prepare_pack` 开始到它返回，SQL 文本含 `mega_view_filter` 的语句恰为 1 条，它同时含 `mega_view_commit_map`，即就绪状态与 commit_map 由同一条语句读出（设计 §4.6「读者的单快照规则」第 4 项）；读尽 pack 流之后，该轮 SQL 文本含 `mega_view_commit_map` 的语句仍恰为这 1 条。对照组：另用一个新实例不调用 `prepare_pack`，直接调用 `full_pack`，不记录任何含 `mega_view_commit_map` 的语句；该调用的返回值不在本行断言） +- [x] `source .env.test && cargo test -p mega2 --test integration_views view_precheck_http_ -- --test-threads=1`（new；前缀过滤，`test result` 中 passed ≥ 3；AC 1 的 HTTP 部分、AC 3、AC 4、AC 5。服务以 `service http` 启动：trunk、sha1、`MEGA_VIEWS__ENABLED=true`、`MEGA_VIEWS__ALLOW_ANONYMOUS_REGISTER=true`、`push_auth = none`、`anonymous_access = true`。夹具按 ADR-HP-06 由测试进程直接写服务所用的测试库：用 git-internal 的 `*_with_kind` API 构造对象，写入 `mega_tree`、`mega_commit` 后 CAS `main@/`，再经 `POST /api/v1/views?wait=true` 逐个注册视图，每个等到就绪再注册下一个（`max_concurrent_cold_starts` 缺省为 2）。根历史在服务初始化得到的 `main@/` 之后追加两个提交。夹具写入之前，先经测试库读出 `main@/` 的提交 id，记为 R0。R1 的父提交为 R0，在 `hp23/mode`、`hp23/gbk`、`hp23/ok`、`hp23/miss/d` 放自洽 tree，其中 `hp23/miss/d` 只含一个文件名独有的 blob `hp23-only-in-miss-d.txt`。R2 的父提交为 R1，把 `hp23/mode`、`hp23/gbk` 分别换成含 `100664` 条目的 T_mode 与含 GBK 文件名的 T_gbk。CAS 以 R0 为期望值，把 `main@/` 置为 R2（R1 若无父，而根链表已接入 R0，按设计 §3.3 第 3 步「c 无父、根链表非空」判为不连续，视图一直 503）。L0 tree 的行按 receive-pack 的落库字节写入：`tree_id` 为原始字节的 sha1，`sub_trees` 为 git-internal `Tree::from_bytes(原始字节, id)` 再 `to_data()` 的结果，与 `process_entry`、`into_mega_model` 落库的字节相同（`src/jupiter/utils/converter.rs:133-143`、`:260-271`）。原始请求用 `reqwest` blocking 发到 `/.filter/.git/git-upload-pack`，v2 带 `Git-Protocol: version=2`，取状态码、`Content-Type` 与响应体字节；v0 have 非空的请求在首个 want 行带 `multi_ack_detailed`。`view_precheck_http_want_not_ours`：视图 `:/hp23/ok`，oid 取 R2，即 `main@/` 的提交 id（在 `mega_commit` 中，不在本视图的 commit_map 中）；五种请求形态为 v0 have 为空、v0 have = [视图 tip]、v2 带 `done`、v2 不带 `done` 且 have 为 R1（不在视图链上）、v2 不带 `done` 且 have = [视图 tip]；每个都返回 200，`Content-Type` 为 `application/x-git-upload-pack-result`，响应体恰为一个载荷为 `ERR upload-pack: not our ref \n` 的 pkt-line，不含 `acknowledgments`、`NAK`、`ACK`、`packfile` 与 `PACK`；对照组 want = [视图 tip]、have 为 R1、不带 `done` 的 v2 请求返回 `0014acknowledgments\n0008NAK\n0000`；用例前后 `view_pack_tree_mismatch_total` 相等。`view_precheck_http_l0_mismatch`：视图 `:/hp23/mode` 与 `:/hp23/gbk:prefix=p` 各有视图提交 V1、V2；对每个视图依次发送四种请求形态（v0 have 为空；v0 have = [V1]；v2 带 `done`；v2 不带 `done`、have = [V1]），want 都为 [V2]，每个都返回 200、同上 `Content-Type`，响应体恰为一个载荷为 `ERR view pack aborted: tree does not match its stored entries\n` 的 pkt-line，不含上述五个标记；每个请求前后各取一次 `GET /api/v1/views/metrics`，响应 `data` 中的 `view_pack_tree_mismatch_total` 恰加 1；每次拒绝之后 v0 与 v2 的 `GET info/refs?service=git-upload-pack` 返回 200；NAK 轮：want = [V2]、have 为 R1、不带 `done`，返回 `0014acknowledgments\n0008NAK\n0000`，计数不变，随后同一 want 带 `done` 的请求得到上述 ERR，计数加 1。`view_precheck_http_missing_tree`：视图 `:/hp23/miss` 就绪后，先断言夹具写入的所有 tree 中 id 等于 `hp23/miss/d` 的只有这一处路径，再删去该 tree 的 `mega_tree` 行；v0 have 为空与 v2 带 `done` 各一次，返回 200、同上 `Content-Type`，响应体恰为一个载荷为 `ERR view pack aborted: tree is missing\n` 的 pkt-line，不含上述五个标记；`view_pack_tree_mismatch_total` 不变；之后 v0 与 v2 的 `GET info/refs?service=git-upload-pack` 返回 200） +- [x] `source .env.test && docker compose -p mega2-it -f docker/docker-compose.test.yml --profile git up -d --wait && cargo test -p mega2 --test integration_git_ssh integration_git_ssh_view_precheck_ -- --test-threads=1`（new；前缀过滤，`test result` 中 passed ≥ 3；AC 1、2、3 的 SSH 部分。三个用例各自建库启动服务：`service multi http ssh`，用 `boot_service_multi`（`tests/integration_git_ssh.rs:1727-1784`）的同形变体启动，该变体多接收一组环境变量，监听地址取 `git_cli::service_listen_host()`；先迁移启动再正式启动，写法同 `boot_storage_only_ssh`（`:523-541`），但 known_hosts 不用 `write_known_hosts_via_host_keyscan`。配置为 trunk、sha1、`MEGA_CEDAR__ENFORCEMENT=off`、`MEGA_VIEWS__ENABLED=true`、`MEGA_VIEWS__ALLOW_ANONYMOUS_REGISTER=true`、`push_auth = none`、`anonymous_access = true`、`ssh_receive_pack = false`。SSH 客户端在 git-cli 容器内：启动之前以 `git_cli::generate_client_ed25519` 在 `env.ssh_dir` 下生成 `client_ed25519`，它只为满足 `git_cli::git_ssh_command` 中的 `-i`，不写入数据库（storage-only 下经 none 认证）；正式启动之后以 `git_cli::write_known_hosts_via_keyscan(&env.ssh_dir.join("known_hosts"), ssh_port)` 在容器内写 known_hosts，该 helper 先为容器的运行 UID 补好 passwd 项（`tests/common/git_cli.rs:1194`），容器内的 `ssh` 依赖这一步；SSH 命令串取 `git_cli::git_ssh_command(&env.case_dir, ssh_port)`，下称 ``；SSH 视图 URL 为 `git_cli::mega2_ssh_repo_url(ssh_port, "git")` 加 `.filter/.git`。夹具经 HTTP 腿的 trunk 推送写入，写法同 `seed_project_foo`（`tests/integration_git_cli.rs:1784-1809`）：宿主 git 经 `git_host_ssh`（`:480-501`）执行，访问 `http://127.0.0.1:/project`（HTTP URL 不受 `GIT_SSH_COMMAND` 影响）；克隆 `/project`，先推一个提交建立 `hp23-mode`、`hp23-gbk`、`hp23-ok`、`hp23-miss/d`（都是自洽 tree，`hp23-miss/d` 只含一个文件名独有的 blob `hp23-only-in-miss-d.txt`）；再用 `git mktree` 生成含 `100664` 条目的 T_mode，以及以 C 风格引号路径 `"\304\343\272\303.txt"`（GBK 编码的「你好」）生成含 GBK 文件名的 T_gbk，用 `git ls-tree HEAD` 的输出替换这两个条目后经 `git mktree` 生成新的 `/project` 树，`git commit-tree -p HEAD` 后推送。推送之后、注册视图之前，经该用例的测试库连接（`TestDatabase`，`:37-60`）读出 T_mode、T_gbk 两行 `mega_tree`，先断言夹具前提：两行都满足 `ObjectHash::from_type_and_data_for_kind(HashKind::Sha1, ObjectType::Tree, sub_trees) ≠ tree_id`；这一断言早于任何 ERR 断言。视图 `:/project/hp23-mode`、`:/project/hp23-gbk:prefix=p`、`:/project/hp23-ok`、`:/project/hp23-miss` 经 `POST /api/v1/views?wait=true` 逐个注册并就绪。开工核对（ER-02）若发现推送被拒，或落库后 T_mode / T_gbk 的行不满足 `hash(Tree, sub_trees) ≠ tree_id`，改为按 ADR-HP-06 直接写库（写法同第 4 行），并按 ER-03 修订本卡。`integration_git_ssh_view_precheck_raw`：原始请求在 git-cli 容器内执行。`` 由用例内的 `docker ps -q --filter label=com.docker.compose.project=mega2-it --filter label=com.docker.compose.service=git-cli --filter status=running` 取得（与 `tests/common/git_cli.rs:349-378` 的私有 helper 写法相同，不走 Compose 的 `exec` 子命令）。每个请求在宿主执行 `timeout -k 5 45 docker exec -i sh -c <脚本>`，脚本由 ``、v2 时追加的 `-o SetEnv=GIT_PROTOCOL=version=2`、`git@` 与远端命令 `git-upload-pack '/.filter/.git'` 组成；stdin 为预先构造的请求字节，经 `docker exec -i` 送入容器；取 stdout 字节与退出码。顺序固定：先测 want 不属于本视图（视图 `:/project/hp23-ok`，oid 取 `main@/` 的提交 id）的五种请求形态，再测两个 L0 视图各自的四种请求形态（形态定义同第 4 行），再测对照组；这些断言全部完成之后，用 `git ls-tree -r -t` 对两个推送提交断言 `hp23-miss/d` 的 tree id 只出现在这一处路径，再经该用例的测试库连接删去这棵 tree 的 `mega_tree` 行，最后对 `:/project/hp23-miss` 发 v0 have 为空与 v2 带 `done` 两种请求。三种触发的 stdout 在宣告之后恰为一个 ERR pkt-line，载荷与第 4 行对应的触发相同，退出码 1。对照组：want 合法、have 不在视图链上、不带 `done` 的 v2 请求（对 `:/project/hp23-ok` 与 `:/project/hp23-mode` 各一次），stdout 在宣告之后恰为 `0014acknowledgments\n0008NAK\n0000`，退出码 0。`integration_git_ssh_view_precheck_git_client_not_our_ref`：经 `git_cli::git_cli_ssh(&env.case_dir, &, …)` 在容器内先 `git init` 一个空仓库，再在其中执行 `git -c protocol.version=2 fetch `，退出非零，stderr 含 `remote error: upload-pack: not our ref `。`integration_git_ssh_view_precheck_git_client_l0_clone`：经 `git_cli::git_cli_ssh` 在容器内对两个 L0 视图各执行 `git -c protocol.version=0 clone` 与 `git -c protocol.version=2 clone`，都退出非零，stderr 含 `remote error:` 与该 tree_id，不含 `did not send all necessary objects`；对照组：`:/project/hp23-ok` 的两种 clone 都成功） +- [x] `source .env.test && docker compose -p mega2-it -f docker/docker-compose.test.yml --profile git up -d --wait && cargo test -p mega2 --test integration_git_cli integration_git_cli_view_precheck_ -- --test-threads=1`（new；前缀过滤，`test result` 中 passed ≥ 2；AC 2 的 HTTP 部分。`boot_service_http_with_env`（`tests/integration_git_cli.rs:308`）以 `trunk_boot_env()`（`:1658-1664`）加 `MEGA_VIEWS__ENABLED=true`、`MEGA_VIEWS__ALLOW_ANONYMOUS_REGISTER=true` 启动；夹具与四个视图的构造同第 5 行，推送用 `trunk_host_git`（`:1669-1697`）访问 `http://127.0.0.1:/project`；夹具前提（T_mode、T_gbk 两行都满足 `hash(Tree, sub_trees) ≠ tree_id`）同样在推送之后、注册视图之前断言，早于任何 ERR 断言，测试库连接取 `env.database.db_url`；开工核对的回落条件与做法同第 5 行；本行不删除任何行。客户端命令在 git-cli runner 中经 `git_cli::git_cli_no_auth`（`tests/common/git_cli.rs:603`）执行，视图 URL 用 `git_cli::mega2_http_url`（`:310`）拼出。`integration_git_cli_view_precheck_not_our_ref`：在空仓库中执行 `git -c protocol.version=2 fetch `，退出非零，stderr 含 `remote error: upload-pack: not our ref `。v0 不做客户端用例：v0 宣告不含 `allow-tip-sha1-in-want` 与 `allow-reachable-sha1-in-want`（`src/ceres/protocol/smart.rs:54-61`），git 在本地拒绝请求未宣告的对象（`git/git@v2.49.1 fetch-pack.c:661-662`、`:2266`），请求到不了服务端；v0 的服务端应答由第 4、5 行覆盖。`integration_git_cli_view_precheck_l0_clone`：对两个 L0 视图各执行 `git -c protocol.version=0 clone` 与 `git -c protocol.version=2 clone`，都退出非零，stderr 含 `remote error:` 与该 tree_id，不含 `did not send all necessary objects`；对照组：`:/project/hp23-ok` 的两种 clone 都成功） +- [x] `source .env.test && bash -c 'p=docs/refactoring/protocol.md; s=target/tmp/hp23/precheck-section.md; chk(){ if rg -q -- "$2" "$1"; then echo "OK: $2"; else rc=$?; echo "FAIL: $2 (rg exit $rc)"; exit 1; fi; }; chkf(){ if rg -q -F -- "$2" "$1"; then echo "OK: $2"; else rc=$?; echo "FAIL: $2 (rg exit $rc)"; exit 1; fi; }; chk "$p" "^### 视图打包预检（HP-23）$"; chk "$p" "^\| 视图 fetch 非本视图提交（not our ref） \| cargo:integration_git_cli_view_precheck_not_our_ref \| cargo:integration_git_ssh_view_precheck_git_client_not_our_ref \|"; chk "$p" "^\| 视图 clone 遇 L0 tree 不自洽 \| cargo:integration_git_cli_view_precheck_l0_clone \| cargo:integration_git_ssh_view_precheck_git_client_l0_clone \|"; mkdir -p target/tmp/hp23 || exit 1; awk "/^### 视图打包预检（HP-23）\$/{f=1;next} /^##/{f=0} f" "$p" > "$s" || { echo "FAIL: awk"; exit 1; }; test -s "$s" || { echo "FAIL: empty section"; exit 1; }; for k in "warming up" "root chain halted" "does not match its stored entries" "is missing" view_pack_tree_mismatch_total NAK mega_view_commit_map history-projection.md; do chkf "$s" "$k"; done'`（文档同步：输出全部为 `OK`，退出码 0。先核对小节标题与场景覆盖表的两行，两行的 HTTP、SSH 两列都是本卡的 `cargo:` 单元格；再按标题切出该小节（到下一个 `##` 或 `###` 标题为止），小节非空，并逐个以 `rg -F` 核对 Docs and compatibility impact 要求写入的八个关键词：两个 reason、两条拒绝消息的措辞、计数名、`NAK`（NAK 轮不预检）、`mega_view_commit_map`（打包方法不再读取）与指向设计 §6.1 的链接文件名。开工前小节标题与两行都零命中；任一项不命中或 `rg` 执行失败都判 FAIL） **Dependencies:** `HP-22`（消费 ViewRepo 的 `full_pack` / `incremental_pack` 及其闭包计算、`view_repo.rs` 的 `tests` 模块与打包用例：本卡把闭包计算移入 `prepare_pack` 的计数，打包方法改为按计数结果编码；写集 `src/ceres/pack/view_repo.rs`、`src/jupiter/storage/view_projection_storage.rs`、`tests/integration_git_cli.rs`、`tests/integration_git_ssh.rs`、`docs/refactoring/protocol.md` 串行于其后，`src/jupiter/service/view_metrics.rs` 经上游链传递串行。Current evidence 所列的钩子、错误变体、SSH 传输、视图分派、计数端点、`ViewMetrics`、可失败批量读取、tree 的共用解析函数与夹具经上游链传递可用）、`HP-21`（写集：`tests/integration_views.rs` 串行于其后） @@ -6106,6 +6106,10 @@ Result 只允许 `PASS` 或 `FAIL`。`FAIL` 必须列出 P0/P1 条目，并在 | HP22-R2 | HP-22 修订后的打包实现与验收，Claude CLI 独立只读复审 | PASS | P0 0；P1 0；低优先级建议 3 项，R3 前关闭 | 容器 fetch 的工作目录及环境、gitlink 精确闭包与 ER-03 措辞补齐 | `/tmp/mega2-hp22-claude-r2.txt`；`VERDICT: PASS` | | HP22-R3 | HP-22 最终差异与八项验收，Claude CLI 独立只读复审 | PASS | P0 0；P1 0；P2 0 | R2 三项建议均关闭；HTTP/SSH v0/v2、单次 SQL 定位、tree 分层与 pack 闭包逐项核对 | `/tmp/mega2-hp22-claude-r3.txt`；`VERDICT: PASS` | | HP22-R4 | HP-22 代码、测试、文档及本地门禁，Codex 复核 | PASS | P0 0；P1 0；P2 0 | 8/8 精确验证：lib 3 条各 1/1，HTTP Josh 与增量各 1/1、clone 3/3，SSH 2/2，文档 20 项 OK；19 个目标的完整测试退出 0；nightly 格式与全目标全特性 Clippy 退出 0；两个构建命令退出 0，专用 macOS 链接参数下 0 警告；`libra diff --check` 通过 | 当前差异、`/tmp/mega2-hp22-ver-final-1.txt` 至 `-8.txt`、`/tmp/mega2-hp22-test-all-r2.txt`、`/tmp/mega2-hp22-build-nowarn.txt`、`/tmp/mega2-hp22-build-tests-nowarn.txt` | +| HP23-R1 | HP-23 预检、HTTP/SSH 黑盒与文档，Claude CLI 只读审查 | PASS | P0 0；P1 0 | 文档小节层级与 HTTP tree id 断言已修正；夹具路径对齐任务卡 | `target/tmp/hp23/claude-r1.txt`；源码审查，差异文件因 CLI 读取限制未读 | +| HP23-R2 | HP-23 首轮修订，Claude CLI 只读复审 | FAIL | P1：夹具修订后 nightly 格式门未重跑；R3 前已执行格式化、格式检查及 Clippy | 测试专用提交读取、`:/e` 告警断言、缺 tree 路径唯一性与 HTTP 四视图夹具已收紧；have 侧旧树验证偏严格由发布者（Codex）作为 fail-closed 残余行为接受 | `target/tmp/hp23/claude-r2.txt` | +| HP23-R3 | HP-23 最终源码、八项验收与七条验证，Claude CLI 只读复审 | PASS | P0 0；P1 0 | 三项非阻断意见：have 侧旧树也校验、同层多个不自洽 tree 的首个 id 由数据库行序决定、SSH 原始请求的 stderr 管道；发布者（Codex）接受为本卡残余风险，均不改变既定错误契约 | `target/tmp/hp23/claude-r3.txt`；`VERDICT: PASS`；7/7 定向验证、nightly 格式、Clippy、ER-04 A 组通过 | +| HP23-R4 | HP-23 代码、测试、协议应答、文档及发布门，Codex 独立复核 | PASS | P0 0；P1 0 | 七条精确验证全通过；ER-04 A 组只读 smoke 8 passed、0 failed（既有 LFS 指针使用 `GIT_LFS_SKIP_SMUDGE=1`）；最终版本树的完整测试 19 个目标均 0 失败，lib 2038 通过、1 项既有忽略；nightly 格式、全目标全特性 Clippy、两个构建命令退出 0，定向 macOS 链接参数下 0 警告；保留 R3 的三项 P2 残余行为 | `target/tmp/hp23/codex-r4.md`、`target/tmp/hp23/full-test-final.log`、`target/tmp/hp23/claude-r3.txt`、`libra diff --check` | ## 非目标与延后项 diff --git a/docs/refactoring/protocol.md b/docs/refactoring/protocol.md index b8252638..753484d5 100644 --- a/docs/refactoring/protocol.md +++ b/docs/refactoring/protocol.md @@ -656,6 +656,12 @@ SSH 中 `git-lfs-transfer` 返回明确 unsupported failure，`git-lfs-authentic **（2026-06-30）CI 自动化回归 gate 已落地；2026-07-01 扩展 CL push 与 LFS；2026-08-03 对齐 Monorepo 分支/tag 规则）**：`.github/workflows/git-protocol-smoke.yml` 在 PR 和 main push 时自动启动 PostgreSQL + Redis 测试栈、构建 release 二进制、seed mail secret、启动 `service http`，然后以 monorepo 根路径 `http://ci-smoke:@127.0.0.1:9000/` 为目标运行 `scripts/git_protocol_smoke.sh` 的 HTTP 矩阵（ls-remote、clone、fetch、protocol v2 fetch/ls-remote、shallow clone depth=1、blob:none partial clone、CL push（无新公开分支）、**拒绝** Git-client tag push、LFS push/clone/pull/locks-list）。workflow 直接在 smoke DB 的 `access_token` 表插入一次性 token 并 mask 日志输出，满足 push 所需认证；同时安装 `git-lfs` 并启用 `MEGA2_GIT_SMOKE_LFS=1`，让 LFS round-trip 成为默认 CI gate。该 workflow 在协议路径变更（`src/ceres/protocol/**`、`src/contract/git_protocol/**`、`src/server/http_server.rs`、`scripts/git_protocol_smoke.sh`、`docs/refactoring/protocol.md`）时触发，满足阶段 0 "每个后续阶段都能复用该矩阵防止回归"的验收标准。 +### 视图打包预检（HP-23） + +`prepare_pack` 以单条 SQL 对就绪状态、根链和 want 归属做单快照复查。未就绪或 want 已被回收时 reason 为 `warming up`；根链停追时为 `root chain halted`。计数阶段读取待发 tree 的存储字节，对 L0 `mega_tree` 行验证哈希，不自洽时拒绝并报告 `does not match its stored entries`；tree 缺失或无法解析时按缺对象处理，报告 `is missing`。这两种拒绝不改变视图就绪状态。 + +每次 L0 拒绝使 `view_pack_tree_mismatch_total` 加一，并发出含 `metric`、`filter_id`、`tree_id` 的 ERROR 事件。缺对象也发出含 `filter_id`、`tree_id` 的 ERROR 事件，不带 `metric`，不增加该计数。v2 的 `NAK` 轮不做打包预检。打包方法只按计数结果编码，不再读取 `mega_view_commit_map`。四类失败的线上应答见[历史投影设计 §6.1 错误契约](history-projection.md)。 + ### 场景覆盖表（权威，plan-20260803 / ADR-GM-01） > **口径（强制）:** `git clone` / `git fetch` **不等于**覆盖字面 `git pull`。字面 pull 仅以下方 `cargo:…_pull_…` 格计；不得用 clone/fetch 冒充 pull 覆盖。 @@ -691,6 +697,8 @@ SSH 中 `git-lfs-transfer` 返回明确 unsupported failure，`git-lfs-authentic | 视图 fetch | cargo:integration_git_cli_view_pack_http_fetch_after_trunk_push | cargo:integration_git_ssh_view_pack_ssh_josh_proxy | N/A+读授权与路径仓库相同 | N/A+视图只有 ViewRepo 一种形态 | | 已就绪的空视图 | cargo:integration_git_cli_view_pack_http_clone_ready_empty | cargo:integration_git_ssh_view_pack_ssh_ready_empty | N/A+读授权与路径仓库相同 | N/A+视图只有 ViewRepo 一种形态 | | 未就绪的视图 | cargo:integration_git_cli_view_layer3_http_unready | cargo:integration_git_ssh_view_layer3_unready | cargo:layer3_before_resolve_auth | N/A+视图只有 ViewRepo 一种形态 | +| 视图 fetch 非本视图提交（not our ref） | cargo:integration_git_cli_view_precheck_not_our_ref | cargo:integration_git_ssh_view_precheck_git_client_not_our_ref | N/A+只读，anonymous_access 默认放行 | N/A+视图 URL（/.filter/、/.view/） | +| 视图 clone 遇 L0 tree 不自洽 | cargo:integration_git_cli_view_precheck_l0_clone | cargo:integration_git_ssh_view_precheck_git_client_l0_clone | N/A+只读，anonymous_access 默认放行 | N/A+视图 URL（/.filter/、/.view/） | #### GAP 映射（plan-20260803） diff --git a/src/ceres/pack/view_repo.rs b/src/ceres/pack/view_repo.rs index 0f5a7a6f..06123e56 100644 --- a/src/ceres/pack/view_repo.rs +++ b/src/ceres/pack/view_repo.rs @@ -18,14 +18,14 @@ use git_internal::{ pack::{encode::PackEncoder, entry::Entry}, }, }; -use tokio::sync::mpsc; +use tokio::sync::{Mutex, mpsc}; use tokio_stream::wrappers::ReceiverStream; use crate::{ ceres::{ pack::RepoHandler, protocol::import_refs::{RefCommand, Refs}, - view::tree_source::{TreeSource, empty_tree_id, read_tree}, + view::tree_source::{InMemoryTreeSource, TreeSource, empty_tree_id, read_tree}, }, common::{ errors::{MegaError, ViewUnavailableReason}, @@ -53,6 +53,7 @@ pub(crate) struct ViewRepo { projection: ViewProjectionService, filter_pk: i64, filter_id: String, + pack_plan: Mutex>, } struct ViewPackPlan { @@ -63,23 +64,86 @@ struct ViewPackPlan { } impl ViewRepo { + async fn count_trees( + &self, + kind: HashKind, + ids: &[String], + empty: &str, + ) -> Result, MegaError> { + let view_storage = self.storage.view_storage(); + let requested = ids + .iter() + .filter(|id| id.as_str() != empty) + .cloned() + .collect::>(); + let mut view_trees = view_storage.view_pack_view_trees(&requested).await?; + let remaining = requested + .iter() + .filter(|id| !view_trees.contains_key(*id)) + .cloned() + .collect::>(); + let mono = self.storage.mono_storage(); + let l0_trees = mono + .get_trees_by_hashes_fallible(view_storage.get_connection(), &remaining) + .await?; + for tree in l0_trees { + let actual = + ObjectHash::from_type_and_data_for_kind(kind, ObjectType::Tree, &tree.sub_trees) + .map_err(GitError::from)?; + if actual.to_string() != tree.tree_id { + self.storage.view_metrics().increment_pack_tree_mismatch(); + tracing::error!( + metric = %"view_pack_tree_mismatch_total", + filter_id = %self.filter_id, + tree_id = %tree.tree_id, + "view pack tree does not match its stored entries" + ); + return Err(MegaError::ViewPackRejected(format!( + "view {} pack aborted: tree {} does not match its stored entries", + self.filter_id, tree.tree_id + ))); + } + view_trees.insert(tree.tree_id, tree.sub_trees); + } + let source = InMemoryTreeSource::new(kind, view_trees, HashSet::new()); + ids.iter() + .map(|id| { + let tree = read_tree(kind, &source, id).map_err(|_| { + tracing::error!( + filter_id = %self.filter_id, + tree_id = %id, + "view pack tree is missing" + ); + MegaError::ViewPackRejected(format!( + "view {} pack aborted: tree {} is missing", + self.filter_id, id + )) + })?; + Ok((id.clone(), tree)) + }) + .collect() + } + async fn count_pack( &self, kind: HashKind, wants: &[String], haves: &[String], - ) -> Result { + ) -> Result { let view_storage = self.storage.view_storage(); - let bounds = view_storage - .view_pack_bounds(self.filter_pk, wants, haves) - .await - .map_err(GitError::from)?; - let commits = view_storage - .view_pack_commits(self.filter_pk, bounds.have_seq, bounds.want_seq) - .await - .map_err(GitError::from)?; - let empty = empty_tree_id(kind)?.to_string(); - let mut previous = bounds.have_tree.unwrap_or_else(|| empty.clone()); + let snapshot = view_storage + .view_pack_snapshot(self.filter_pk, wants, haves) + .await? + .ok_or_else(|| self.unavailable(ViewUnavailableReason::WarmingUp))?; + if snapshot.halted { + return Err(self.unavailable(ViewUnavailableReason::RootChainHalted)); + } + if snapshot.ready_seq.is_none() || !snapshot.wants_valid { + return Err(self.unavailable(ViewUnavailableReason::WarmingUp)); + } + let commits = snapshot.commits; + let empty = empty_tree_id(kind).map_err(GitError::from)?.to_string(); + let mut previous = snapshot.have_tree.unwrap_or_else(|| empty.clone()); let mut frontier = BTreeSet::new(); for commit in &commits { if commit.tree_id != previous { @@ -98,18 +162,7 @@ impl ViewRepo { .collect::>() .into_iter() .collect::>(); - let mono = self.storage.mono_storage(); - let mut source = ViewTreeSource::new(&mono, view_storage.get_connection(), kind) - .map_err(GitError::from)?; - source.prefetch(&ids).await.map_err(GitError::from)?; - let trees = ids - .iter() - .map(|id| { - read_tree(kind, &source, id) - .map(|tree| (id.clone(), tree)) - .map_err(|missing| GitError::CustomError(format!("view tree {missing:?}"))) - }) - .collect::, _>>()?; + let trees = self.count_trees(kind, &ids, &empty).await?; let mut next = BTreeSet::new(); let mut level = Vec::new(); for (old_id, new_id) in frontier { @@ -292,6 +345,7 @@ impl ViewRepo { projection, filter_pk, filter_id, + pack_plan: Mutex::new(None), } } @@ -423,6 +477,14 @@ impl RepoHandler for ViewRepo { Ok(()) } + async fn prepare_pack(&self, want: &[String], have: &[String]) -> Result<(), MegaError> { + *self.pack_plan.lock().await = None; + let kind = self.object_hash_kind()?; + let plan = self.count_pack(kind, want, have).await?; + *self.pack_plan.lock().await = Some(plan); + Ok(()) + } + async fn finalize_receive_pack(&self) -> Result<(), MegaError> { Err(read_only_mega_error()) } @@ -442,19 +504,29 @@ impl RepoHandler for ViewRepo { Err(read_only_git_error()) } - async fn full_pack(&self, want: Vec) -> Result>, GitError> { + async fn full_pack(&self, _want: Vec) -> Result>, GitError> { let kind = self.object_hash_kind().map_err(GitError::from)?; - let plan = self.count_pack(kind, &want, &[]).await?; + let plan = self + .pack_plan + .lock() + .await + .take() + .ok_or_else(|| GitError::CustomError("view pack was not prepared".to_owned()))?; self.encode_pack(plan, kind).await } async fn incremental_pack( &self, - want: Vec, - have: Vec, + _want: Vec, + _have: Vec, ) -> Result>, GitError> { let kind = self.object_hash_kind().map_err(GitError::from)?; - let plan = self.count_pack(kind, &want, &have).await?; + let plan = self + .pack_plan + .lock() + .await + .take() + .ok_or_else(|| GitError::CustomError("view pack was not prepared".to_owned()))?; self.encode_pack(plan, kind).await } @@ -582,9 +654,106 @@ mod tests { }, }, tests::{test_db_config, test_storage_with_config}, + utils::converter::{MegaObjectModel, process_entry}, }, }; + fn raw_l0_tree(mode: &str, name: &[u8], blob: ObjectHash) -> (ObjectHash, Vec) { + let mut bytes = format!("{mode} ").into_bytes(); + bytes.extend_from_slice(name); + bytes.push(0); + bytes.extend(hex::decode(blob.to_string()).unwrap()); + let id = ObjectHash::from_type_and_data_for_kind(HashKind::Sha1, ObjectType::Tree, &bytes) + .unwrap(); + (id, bytes) + } + + fn replace_fixture_subtree( + fixture: &mut RootTreeFixture, + parent_name: &str, + child_name: &str, + child_id: ObjectHash, + ) { + let old_root = fixture.root.id; + let old_parent = fixture + .root + .tree_items + .iter() + .find(|item| item.name == parent_name) + .unwrap() + .id; + let mut parent_items = fixture + .trees + .iter() + .find(|tree| tree.id == old_parent) + .unwrap() + .tree_items + .clone(); + let old_child = parent_items + .iter_mut() + .find(|item| item.name == child_name) + .unwrap(); + let old_child_id = old_child.id; + old_child.id = child_id; + let new_parent = Tree::from_tree_items_with_kind(HashKind::Sha1, parent_items).unwrap(); + let mut root_items = fixture.root.tree_items.clone(); + root_items + .iter_mut() + .find(|item| item.name == parent_name) + .unwrap() + .id = new_parent.id; + let new_root = Tree::from_tree_items_with_kind(HashKind::Sha1, root_items).unwrap(); + fixture + .trees + .retain(|tree| tree.id != old_root && tree.id != old_parent && tree.id != old_child_id); + fixture.trees.extend([new_parent, new_root.clone()]); + fixture.root = new_root; + } + + fn capture_tracing(f: impl FnOnce() -> T) -> (T, String) { + use std::io::Write; + + use tracing_subscriber::fmt::MakeWriter; + + #[derive(Clone)] + struct TestWriter(Arc>>); + + impl Write for TestWriter { + fn write(&mut self, buf: &[u8]) -> std::io::Result { + self.0.lock().unwrap().extend_from_slice(buf); + Ok(buf.len()) + } + + fn flush(&mut self) -> std::io::Result<()> { + Ok(()) + } + } + + impl MakeWriter<'_> for TestWriter { + type Writer = TestWriter; + + fn make_writer(&self) -> Self::Writer { + self.clone() + } + } + + let _pin_registry = tracing::Dispatch::new( + tracing_subscriber::fmt() + .with_writer(std::io::sink) + .with_max_level(tracing::Level::DEBUG) + .finish(), + ); + let buffer = Arc::new(Mutex::new(Vec::new())); + let subscriber = tracing_subscriber::fmt() + .with_writer(TestWriter(buffer.clone())) + .with_ansi(false) + .with_max_level(tracing::Level::DEBUG) + .finish(); + let result = tracing::subscriber::with_default(subscriber, f); + let text = String::from_utf8(buffer.lock().unwrap().clone()).unwrap(); + (result, text) + } + struct Fixture { _temp: TempDir, storage: Storage, @@ -795,6 +964,20 @@ mod tests { (declared, unique) } + async fn prepared_full_pack(repo: &ViewRepo, want: Vec) -> ReceiverStream> { + repo.prepare_pack(&want, &[]).await.unwrap(); + repo.full_pack(want).await.unwrap() + } + + async fn prepared_incremental_pack( + repo: &ViewRepo, + want: Vec, + have: Vec, + ) -> ReceiverStream> { + repo.prepare_pack(&want, &have).await.unwrap(); + repo.incremental_pack(want, have).await.unwrap() + } + fn fixture_view_closure(root: &RootCommitFixture, view_tree: &str) -> HashSet { let mut ids = HashSet::new(); let mut pending = vec![view_tree.to_owned()]; @@ -852,7 +1035,7 @@ mod tests { for index in [2, 7] { let want = commits[index].object_id.clone(); let (count, ids) = - pack_ids(repo.full_pack(vec![want]).await.unwrap(), &fixture._temp).await; + pack_ids(prepared_full_pack(&repo, vec![want]).await, &fixture._temp).await; let mut expected = HashSet::new(); for (source_index, commit) in commits.iter().take(index + 1).enumerate() { expected.insert(commit.object_id.clone()); @@ -983,9 +1166,7 @@ mod tests { assert_eq!(commits.len(), 8); for index in [2, 7] { let (_, ids) = pack_ids( - repo.full_pack(vec![commits[index].object_id.clone()]) - .await - .unwrap(), + prepared_full_pack(&repo, vec![commits[index].object_id.clone()]).await, &fixture._temp, ) .await; @@ -1086,12 +1267,12 @@ mod tests { let want = commits[11].object_id.clone(); for have_index in [1, 4, 8] { let (_, ids) = pack_ids( - repo.incremental_pack( + prepared_incremental_pack( + &repo, vec![want.clone()], vec![commits[have_index].object_id.clone()], ) - .await - .unwrap(), + .await, &fixture._temp, ) .await; @@ -1131,20 +1312,24 @@ mod tests { assert!(have_closure.is_superset(&want_closure)); } let (_, full) = pack_ids( - repo.full_pack(vec![want.clone()]).await.unwrap(), + prepared_full_pack(&repo, vec![want.clone()]).await, &fixture._temp, ) .await; assert!(!full.contains(&old_target) && !full.contains(&new_target)); let (_, one_have) = pack_ids( - repo.incremental_pack(vec![want.clone()], vec![commits[4].object_id.clone()]) - .await - .unwrap(), + prepared_incremental_pack( + &repo, + vec![want.clone()], + vec![commits[4].object_id.clone()], + ) + .await, &fixture._temp, ) .await; let (_, multiple_haves) = pack_ids( - repo.incremental_pack( + prepared_incremental_pack( + &repo, vec![want.clone()], vec![ commits[1].object_id.clone(), @@ -1152,8 +1337,7 @@ mod tests { fixture.roots[0].commit.id.to_string(), ], ) - .await - .unwrap(), + .await, &fixture._temp, ) .await; @@ -1174,7 +1358,8 @@ mod tests { .view_tip .unwrap(); let (_, off_chain) = pack_ids( - repo.incremental_pack( + prepared_incremental_pack( + &repo, vec![want], vec![ fixture.roots[0].commit.id.to_string(), @@ -1182,8 +1367,7 @@ mod tests { "f".repeat(40), ], ) - .await - .unwrap(), + .await, &fixture._temp, ) .await; @@ -1271,46 +1455,440 @@ mod tests { assert_eq!(a_commits.len(), 200); assert_eq!(b_commits.len(), 10); counter.store(0, Ordering::Relaxed); - a_repo - .full_pack(vec![a_commits[199].object_id.clone()]) + prepared_full_pack(&a_repo, vec![a_commits[199].object_id.clone()]) .await - .unwrap() .concat() .await; let full_a = counter.load(Ordering::Relaxed); counter.store(0, Ordering::Relaxed); - b_repo - .full_pack(vec![b_commits[9].object_id.clone()]) + prepared_full_pack(&b_repo, vec![b_commits[9].object_id.clone()]) .await - .unwrap() .concat() .await; let full_b = counter.load(Ordering::Relaxed); assert_eq!(full_a, full_b); counter.store(0, Ordering::Relaxed); - a_repo - .incremental_pack( - vec![a_commits[199].object_id.clone()], - vec![a_commits[198].object_id.clone()], - ) - .await - .unwrap() - .concat() - .await; + prepared_incremental_pack( + &a_repo, + vec![a_commits[199].object_id.clone()], + vec![a_commits[198].object_id.clone()], + ) + .await + .concat() + .await; let short = counter.load(Ordering::Relaxed); counter.store(0, Ordering::Relaxed); - a_repo - .incremental_pack( - vec![a_commits[199].object_id.clone()], - vec![a_commits[49].object_id.clone()], - ) - .await - .unwrap() - .concat() - .await; + prepared_incremental_pack( + &a_repo, + vec![a_commits[199].object_id.clone()], + vec![a_commits[49].object_id.clone()], + ) + .await + .concat() + .await; assert_eq!(short, counter.load(Ordering::Relaxed)); } + #[tokio::test] + async fn commit_map_read_only_by_prepare_pack() { + let temp = tempfile::tempdir().unwrap(); + let (db_config, _schema) = test_db_config(temp.path()).await; + let mut config = isolated_config(temp.path().join("config")); + config.database = db_config.clone(); + let statements = Arc::new(Mutex::new(Vec::::new())); + let recorded = statements.clone(); + let mut connection = database_connection(&db_config).await.unwrap(); + connection.set_metric_callback(move |info| { + recorded.lock().unwrap().push(info.statement.to_string()); + }); + let storage = Storage::new_with_connection( + Arc::new(config), + Arc::new(connection), + mock_object_storage(), + ) + .await + .unwrap(); + let trees = (0..3) + .map(|index| { + root_tree_from_paths( + HashKind::Sha1, + &[("a/file".to_owned(), format!("content-{index}").into_bytes())], + ) + }) + .collect::>(); + let roots = seed_linear_root_history_with_trees( + storage.view_storage().get_connection(), + HashKind::Sha1, + trees, + ) + .await; + for root in &roots { + for blob in &root.blobs { + storage + .git_service + .save_object_from_raw(Bytes::copy_from_slice(&blob.data)) + .await + .unwrap(); + } + } + assert_eq!( + storage + .view_storage() + .extend_root_chain(None, 100, ViewLockMode::Try) + .await + .unwrap(), + RootChainOutcome::CaughtUp + ); + let filter_id = insert_filter(&storage, 5001, ":/a").await; + let projection = ViewProjectionService::new(storage.clone(), storage.view_metrics()); + projection + .catch_up_one_batch(5001, &storage.config(), 100) + .await + .unwrap(); + let commits = storage + .view_storage() + .view_pack_commits(5001, 0, 3) + .await + .unwrap(); + assert_eq!(commits.len(), 3); + let want = vec![commits[2].object_id.clone()]; + for have in [Vec::new(), vec![commits[0].object_id.clone()]] { + let repo = ViewRepo::new( + storage.clone(), + storage.config(), + projection.clone(), + 5001, + filter_id.clone(), + ); + statements.lock().unwrap().clear(); + repo.prepare_pack(&want, &have).await.unwrap(); + { + let sql = statements.lock().unwrap(); + let filter_reads = sql + .iter() + .filter(|statement| statement.contains("mega_view_filter")) + .collect::>(); + assert_eq!(filter_reads.len(), 1); + assert!(filter_reads[0].contains("mega_view_commit_map")); + } + if have.is_empty() { + repo.full_pack(want.clone()).await.unwrap().concat().await; + } else { + repo.incremental_pack(want.clone(), have) + .await + .unwrap() + .concat() + .await; + } + assert_eq!( + statements + .lock() + .unwrap() + .iter() + .filter(|statement| statement.contains("mega_view_commit_map")) + .count(), + 1 + ); + } + let config = storage.config(); + let control = ViewRepo::new(storage, config, projection, 5001, filter_id); + statements.lock().unwrap().clear(); + assert!(control.full_pack(want).await.is_err()); + assert_eq!( + statements + .lock() + .unwrap() + .iter() + .filter(|statement| statement.contains("mega_view_commit_map")) + .count(), + 0 + ); + } + + #[test] + fn prepare_pack_rejects_l0_and_missing_trees() { + let (expected, logs) = capture_tracing(|| { + let rt = tokio::runtime::Builder::new_current_thread() + .enable_all() + .build() + .unwrap(); + rt.block_on(async { + let paths = |index| { + [ + ( + "a/mode/good.txt".to_owned(), + format!("mode-{index}").into_bytes(), + ), + ( + "b/gbk/good.txt".to_owned(), + format!("gbk-{index}").into_bytes(), + ), + ( + "c/d/hp23-only-in-c-d.txt".to_owned(), + format!("missing-{index}").into_bytes(), + ), + ( + "e/ok.txt".to_owned(), + format!("control-{index}").into_bytes(), + ), + ] + }; + let first = root_tree_from_paths(HashKind::Sha1, &paths(0)); + let mut second = root_tree_from_paths(HashKind::Sha1, &paths(1)); + let blob = second.blobs[0].id; + let (mode_id, mode_raw) = raw_l0_tree("100664", b"file.txt", blob); + let (gbk_id, gbk_raw) = + raw_l0_tree("100644", &[0xc4, 0xe3, 0xba, 0xc3, b'.', b't', b'x', b't'], blob); + replace_fixture_subtree(&mut second, "a", "mode", mode_id); + replace_fixture_subtree(&mut second, "b", "gbk", gbk_id); + let fixture = Fixture::from_trees(vec![first, second], ":/a/mode", 2, 100).await; + let db = fixture.storage.view_storage(); + for (tree_id, raw) in [(mode_id, mode_raw), (gbk_id, gbk_raw)] { + let entry = Entry { + obj_type: ObjectType::Tree, + data: raw, + hash: tree_id, + chain_len: 0, + }; + let MegaObjectModel::Tree(mut model) = + process_entry(entry).convert_to_mega_model(EntryMeta::new()) + else { + panic!("tree entry must make a tree model") + }; + model.commit_id = fixture.roots[1].commit.id.to_string(); + assert_ne!( + ObjectHash::from_type_and_data_for_kind( + HashKind::Sha1, + ObjectType::Tree, + &model.sub_trees + ) + .unwrap() + .to_string(), + model.tree_id + ); + fixture + .storage + .mono_storage() + .base + .batch_save_model::(vec![ + model.into_active_model(), + ]) + .await + .unwrap(); + } + let c_d_id = fixture.roots[1] + .trees + .iter() + .find(|tree| { + tree.tree_items + .iter() + .any(|item| item.name == "hp23-only-in-c-d.txt") + }) + .unwrap() + .id + .to_string(); + let filters = [ + (5001, fixture.filter_id.clone(), mode_id.to_string()), + ( + 5002, + insert_filter(&fixture.storage, 5002, ":/b/gbk:prefix=p").await, + gbk_id.to_string(), + ), + ( + 5003, + insert_filter(&fixture.storage, 5003, ":/c").await, + c_d_id.clone(), + ), + ( + 5004, + insert_filter(&fixture.storage, 5004, ":/e").await, + String::new(), + ), + ]; + let mut original = HashMap::new(); + for (pk, _, _) in &filters { + ViewProjectionService::new( + fixture.storage.clone(), + fixture.storage.view_metrics(), + ) + .catch_up_one_batch(*pk, &fixture.storage.config(), 100) + .await + .unwrap(); + let commits = db.view_pack_commits(*pk, 0, 2).await.unwrap(); + assert_eq!(commits.len(), 2); + original.insert( + *pk, + commits + .into_iter() + .map(|commit| commit.object_id) + .collect::>(), + ); + } + for table in ["mega_view_object_ref", "mega_view_commit_map"] { + db.get_connection() + .execute_unprepared(&format!("DELETE FROM {table}")) + .await + .unwrap(); + } + db.get_connection() + .execute_unprepared("DELETE FROM mega_view_object") + .await + .unwrap(); + db.get_connection() + .execute_unprepared( + "UPDATE mega_view_filter SET projected_seq = 0, ready_seq = NULL, \ + warming_since = now() WHERE id BETWEEN 5001 AND 5004", + ) + .await + .unwrap(); + for (pk, _, _) in &filters { + ViewProjectionService::new( + fixture.storage.clone(), + fixture.storage.view_metrics(), + ) + .catch_up_one_batch(*pk, &fixture.storage.config(), 100) + .await + .unwrap(); + let current = db + .view_pack_commits(*pk, 0, 2) + .await + .unwrap() + .into_iter() + .map(|commit| commit.object_id) + .collect::>(); + assert_eq!(current, original[pk]); + assert!( + db.view_reader_state(*pk) + .await + .unwrap() + .unwrap() + .ready_seq + .is_some() + ); + } + for (pk, filter_id, tree_id) in [&filters[0], &filters[1], &filters[3]] { + let before = mega_view_filter::Entity::find_by_id(*pk) + .one(db.get_connection()) + .await + .unwrap() + .unwrap(); + let repo = ViewRepo::new( + fixture.storage.clone(), + fixture.storage.config(), + ViewProjectionService::new( + fixture.storage.clone(), + fixture.storage.view_metrics(), + ), + *pk, + filter_id.clone(), + ); + let want = vec![original[pk][1].clone()]; + repo.check_wants_and_ready(&want).await.unwrap(); + let result = repo.prepare_pack(&want, &[]).await; + if *pk == 5004 { + result.unwrap(); + } else { + assert!(matches!( + result.unwrap_err(), + MegaError::ViewPackRejected(message) + if message == format!( + "view {filter_id} pack aborted: tree {tree_id} does not match its stored entries" + ) + )); + } + let after = mega_view_filter::Entity::find_by_id(*pk) + .one(db.get_connection()) + .await + .unwrap() + .unwrap(); + assert_eq!(before, after); + } + assert_eq!( + fixture + .storage + .view_metrics() + .counters() + .view_pack_tree_mismatch_total, + 2 + ); + assert_eq!( + fixture + .roots + .iter() + .flat_map(|root| &root.trees) + .filter(|tree| tree.id.to_string() == c_d_id) + .count(), + 1 + ); + db.get_connection() + .execute_raw(Statement::from_sql_and_values( + DbBackend::Postgres, + "DELETE FROM mega_tree WHERE tree_id = $1", + [sea_orm::Value::from(c_d_id.clone())], + )) + .await + .unwrap(); + let (pk, filter_id, tree_id) = &filters[2]; + let before = mega_view_filter::Entity::find_by_id(*pk) + .one(db.get_connection()) + .await + .unwrap() + .unwrap(); + let repo = ViewRepo::new( + fixture.storage.clone(), + fixture.storage.config(), + ViewProjectionService::new( + fixture.storage.clone(), + fixture.storage.view_metrics(), + ), + *pk, + filter_id.clone(), + ); + let want = vec![original[pk][1].clone()]; + repo.check_wants_and_ready(&want).await.unwrap(); + assert!(matches!( + repo.prepare_pack(&want, &[]).await.unwrap_err(), + MegaError::ViewPackRejected(message) + if message == format!("view {filter_id} pack aborted: tree {tree_id} is missing") + )); + let after = mega_view_filter::Entity::find_by_id(*pk) + .one(db.get_connection()) + .await + .unwrap() + .unwrap(); + assert_eq!(before, after); + assert_eq!( + fixture + .storage + .view_metrics() + .counters() + .view_pack_tree_mismatch_total, + 2 + ); + filters + }) + }); + for (pk, filter_id, tree_id) in &expected { + let lines = logs + .lines() + .filter(|line| { + line.contains("ERROR") + && line.contains(&format!("filter_id={filter_id}")) + && (*pk == 5004 || line.contains(&format!("tree_id={tree_id}"))) + }) + .collect::>(); + if *pk == 5004 { + assert!(lines.is_empty()); + } else { + assert_eq!(lines.len(), 1, "{pk}: {logs}"); + assert_eq!( + lines[0].contains("metric=view_pack_tree_mismatch_total"), + *pk != 5003 + ); + } + } + } + async fn insert_filter(storage: &Storage, filter_pk: i64, spec: &str) -> String { let canonical = parse_for_registration(spec).unwrap(); let filter_id = canonical.filter_id.clone(); @@ -1354,6 +1932,93 @@ mod tests { } } + #[tokio::test] + async fn prepare_pack_rechecks_single_snapshot() { + for scenario in 0..5 { + let fixture = Fixture::new(3, ":/repo").await; + fixture.catch_up(1).await; + let commits = fixture + .storage + .view_storage() + .view_pack_commits(fixture.filter_pk, 0, 3) + .await + .unwrap(); + assert_eq!(commits.len(), 3); + let want = vec![commits[2].object_id.clone()]; + let r1 = fixture.current_repo(); + let r2 = fixture.current_repo(); + r1.check_wants_and_ready(&want).await.unwrap(); + r2.check_wants_and_ready(&want).await.unwrap(); + let db = fixture.storage.view_storage(); + match scenario { + 0 => { + fixture.set_state(0, None, false).await; + db.get_connection() + .execute_raw(Statement::from_sql_and_values( + DbBackend::Postgres, + "DELETE FROM mega_view_object_ref WHERE filter_pk = $1", + [sea_orm::Value::from(fixture.filter_pk)], + )) + .await + .unwrap(); + db.get_connection() + .execute_raw(Statement::from_sql_and_values( + DbBackend::Postgres, + "DELETE FROM mega_view_commit_map WHERE filter_pk = $1", + [sea_orm::Value::from(fixture.filter_pk)], + )) + .await + .unwrap(); + } + 1 => fixture.set_state(3, None, false).await, + 2 => { + db.get_connection() + .execute_raw(Statement::from_sql_and_values( + DbBackend::Postgres, + "DELETE FROM mega_view_commit_map \ + WHERE filter_pk = $1 AND view_commit = $2", + [ + sea_orm::Value::from(fixture.filter_pk), + sea_orm::Value::from(want[0].clone()), + ], + )) + .await + .unwrap(); + } + 3 => { + assert!( + cas_fixture_main(db.get_connection(), &fixture.roots[2], &fixture.roots[0]) + .await + ); + assert!(matches!( + db.extend_root_chain(None, 100, ViewLockMode::Try) + .await + .unwrap(), + RootChainOutcome::Discontinuous(_) + )); + } + 4 => {} + _ => unreachable!(), + } + for (repo, have) in [(&r1, Vec::new()), (&r2, vec![commits[0].object_id.clone()])] { + let result = repo.prepare_pack(&want, &have).await; + if scenario == 4 { + result.unwrap(); + } else { + assert_unavailable( + result.unwrap_err(), + &fixture.filter_id, + if scenario == 3 { + ViewUnavailableReason::RootChainHalted + } else { + ViewUnavailableReason::WarmingUp + }, + ); + } + } + } + } + #[tokio::test] async fn unavailable_rejects_refs_and_wants() { let fixture = Fixture::new(3, ":/repo").await; diff --git a/src/jupiter/service/view_metrics.rs b/src/jupiter/service/view_metrics.rs index d78bc700..29f1a164 100644 --- a/src/jupiter/service/view_metrics.rs +++ b/src/jupiter/service/view_metrics.rs @@ -7,6 +7,7 @@ use std::sync::{ pub(crate) struct ViewMetrics { pub(crate) view_batch_premise_failures_total: Arc, pub(crate) view_projection_stops_total: Arc, + pub(crate) view_pack_tree_mismatch_total: Arc, } impl ViewMetrics { @@ -20,12 +21,20 @@ impl ViewMetrics { .fetch_add(1, Ordering::Relaxed); } + pub(crate) fn increment_pack_tree_mismatch(&self) { + self.view_pack_tree_mismatch_total + .fetch_add(1, Ordering::Relaxed); + } + pub(crate) fn counters(&self) -> ViewMetricCounters { ViewMetricCounters { view_batch_premise_failures_total: self .view_batch_premise_failures_total .load(Ordering::Relaxed), view_projection_stops_total: self.view_projection_stops_total.load(Ordering::Relaxed), + view_pack_tree_mismatch_total: self + .view_pack_tree_mismatch_total + .load(Ordering::Relaxed), } } } @@ -34,6 +43,7 @@ impl ViewMetrics { pub(crate) struct ViewMetricCounters { pub(crate) view_batch_premise_failures_total: u64, pub(crate) view_projection_stops_total: u64, + pub(crate) view_pack_tree_mismatch_total: u64, } #[derive(Clone, Debug, Eq, PartialEq, serde::Serialize, utoipa::ToSchema)] diff --git a/src/jupiter/storage/view_projection_storage.rs b/src/jupiter/storage/view_projection_storage.rs index df38c170..77d4f7b9 100644 --- a/src/jupiter/storage/view_projection_storage.rs +++ b/src/jupiter/storage/view_projection_storage.rs @@ -1,4 +1,4 @@ -use std::collections::BTreeMap; +use std::collections::{BTreeMap, HashMap}; use chrono::Utc; use sea_orm::{ @@ -17,7 +17,8 @@ use crate::{ }, common::{errors::MegaError, utils::MEGA_BRANCH_NAME}, jupiter::storage::{ - base_storage::StorageConnector, view_root_chain::ROOT_CHAIN_HALTED_SQL, + base_storage::{BaseStorage, StorageConnector}, + view_root_chain::ROOT_CHAIN_HALTED_SQL, view_storage::ViewStorage, }, }; @@ -59,10 +60,12 @@ pub(crate) struct ViewWantState { pub(crate) wants: Vec<(String, Option)>, } -pub(crate) struct ViewPackBounds { - pub(crate) want_seq: i64, - pub(crate) have_seq: i64, +pub(crate) struct ViewPackSnapshot { + pub(crate) ready_seq: Option, + pub(crate) halted: bool, + pub(crate) wants_valid: bool, pub(crate) have_tree: Option, + pub(crate) commits: Vec, } pub(crate) struct ViewPackCommit { @@ -72,48 +75,110 @@ pub(crate) struct ViewPackCommit { } impl ViewStorage { - pub(crate) async fn view_pack_bounds( + pub(crate) async fn view_pack_snapshot( &self, filter_pk: i64, wants: &[String], haves: &[String], - ) -> Result { - let row = self + ) -> Result, MegaError> { + let rows = self .get_connection() - .query_one_raw(Statement::from_sql_and_values( + .query_all_raw(Statement::from_sql_and_values( sea_orm::DbBackend::Postgres, - "WITH requested AS ( \ - SELECT 'want' AS role, unnest($2::text[]) AS oid \ - UNION ALL SELECT 'have', unnest($3::text[]) \ - ), matched AS ( \ - SELECT r.role, m.seq_from FROM requested r \ - JOIN mega_view_commit_map m ON m.filter_pk = $1 AND m.view_commit = r.oid \ - ), bounds AS ( \ - SELECT max(seq_from) FILTER (WHERE role = 'want') AS want_seq, \ - coalesce(max(seq_from) FILTER (WHERE role = 'have'), 0) AS have_seq \ - FROM matched \ - ) \ - SELECT b.want_seq, b.have_seq, m.view_tree AS have_tree \ - FROM bounds b LEFT JOIN mega_view_commit_map m \ - ON m.filter_pk = $1 AND m.seq_from = b.have_seq", + format!( + "WITH state AS MATERIALIZED ( \ + SELECT f.id, f.ready_seq, f.projected_seq, \ + {ROOT_CHAIN_HALTED_SQL} AS halted \ + FROM mega_view_filter f WHERE f.id = $1 \ + ), wanted AS ( \ + SELECT mapped.seq_from FROM state s \ + CROSS JOIN LATERAL unnest($2::text[]) AS w(oid) \ + LEFT JOIN LATERAL ( \ + SELECT m.seq_from FROM mega_view_commit_map m \ + WHERE m.filter_pk = s.id AND m.view_commit = w.oid \ + AND m.seq_from <= s.projected_seq \ + ORDER BY m.seq_from LIMIT 1 \ + ) mapped ON TRUE \ + ), bounds AS ( \ + SELECT (SELECT max(seq_from) FROM wanted) AS want_seq, \ + (SELECT coalesce(bool_and(seq_from IS NOT NULL), FALSE) FROM wanted) \ + AS wants_valid, \ + (SELECT coalesce(max(m.seq_from), 0) FROM state s \ + JOIN mega_view_commit_map m ON m.filter_pk = s.id \ + WHERE m.view_commit = ANY($3::text[]) \ + AND m.seq_from <= s.projected_seq) AS have_seq \ + ) \ + SELECT s.ready_seq, s.halted, b.wants_valid, \ + h.view_tree AS have_tree, m.view_commit, m.view_tree, o.data \ + FROM state s CROSS JOIN bounds b \ + LEFT JOIN mega_view_commit_map h \ + ON h.filter_pk = s.id AND h.seq_from = b.have_seq \ + LEFT JOIN mega_view_commit_map m \ + ON m.filter_pk = s.id AND m.seq_from > b.have_seq \ + AND m.seq_from <= b.want_seq AND m.view_commit IS NOT NULL \ + LEFT JOIN mega_view_object o \ + ON o.object_id = m.view_commit AND o.kind = 1 \ + ORDER BY m.seq_from" + ), [ Value::from(filter_pk), Value::from(wants.to_vec()), Value::from(haves.to_vec()), ], )) - .await? - .ok_or_else(|| MegaError::Other("view pack bounds returned no row".to_owned()))?; - let want_seq: Option = row.try_get("", "want_seq")?; - Ok(ViewPackBounds { - want_seq: want_seq.ok_or_else(|| { - MegaError::Other("view pack want is outside the view chain".to_owned()) - })?, - have_seq: row.try_get("", "have_seq")?, - have_tree: row.try_get("", "have_tree")?, - }) + .await?; + let Some(first) = rows.first() else { + return Ok(None); + }; + let mut snapshot = ViewPackSnapshot { + ready_seq: first.try_get("", "ready_seq")?, + halted: first.try_get("", "halted")?, + wants_valid: first.try_get("", "wants_valid")?, + have_tree: first.try_get("", "have_tree")?, + commits: Vec::new(), + }; + if snapshot.halted || snapshot.ready_seq.is_none() || !snapshot.wants_valid { + return Ok(Some(snapshot)); + } + for row in rows { + let Some(object_id) = row.try_get::>("", "view_commit")? else { + continue; + }; + let data: Option> = row.try_get("", "data")?; + snapshot.commits.push(ViewPackCommit { + tree_id: row.try_get("", "view_tree")?, + data: data.ok_or_else(|| { + MegaError::Other(format!("view commit object missing: {object_id}")) + })?, + object_id, + }); + } + Ok(Some(snapshot)) + } + + pub(crate) async fn view_pack_view_trees( + &self, + ids: &[String], + ) -> Result>, MegaError> { + let mut trees = HashMap::new(); + for chunk in ids.chunks(::BATCH_CHUNK_SIZE) { + let rows = self + .get_connection() + .query_all_raw(Statement::from_sql_and_values( + sea_orm::DbBackend::Postgres, + "SELECT object_id, data FROM mega_view_object \ + WHERE kind = 2 AND object_id = ANY($1::text[])", + [Value::from(chunk.to_vec())], + )) + .await?; + for row in rows { + trees.insert(row.try_get("", "object_id")?, row.try_get("", "data")?); + } + } + Ok(trees) } + #[cfg(test)] pub(crate) async fn view_pack_commits( &self, filter_pk: i64, diff --git a/tests/integration_git_cli.rs b/tests/integration_git_cli.rs index a17a06ff..f0c326b4 100644 --- a/tests/integration_git_cli.rs +++ b/tests/integration_git_cli.rs @@ -5951,6 +5951,243 @@ fn hp22_http_boot() -> (GitCliEnv, ServiceProcess, u16, PathBuf) { (env, service, port, err) } +fn hp23_host_git_input(case_dir: &Path, repo: &str, args: &[&str], input: &[u8]) -> String { + let mut child = Command::new("git") + .current_dir(case_dir) + .args(["-C", repo]) + .args(args) + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .spawn() + .unwrap(); + child.stdin.take().unwrap().write_all(input).unwrap(); + let output = child.wait_with_output().unwrap(); + git_cli::assert_git_success(&output, "HP-23 fixture git"); + String::from_utf8(output.stdout).unwrap().trim().to_owned() +} + +fn hp23_http_fixture( + case_dir: &Path, + port: u16, + db_url: &str, +) -> (String, Vec<(String, String)>, String, String) { + git_ok_no_auth( + case_dir, + &["clone", &trunk_subpath_url(port, "/project"), "hp23-source"], + ); + configure_git_identity_no_auth(case_dir, "hp23-source"); + for name in ["mode", "gbk", "ok", "miss/d"] { + let file = if name == "miss/d" { + "hp23-only-in-miss-d.txt" + } else { + "good.txt" + }; + let path = case_dir + .join("hp23-source") + .join(format!("hp23-{name}")) + .join(file); + fs::create_dir_all(path.parent().unwrap()).unwrap(); + fs::write(path, name).unwrap(); + } + git_ok_no_auth( + case_dir, + &[ + "-C", + "hp23-source", + "add", + "hp23-mode", + "hp23-gbk", + "hp23-ok", + "hp23-miss", + ], + ); + git_ok_no_auth(case_dir, &["-C", "hp23-source", "commit", "-m", "HP23 R1"]); + git_cli::assert_git_success(&trunk_push(case_dir, "hp23-source"), "HP-23 R1 push"); + let blob = git_stdout_no_auth( + case_dir, + &["-C", "hp23-source", "rev-parse", "HEAD:hp23-mode/good.txt"], + ); + let mode_input = format!("100664 blob {blob}\tmode.txt\n"); + let mode_tree = + hp23_host_git_input(case_dir, "hp23-source", &["mktree"], mode_input.as_bytes()); + let gbk_input = format!("100644 blob {blob}\t\"\\304\\343\\272\\303.txt\"\n"); + let gbk_tree = hp23_host_git_input(case_dir, "hp23-source", &["mktree"], gbk_input.as_bytes()); + let ok_tree = git_stdout_no_auth( + case_dir, + &["-C", "hp23-source", "rev-parse", "HEAD:hp23-ok"], + ); + let miss_tree = git_stdout_no_auth( + case_dir, + &["-C", "hp23-source", "rev-parse", "HEAD:hp23-miss"], + ); + let prior_root = git_stdout_no_auth(case_dir, &["-C", "hp23-source", "ls-tree", "HEAD"]); + let root_input = prior_root + .lines() + .map(|line| { + let name = line.rsplit_once('\t').unwrap().1; + let replacement = match name { + "hp23-mode" => Some(&mode_tree), + "hp23-gbk" => Some(&gbk_tree), + "hp23-ok" => Some(&ok_tree), + "hp23-miss" => Some(&miss_tree), + _ => None, + }; + replacement.map_or_else( + || format!("{line}\n"), + |id| format!("040000 tree {id}\t{name}\n"), + ) + }) + .collect::(); + let root_tree = + hp23_host_git_input(case_dir, "hp23-source", &["mktree"], root_input.as_bytes()); + let parent = git_stdout_no_auth(case_dir, &["-C", "hp23-source", "rev-parse", "HEAD"]); + let r2 = hp23_host_git_input( + case_dir, + "hp23-source", + &["commit-tree", &root_tree, "-p", &parent, "-m", "HP23 R2"], + b"", + ); + git_ok_no_auth( + case_dir, + &["-C", "hp23-source", "update-ref", "refs/heads/main", &r2], + ); + git_cli::assert_git_success(&trunk_push(case_dir, "hp23-source"), "HP-23 R2 push"); + let rows = with_runtime(async { + let db = Database::connect(db_url).await.unwrap(); + let mut rows = Vec::new(); + for id in [&mode_tree, &gbk_tree] { + let row = db + .query_one_raw(Statement::from_string( + DatabaseBackend::Postgres, + format!("SELECT sub_trees FROM mega_tree WHERE tree_id = '{id}'"), + )) + .await + .unwrap() + .expect("L0 tree row"); + let bytes: Vec = row.try_get("", "sub_trees").unwrap(); + let actual = git_internal::hash::ObjectHash::from_type_and_data_for_kind( + git_internal::hash::HashKind::Sha1, + git_internal::internal::object::types::ObjectType::Tree, + &bytes, + ) + .unwrap(); + assert_ne!(actual.to_string(), *id, "L0 fixture must be noncanonical"); + rows.push(id.to_owned()); + } + rows + }); + let mut views = Vec::new(); + for (name, spec) in [ + ("mode", ":/project/hp23-mode"), + ("gbk", ":/project/hp23-gbk:prefix=p"), + ("ok", ":/project/hp23-ok"), + ("miss", ":/project/hp23-miss"), + ] { + let id = hp22_register(port, &format!("hp23-{name}"), spec); + hp22_wait_lag_zero(port, &id); + views.push((name.to_owned(), id)); + } + assert_eq!(rows, vec![mode_tree.clone(), gbk_tree.clone()]); + (r2, views, mode_tree, gbk_tree) +} + +#[test] +fn integration_git_cli_view_precheck_not_our_ref() { + let (env, mut service, port, err) = hp22_http_boot(); + let (root_tip, views, _, _) = hp23_http_fixture(&env.case_dir, port, &env.database.db_url); + let ok = &views.iter().find(|(name, _)| name == "ok").unwrap().1; + let url = git_cli::mega2_http_url(port, &format!("/.filter/{ok}.git")); + git_cli::assert_git_success( + &git_cli::git_cli_no_auth(&env.case_dir, &["init", "hp23-fetch"]), + "HP-23 init fetch client", + ); + let output = git_cli::git_cli_no_auth( + &env.case_dir, + &[ + "-C", + "hp23-fetch", + "-c", + "protocol.version=2", + "fetch", + &url, + &root_tip, + ], + ); + assert!(!output.status.success(), "{output:?}"); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!( + stderr.contains(&format!( + "remote error: upload-pack: not our ref {root_tip}" + )), + "{stderr}" + ); + assert!( + service + .shutdown_via_sigint(Duration::from_secs(10)) + .success(), + "{}", + read_log(&err) + ); +} + +#[test] +fn integration_git_cli_view_precheck_l0_clone() { + let (env, mut service, port, err) = hp22_http_boot(); + let (_, views, mode_tree, gbk_tree) = + hp23_http_fixture(&env.case_dir, port, &env.database.db_url); + for (name, id) in &views { + if name == "miss" { + continue; + } + let url = git_cli::mega2_http_url(port, &format!("/.filter/{id}.git")); + for version in ["0", "2"] { + let clone = format!("hp23-{name}-v{version}"); + let output = git_cli::git_cli_no_auth( + &env.case_dir, + &[ + "-c", + &format!("protocol.version={version}"), + "clone", + &url, + &clone, + ], + ); + let stderr = String::from_utf8_lossy(&output.stderr); + if name == "ok" { + git_cli::assert_git_success(&output, "HP-23 control clone"); + } else { + assert!(!output.status.success(), "{output:?}"); + let tree_id = if name == "mode" { + &mode_tree + } else { + &gbk_tree + }; + assert!( + stderr.contains("remote error:") && stderr.contains(tree_id), + "{stderr}" + ); + assert!( + stderr.contains("tree ") + && stderr.contains("does not match its stored entries"), + "{stderr}" + ); + assert!( + !stderr.contains("did not send all necessary objects"), + "{stderr}" + ); + } + } + } + assert!( + service + .shutdown_via_sigint(Duration::from_secs(10)) + .success(), + "{}", + read_log(&err) + ); +} + fn hp22_register(port: u16, name: &str, spec: &str) -> String { let response = reqwest::blocking::Client::builder() .no_proxy() diff --git a/tests/integration_git_ssh.rs b/tests/integration_git_ssh.rs index 98df5ded..8bf30648 100644 --- a/tests/integration_git_ssh.rs +++ b/tests/integration_git_ssh.rs @@ -3645,6 +3645,463 @@ fn hp22_ssh_boot() -> (GitSshEnv, ServiceProcess, u16, u16, PathBuf) { (env, service, http_port, ssh_port, err) } +fn hp23_ssh_git_input(case_dir: &Path, args: &[&str], input: &[u8]) -> String { + let mut child = Command::new("git") + .current_dir(case_dir) + .args(["-C", "hp23-source"]) + .args(args) + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .spawn() + .unwrap(); + child.stdin.take().unwrap().write_all(input).unwrap(); + let output = child.wait_with_output().unwrap(); + git_cli::assert_git_success(&output, "HP-23 SSH fixture git"); + String::from_utf8(output.stdout).unwrap().trim().to_owned() +} + +struct Hp23SshFixture { + root_tip: String, + prior_root: String, + views: Vec<(String, String)>, + mode_tree: String, + gbk_tree: String, + missing_tree: String, +} + +fn hp23_ssh_fixture(env: &GitSshEnv, http_port: u16) -> Hp23SshFixture { + let case_dir = env.case_dir.as_path(); + let project_url = git_cli::mega2_host_http_url(http_port, "/project"); + hp22_ssh_host_git(case_dir, &["clone", &project_url, "hp23-source"]); + hp22_ssh_host_git( + case_dir, + &["-C", "hp23-source", "config", "user.name", "HP23"], + ); + hp22_ssh_host_git( + case_dir, + &[ + "-C", + "hp23-source", + "config", + "user.email", + "hp23@example.invalid", + ], + ); + for (name, file) in [ + ("mode", "good.txt"), + ("gbk", "good.txt"), + ("ok", "good.txt"), + ("miss/d", "hp23-only-in-miss-d.txt"), + ] { + let path = case_dir + .join("hp23-source") + .join(format!("hp23-{name}")) + .join(file); + fs::create_dir_all(path.parent().unwrap()).unwrap(); + fs::write(path, name).unwrap(); + } + hp22_ssh_host_git( + case_dir, + &[ + "-C", + "hp23-source", + "add", + "hp23-mode", + "hp23-gbk", + "hp23-ok", + "hp23-miss", + ], + ); + hp22_ssh_host_git(case_dir, &["-C", "hp23-source", "commit", "-m", "HP23 R1"]); + hp22_ssh_host_git( + case_dir, + &[ + "-C", + "hp23-source", + "push", + "--no-thin", + "origin", + "HEAD:refs/heads/main", + ], + ); + let blob = hp22_ssh_host_git( + case_dir, + &["-C", "hp23-source", "rev-parse", "HEAD:hp23-mode/good.txt"], + ) + .trim() + .to_owned(); + let mode_input = format!("100664 blob {blob}\tmode.txt\n"); + let mode_tree = hp23_ssh_git_input(case_dir, &["mktree"], mode_input.as_bytes()); + let gbk_input = format!("100644 blob {blob}\t\"\\304\\343\\272\\303.txt\"\n"); + let gbk_tree = hp23_ssh_git_input(case_dir, &["mktree"], gbk_input.as_bytes()); + let ok_tree = hp22_ssh_host_git( + case_dir, + &["-C", "hp23-source", "rev-parse", "HEAD:hp23-ok"], + ) + .trim() + .to_owned(); + let miss_tree = hp22_ssh_host_git( + case_dir, + &["-C", "hp23-source", "rev-parse", "HEAD:hp23-miss"], + ) + .trim() + .to_owned(); + let missing_tree = hp22_ssh_host_git( + case_dir, + &["-C", "hp23-source", "rev-parse", "HEAD:hp23-miss/d"], + ) + .trim() + .to_owned(); + let prior_tree = hp22_ssh_host_git(case_dir, &["-C", "hp23-source", "ls-tree", "HEAD"]); + let root_input = prior_tree + .lines() + .map(|line| { + let name = line.rsplit_once('\t').unwrap().1; + let replacement = match name { + "hp23-mode" => Some(&mode_tree), + "hp23-gbk" => Some(&gbk_tree), + "hp23-ok" => Some(&ok_tree), + "hp23-miss" => Some(&miss_tree), + _ => None, + }; + replacement.map_or_else( + || format!("{line}\n"), + |id| format!("040000 tree {id}\t{name}\n"), + ) + }) + .collect::(); + let root_tree = hp23_ssh_git_input(case_dir, &["mktree"], root_input.as_bytes()); + let parent = hp22_ssh_host_git(case_dir, &["-C", "hp23-source", "rev-parse", "HEAD"]) + .trim() + .to_owned(); + let r2 = hp23_ssh_git_input( + case_dir, + &["commit-tree", &root_tree, "-p", &parent, "-m", "HP23 R2"], + b"", + ); + hp22_ssh_host_git( + case_dir, + &["-C", "hp23-source", "update-ref", "refs/heads/main", &r2], + ); + hp22_ssh_host_git( + case_dir, + &[ + "-C", + "hp23-source", + "push", + "--no-thin", + "origin", + "HEAD:refs/heads/main", + ], + ); + with_runtime(async { + let db = Database::connect(env.database.db_url.as_str()) + .await + .unwrap(); + for id in [&mode_tree, &gbk_tree] { + let row = db + .query_one_raw(Statement::from_string( + DatabaseBackend::Postgres, + format!("SELECT sub_trees FROM mega_tree WHERE tree_id = '{id}'"), + )) + .await + .unwrap() + .expect("L0 tree row"); + let bytes: Vec = row.try_get("", "sub_trees").unwrap(); + let actual = git_internal::hash::ObjectHash::from_type_and_data_for_kind( + git_internal::hash::HashKind::Sha1, + git_internal::internal::object::types::ObjectType::Tree, + &bytes, + ) + .unwrap(); + assert_ne!(actual.to_string(), *id, "L0 fixture must be noncanonical"); + } + }); + let mut views = Vec::new(); + for (name, spec) in [ + ("mode", ":/project/hp23-mode"), + ("gbk", ":/project/hp23-gbk:prefix=p"), + ("ok", ":/project/hp23-ok"), + ("miss", ":/project/hp23-miss"), + ] { + let id = hp22_ssh_register(http_port, &format!("hp23-{name}"), spec); + hp22_ssh_wait_lag_zero(http_port, &id); + views.push((name.to_owned(), id)); + } + Hp23SshFixture { + root_tip: r2, + prior_root: parent, + views, + mode_tree, + gbk_tree, + missing_tree, + } +} + +fn hp23_ssh_view<'a>(fixture: &'a Hp23SshFixture, name: &str) -> &'a str { + &fixture.views.iter().find(|(key, _)| key == name).unwrap().1 +} + +fn hp23_ssh_commits(db_url: &str, filter_id: &str) -> Vec { + with_runtime(async { + let db = Database::connect(db_url).await.unwrap(); + db.query_all_raw(Statement::from_string(DatabaseBackend::Postgres, + format!("SELECT m.view_commit FROM mega_view_commit_map m JOIN mega_view_filter f ON f.id = m.filter_pk WHERE f.filter_id = '{filter_id}' AND m.view_commit IS NOT NULL ORDER BY m.seq_from"))) + .await.unwrap().into_iter().map(|row| row.try_get("", "view_commit").unwrap()).collect() + }) +} + +fn hp23_ssh_fetch_request(v2: bool, want: &str, have: Option<&str>, done: bool) -> Vec { + let mut body = Vec::new(); + if v2 { + body.extend(layer3_pkt("command=fetch\n")); + body.extend(b"0001"); + body.extend(layer3_pkt(&format!("want {want}\n"))); + } else { + let capability = if have.is_some() { + " multi_ack_detailed" + } else { + "" + }; + body.extend(layer3_pkt(&format!("want {want}{capability}\n"))); + } + if let Some(have) = have { + body.extend(layer3_pkt(&format!("have {have}\n"))); + } + if v2 && done { + body.extend(layer3_pkt("done\n")); + } + body.extend(b"0000"); + body +} + +fn hp23_ssh_raw( + case_dir: &Path, + port: u16, + id: &str, + v2: bool, + request: &[u8], + err: bool, +) -> (Vec, i32) { + let mut child = layer3_spawn_ssh(case_dir, port, v2, &format!("/.filter/{id}.git")); + let mut stdin = child.stdin.take().unwrap(); + let mut stdout = child.stdout.take().unwrap(); + let advertisement = layer3_read_to_flush(&mut stdout); + assert!(!advertisement.is_empty()); + stdin.write_all(request).unwrap(); + stdin.flush().unwrap(); + drop(stdin); + let response = if err { + layer3_read_pkt(&mut stdout) + } else { + layer3_read_to_flush(&mut stdout) + }; + let mut trailing = Vec::new(); + stdout.read_to_end(&mut trailing).unwrap(); + assert!( + trailing.is_empty(), + "unexpected SSH trailing bytes: {trailing:?}" + ); + let code = child.wait().unwrap().code().unwrap(); + (response, code) +} + +#[test] +fn integration_git_ssh_view_precheck_raw() { + let (env, mut service, http_port, ssh_port, err) = hp22_ssh_boot(); + let fixture = hp23_ssh_fixture(&env, http_port); + let db_url = env.database.db_url.as_str(); + let ok_id = hp23_ssh_view(&fixture, "ok"); + let ok_tip = hp23_ssh_commits(db_url, ok_id).last().unwrap().clone(); + let not_our_ref = format!("ERR upload-pack: not our ref {}\n", fixture.root_tip); + for (v2, have, done) in [ + (false, None, false), + (false, Some(ok_tip.as_str()), false), + (true, None, true), + (true, Some(fixture.prior_root.as_str()), false), + (true, Some(ok_tip.as_str()), false), + ] { + let request = hp23_ssh_fetch_request(v2, &fixture.root_tip, have, done); + let (body, code) = hp23_ssh_raw(&env.case_dir, ssh_port, ok_id, v2, &request, true); + assert_eq!(body, layer3_pkt(¬_our_ref)); + assert_eq!(code, 1); + } + let mut nak = layer3_pkt("acknowledgments\n"); + nak.extend(layer3_pkt("NAK\n")); + nak.extend(b"0000"); + for name in ["ok", "mode", "gbk"] { + let id = hp23_ssh_view(&fixture, name); + let commits = hp23_ssh_commits(db_url, id); + assert!(!commits.is_empty(), "{name}: {commits:?}"); + let want = commits.last().unwrap(); + if name != "ok" { + assert!(commits.len() >= 2, "{name}: {commits:?}"); + let tree = if name == "mode" { + &fixture.mode_tree + } else { + &fixture.gbk_tree + }; + let error = format!( + "ERR view {id} pack aborted: tree {tree} does not match its stored entries\n" + ); + for (v2, have, done) in [ + (false, None, false), + (false, Some(commits[commits.len() - 2].as_str()), false), + (true, None, true), + (true, Some(commits[commits.len() - 2].as_str()), false), + ] { + let request = hp23_ssh_fetch_request(v2, want, have, done); + let (body, code) = hp23_ssh_raw(&env.case_dir, ssh_port, id, v2, &request, true); + assert_eq!(body, layer3_pkt(&error)); + assert_eq!(code, 1); + } + } + let request = hp23_ssh_fetch_request(true, want, Some(&fixture.prior_root), false); + let (body, code) = hp23_ssh_raw(&env.case_dir, ssh_port, id, true, &request, false); + assert_eq!(body, nak); + assert_eq!(code, 0); + } + for rev in ["HEAD", &fixture.prior_root] { + let tree_listing = hp22_ssh_host_git( + &env.case_dir, + &["-C", "hp23-source", "ls-tree", "-r", "-t", rev], + ); + assert_eq!(tree_listing.matches(&fixture.missing_tree).count(), 1); + } + with_runtime(async { + let db = Database::connect(db_url).await.unwrap(); + let deleted = db + .execute_raw(Statement::from_string( + DatabaseBackend::Postgres, + format!( + "DELETE FROM mega_tree WHERE tree_id = '{}'", + fixture.missing_tree + ), + )) + .await + .unwrap(); + assert_eq!(deleted.rows_affected(), 1); + }); + let miss_id = hp23_ssh_view(&fixture, "miss"); + let want = hp23_ssh_commits(db_url, miss_id).last().unwrap().clone(); + let error = format!( + "ERR view {miss_id} pack aborted: tree {} is missing\n", + fixture.missing_tree + ); + for (v2, done) in [(false, false), (true, true)] { + let request = hp23_ssh_fetch_request(v2, &want, None, done); + let (body, code) = hp23_ssh_raw(&env.case_dir, ssh_port, miss_id, v2, &request, true); + assert_eq!(body, layer3_pkt(&error)); + assert_eq!(code, 1); + } + assert!( + service + .shutdown_via_sigint(Duration::from_secs(10)) + .success(), + "{}", + read_log(&err) + ); +} + +#[test] +fn integration_git_ssh_view_precheck_git_client_not_our_ref() { + let (env, mut service, http_port, ssh_port, err) = hp22_ssh_boot(); + let fixture = hp23_ssh_fixture(&env, http_port); + let id = hp23_ssh_view(&fixture, "ok"); + let url = format!( + "{}.filter/{id}.git", + git_cli::mega2_ssh_repo_url(ssh_port, "git") + ); + let ssh_command = git_cli::git_ssh_command(&env.case_dir, ssh_port); + git_cli::assert_git_success( + &git_cli::git_cli_ssh(&env.case_dir, &ssh_command, &["init", "hp23-fetch"]), + "HP-23 SSH fetch client init", + ); + let output = git_cli::git_cli_ssh( + &env.case_dir, + &ssh_command, + &[ + "-C", + "hp23-fetch", + "-c", + "protocol.version=2", + "fetch", + &url, + &fixture.root_tip, + ], + ); + assert!(!output.status.success(), "{output:?}"); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!( + stderr.contains(&format!( + "remote error: upload-pack: not our ref {}", + fixture.root_tip + )), + "{stderr}" + ); + assert!( + service + .shutdown_via_sigint(Duration::from_secs(10)) + .success(), + "{}", + read_log(&err) + ); +} + +#[test] +fn integration_git_ssh_view_precheck_git_client_l0_clone() { + let (env, mut service, http_port, ssh_port, err) = hp22_ssh_boot(); + let fixture = hp23_ssh_fixture(&env, http_port); + let ssh_command = git_cli::git_ssh_command(&env.case_dir, ssh_port); + for (name, id) in &fixture.views { + let url = format!( + "{}.filter/{id}.git", + git_cli::mega2_ssh_repo_url(ssh_port, "git") + ); + let tree_id = if name == "mode" { + &fixture.mode_tree + } else { + &fixture.gbk_tree + }; + for version in ["0", "2"] { + let clone = format!("hp23-{name}-v{version}"); + let output = git_cli::git_cli_ssh( + &env.case_dir, + &ssh_command, + &[ + "-c", + &format!("protocol.version={version}"), + "clone", + &url, + &clone, + ], + ); + let stderr = String::from_utf8_lossy(&output.stderr); + if name == "ok" || name == "miss" { + git_cli::assert_git_success(&output, "HP-23 SSH control clone"); + } else { + assert!(!output.status.success(), "{output:?}"); + assert!( + stderr.contains("remote error:") && stderr.contains(tree_id), + "{stderr}" + ); + assert!( + !stderr.contains("did not send all necessary objects"), + "{stderr}" + ); + } + } + } + assert!( + service + .shutdown_via_sigint(Duration::from_secs(10)) + .success(), + "{}", + read_log(&err) + ); +} + fn hp22_ssh_register(port: u16, name: &str, spec: &str) -> String { let response = reqwest::blocking::Client::builder() .no_proxy() diff --git a/tests/integration_views.rs b/tests/integration_views.rs index d680d1b8..c67e2418 100644 --- a/tests/integration_views.rs +++ b/tests/integration_views.rs @@ -1,6 +1,7 @@ mod common; use std::{ + collections::HashMap, fs, io::{Read, Write}, net::{TcpListener, TcpStream}, @@ -11,6 +12,16 @@ use std::{ time::{Duration, Instant}, }; +use git_internal::{ + hash::{HashKind, ObjectHash}, + internal::object::{ + ObjectTrait, + blob::Blob, + commit::Commit, + tree::{Tree, TreeItem, TreeItemMode}, + types::ObjectType, + }, +}; use reqwest::{Method, StatusCode, blocking::Client}; use sea_orm::{ConnectionTrait, Database, DatabaseBackend, Statement, Value}; use tempfile::TempDir; @@ -89,6 +100,15 @@ struct Case { impl Case { fn boot(enabled: bool, push_auth: &str, anonymous_access: bool) -> Self { + Self::boot_with_env(enabled, push_auth, anonymous_access, &[]) + } + + fn boot_with_env( + enabled: bool, + push_auth: &str, + anonymous_access: bool, + extra_env: &[(&str, &str)], + ) -> Self { let temp = tempfile::tempdir().unwrap(); let database = TestDatabase::create(); let config_path = temp.path().join("config.toml"); @@ -102,6 +122,9 @@ impl Case { .unwrap() .port(); let mut init = Self::command(&temp, &database, &config_path, enabled, port); + for (key, value) in extra_env { + init.env(key, value); + } let result = init.args(["service", "init", "--yes"]).output().unwrap(); assert!( result.status.success(), @@ -111,6 +134,9 @@ impl Case { let stdout = temp.path().join("service.out"); let stderr = temp.path().join("service.err"); let mut command = Self::command(&temp, &database, &config_path, enabled, port); + for (key, value) in extra_env { + command.env(key, value); + } let mut child = command .args([ "service", @@ -542,6 +568,466 @@ impl Drop for Case { } } +struct PrecheckHttpFixture { + case: Case, + root_tip: String, + prior_root: String, + views: HashMap<&'static str, (i64, String, Vec)>, + mode_tree: String, + gbk_tree: String, + missing_tree: String, + missing_tree_refs: usize, +} + +fn hp23_tree(items: Vec, trees: &mut Vec) -> Tree { + let tree = Tree::from_tree_items_with_kind(HashKind::Sha1, items).unwrap(); + trees.push(tree.clone()); + tree +} + +fn hp23_blob(content: &str) -> ObjectHash { + Blob::from_content_bytes_with_kind(HashKind::Sha1, content.as_bytes().to_vec()) + .unwrap() + .id +} + +fn hp23_raw_l0_tree(mode: &str, name: &[u8], blob: ObjectHash) -> (ObjectHash, Vec) { + let mut raw = format!("{mode} ").into_bytes(); + raw.extend_from_slice(name); + raw.push(0); + raw.extend(hex::decode(blob.to_string()).unwrap()); + let id = + ObjectHash::from_type_and_data_for_kind(HashKind::Sha1, ObjectType::Tree, &raw).unwrap(); + let stored = ::from_bytes(&raw, id) + .unwrap() + .to_data() + .unwrap(); + assert_ne!( + ObjectHash::from_type_and_data_for_kind(HashKind::Sha1, ObjectType::Tree, &stored).unwrap(), + id + ); + (id, stored) +} + +fn hp23_pkt(payload: &str) -> Vec { + format!("{:04x}{payload}", payload.len() + 4).into_bytes() +} + +fn hp23_fetch_request(v2: bool, want: &str, have: Option<&str>, done: bool) -> Vec { + let mut body = Vec::new(); + if v2 { + body.extend(hp23_pkt("command=fetch\n")); + body.extend(b"0001"); + body.extend(hp23_pkt(&format!("want {want}\n"))); + } else { + let capability = if have.is_some() { + " multi_ack_detailed" + } else { + "" + }; + body.extend(hp23_pkt(&format!("want {want}{capability}\n"))); + } + if let Some(have) = have { + body.extend(hp23_pkt(&format!("have {have}\n"))); + } + if v2 && done { + body.extend(hp23_pkt("done\n")); + } + body.extend(b"0000"); + body +} + +impl PrecheckHttpFixture { + fn new() -> Self { + let case = Case::boot_with_env( + true, + "none", + true, + &[("MEGA_VIEWS__ALLOW_ANONYMOUS_REGISTER", "true")], + ); + let db_url = case._database.url.clone(); + let (root_tip, prior_root, mode_tree, gbk_tree, missing_tree, missing_tree_refs) = + with_runtime(async { + let db = Database::connect(&db_url).await.unwrap(); + let row = db + .query_one_raw(Statement::from_string( + DatabaseBackend::Postgres, + "SELECT ref_commit_hash FROM mega_refs \ + WHERE path = '/' AND ref_name = 'refs/heads/main' AND NOT is_cl" + .to_owned(), + )) + .await + .unwrap() + .unwrap(); + let r0: String = row.try_get("", "ref_commit_hash").unwrap(); + let r0_hash = ObjectHash::from_hex_for_kind(HashKind::Sha1, &r0).unwrap(); + let mut trees = Vec::new(); + let mode_good = hp23_tree( + vec![TreeItem::new( + TreeItemMode::Blob, + hp23_blob("mode good"), + "good.txt".to_owned(), + )], + &mut trees, + ); + let gbk_good = hp23_tree( + vec![TreeItem::new( + TreeItemMode::Blob, + hp23_blob("gbk good"), + "good.txt".to_owned(), + )], + &mut trees, + ); + let missing = hp23_tree( + vec![TreeItem::new( + TreeItemMode::Blob, + hp23_blob("missing only"), + "hp23-only-in-miss-d.txt".to_owned(), + )], + &mut trees, + ); + let miss = hp23_tree( + vec![TreeItem::new( + TreeItemMode::Tree, + missing.id, + "d".to_owned(), + )], + &mut trees, + ); + let ok = hp23_tree( + vec![TreeItem::new( + TreeItemMode::Blob, + hp23_blob("ok good"), + "ok.txt".to_owned(), + )], + &mut trees, + ); + let hp23_r1 = hp23_tree( + vec![ + TreeItem::new(TreeItemMode::Tree, gbk_good.id, "gbk".to_owned()), + TreeItem::new(TreeItemMode::Tree, miss.id, "miss".to_owned()), + TreeItem::new(TreeItemMode::Tree, mode_good.id, "mode".to_owned()), + TreeItem::new(TreeItemMode::Tree, ok.id, "ok".to_owned()), + ], + &mut trees, + ); + let root_r1 = hp23_tree( + vec![TreeItem::new( + TreeItemMode::Tree, + hp23_r1.id, + "hp23".to_owned(), + )], + &mut trees, + ); + let (mode_id, mode_stored) = + hp23_raw_l0_tree("100664", b"mode.txt", hp23_blob("mode raw")); + let (gbk_id, gbk_stored) = hp23_raw_l0_tree( + "100644", + &[0xc4, 0xe3, 0xba, 0xc3, b'.', b't', b'x', b't'], + hp23_blob("gbk raw"), + ); + let hp23_r2 = hp23_tree( + vec![ + TreeItem::new(TreeItemMode::Tree, gbk_id, "gbk".to_owned()), + TreeItem::new(TreeItemMode::Tree, miss.id, "miss".to_owned()), + TreeItem::new(TreeItemMode::Tree, mode_id, "mode".to_owned()), + TreeItem::new(TreeItemMode::Tree, ok.id, "ok".to_owned()), + ], + &mut trees, + ); + let root_r2 = hp23_tree( + vec![TreeItem::new( + TreeItemMode::Tree, + hp23_r2.id, + "hp23".to_owned(), + )], + &mut trees, + ); + let r1 = Commit::from_tree_id_with_kind( + HashKind::Sha1, + root_r1.id, + vec![r0_hash], + "hp23 R1", + ) + .unwrap(); + let r2 = Commit::from_tree_id_with_kind( + HashKind::Sha1, + root_r2.id, + vec![r1.id], + "hp23 R2", + ) + .unwrap(); + let missing_tree_refs = trees + .iter() + .flat_map(|tree| &tree.tree_items) + .filter(|item| item.id == missing.id) + .count(); + let mut next_id = 8_000_000_i64; + for (tree_id, bytes) in trees + .into_iter() + .map(|tree| (tree.id, tree.to_data().unwrap())) + .chain([(mode_id, mode_stored), (gbk_id, gbk_stored)]) + { + db.execute_raw(Statement::from_sql_and_values( + DatabaseBackend::Postgres, + "INSERT INTO mega_tree \ + (id, tree_id, sub_trees, size, created_at, pack_id, pack_offset, commit_id) \ + VALUES ($1, $2, $3, 0, now(), '', 0, $4) \ + ON CONFLICT (tree_id) DO NOTHING", + [ + Value::from(next_id), + Value::from(tree_id.to_string()), + Value::from(bytes), + Value::from(r2.id.to_string()), + ], + )) + .await + .unwrap(); + next_id += 1; + } + for commit in [&r1, &r2] { + db.execute_raw(Statement::from_sql_and_values( + DatabaseBackend::Postgres, + "INSERT INTO mega_commit \ + (id, commit_id, tree, parents_id, author, committer, content, \ + created_at, pack_id, pack_offset) \ + VALUES ($1, $2, $3, $4, $5, $6, $7, now(), '', 0)", + [ + Value::from(next_id), + Value::from(commit.id.to_string()), + Value::from(commit.tree_id.to_string()), + Value::from(serde_json::json!( + commit + .parent_commit_ids + .iter() + .map(ToString::to_string) + .collect::>() + )), + Value::from( + String::from_utf8(commit.author.to_data().unwrap()).unwrap(), + ), + Value::from( + String::from_utf8(commit.committer.to_data().unwrap()).unwrap(), + ), + Value::from(commit.message.clone()), + ], + )) + .await + .unwrap(); + next_id += 1; + } + let result = db + .execute_raw(Statement::from_sql_and_values( + DatabaseBackend::Postgres, + "UPDATE mega_refs SET ref_commit_hash = $1, ref_tree_hash = $2, \ + updated_at = now() WHERE path = '/' AND ref_name = 'refs/heads/main' \ + AND NOT is_cl AND ref_commit_hash = $3", + [ + Value::from(r2.id.to_string()), + Value::from(root_r2.id.to_string()), + Value::from(r0), + ], + )) + .await + .unwrap(); + assert_eq!(result.rows_affected(), 1); + ( + r2.id.to_string(), + r1.id.to_string(), + mode_id.to_string(), + gbk_id.to_string(), + missing.id.to_string(), + missing_tree_refs, + ) + }); + let mut views = HashMap::new(); + for (name, spec) in [ + ("mode", ":/hp23/mode"), + ("gbk", ":/hp23/gbk:prefix=p"), + ("ok", ":/hp23/ok"), + ("miss", ":/hp23/miss"), + ] { + let (pk, filter_id) = case.register_filter(spec); + let url = case._database.url.clone(); + let commits = with_runtime(async move { + let db = Database::connect(&url).await.unwrap(); + db.query_all_raw(Statement::from_sql_and_values( + DatabaseBackend::Postgres, + "SELECT view_commit FROM mega_view_commit_map \ + WHERE filter_pk = $1 AND view_commit IS NOT NULL ORDER BY seq_from", + [Value::from(pk)], + )) + .await + .unwrap() + .into_iter() + .map(|row| row.try_get("", "view_commit").unwrap()) + .collect::>() + }); + assert!(!commits.is_empty(), "{name}"); + views.insert(name, (pk, filter_id, commits)); + } + Self { + case, + root_tip, + prior_root, + views, + mode_tree, + gbk_tree, + missing_tree, + missing_tree_refs, + } + } + + fn metric(&self) -> u64 { + let response = self + .case + .response(Method::GET, "/api/v1/views/metrics", false, false, None); + assert_eq!(response.status(), StatusCode::OK); + let body: serde_json::Value = response.json().unwrap(); + body["data"]["view_pack_tree_mismatch_total"] + .as_u64() + .unwrap() + } + + fn fetch(&self, name: &str, v2: bool, body: &[u8]) -> Vec { + let filter_id = &self.views[name].1; + let response = self.case.response( + Method::POST, + &format!("/.filter/{filter_id}.git/git-upload-pack"), + v2, + false, + Some(body), + ); + assert_eq!(response.status(), StatusCode::OK); + assert_eq!( + response.headers()["content-type"], + "application/x-git-upload-pack-result" + ); + response.bytes().unwrap().to_vec() + } +} + +#[test] +fn view_precheck_http_want_not_ours() { + let fixture = PrecheckHttpFixture::new(); + let tip = fixture.views["ok"].2.last().unwrap(); + let initial_metric = fixture.metric(); + for (v2, have, done) in [ + (false, None, false), + (false, Some(tip.as_str()), false), + (true, None, true), + (true, Some(fixture.prior_root.as_str()), false), + (true, Some(tip.as_str()), false), + ] { + let body = hp23_fetch_request(v2, &fixture.root_tip, have, done); + assert_eq!( + fixture.fetch("ok", v2, &body), + hp23_pkt(&format!( + "ERR upload-pack: not our ref {}\n", + fixture.root_tip + )) + ); + } + let body = hp23_fetch_request(true, tip, Some(&fixture.prior_root), false); + let mut expected = hp23_pkt("acknowledgments\n"); + expected.extend(hp23_pkt("NAK\n")); + expected.extend(b"0000"); + assert_eq!(fixture.fetch("ok", true, &body), expected); + assert_eq!(fixture.metric(), initial_metric); +} + +#[test] +fn view_precheck_http_l0_mismatch() { + let fixture = PrecheckHttpFixture::new(); + for (name, tree_id) in [("mode", &fixture.mode_tree), ("gbk", &fixture.gbk_tree)] { + let (_, filter_id, commits) = &fixture.views[name]; + assert_eq!(commits.len(), 2); + let want = &commits[1]; + let have = &commits[0]; + let message = format!( + "ERR view {filter_id} pack aborted: tree {tree_id} does not match its stored entries\n" + ); + for (v2, have, done) in [ + (false, None, false), + (false, Some(have.as_str()), false), + (true, None, true), + (true, Some(have.as_str()), false), + ] { + let before = fixture.metric(); + let body = hp23_fetch_request(v2, want, have, done); + assert_eq!(fixture.fetch(name, v2, &body), hp23_pkt(&message)); + assert_eq!(fixture.metric(), before + 1); + for v2 in [false, true] { + let response = fixture.case.response( + Method::GET, + &format!("/.filter/{filter_id}.git/info/refs?service=git-upload-pack"), + v2, + false, + None, + ); + assert_eq!(response.status(), StatusCode::OK); + } + } + let before = fixture.metric(); + let body = hp23_fetch_request(true, want, Some(&fixture.prior_root), false); + let mut expected = hp23_pkt("acknowledgments\n"); + expected.extend(hp23_pkt("NAK\n")); + expected.extend(b"0000"); + assert_eq!(fixture.fetch(name, true, &body), expected); + assert_eq!(fixture.metric(), before); + let body = hp23_fetch_request(true, want, Some(&fixture.prior_root), true); + assert_eq!(fixture.fetch(name, true, &body), hp23_pkt(&message)); + assert_eq!(fixture.metric(), before + 1); + } +} + +#[test] +fn view_precheck_http_missing_tree() { + let fixture = PrecheckHttpFixture::new(); + assert_eq!(fixture.missing_tree_refs, 1); + let (_, filter_id, commits) = &fixture.views["miss"]; + let tree_id = &fixture.missing_tree; + let url = fixture.case._database.url.clone(); + let id = tree_id.clone(); + with_runtime(async move { + let db = Database::connect(&url).await.unwrap(); + let row = db + .query_one_raw(Statement::from_sql_and_values( + DatabaseBackend::Postgres, + "SELECT COUNT(*)::bigint AS count FROM mega_tree WHERE tree_id = $1", + [Value::from(id.clone())], + )) + .await + .unwrap() + .unwrap(); + let count: i64 = row.try_get("", "count").unwrap(); + assert_eq!(count, 1); + db.execute_raw(Statement::from_sql_and_values( + DatabaseBackend::Postgres, + "DELETE FROM mega_tree WHERE tree_id = $1", + [Value::from(id)], + )) + .await + .unwrap(); + }); + let metric = fixture.metric(); + let message = format!("ERR view {filter_id} pack aborted: tree {tree_id} is missing\n"); + let tip = commits.last().unwrap(); + for v2 in [false, true] { + let body = hp23_fetch_request(v2, tip, None, v2); + assert_eq!(fixture.fetch("miss", v2, &body), hp23_pkt(&message)); + assert_eq!(fixture.metric(), metric); + let response = fixture.case.response( + Method::GET, + &format!("/.filter/{filter_id}.git/info/refs?service=git-upload-pack"), + v2, + false, + None, + ); + assert_eq!(response.status(), StatusCode::OK); + } +} + #[test] fn layer2_disabled_not_found() { let case = Case::boot(false, "token", false); From 7acc2b569475c137f905496438027fa174b002f0 Mon Sep 17 00:00:00 2001 From: Eli Ma Date: Thu, 8 Oct 2026 03:16:49 +0800 Subject: [PATCH 3/5] HP-24: freeze view_bench params Signed-off-by: Eli Ma --- examples/view_bench.params.json | 1449 +++++++++++++++++++++++++++++++ 1 file changed, 1449 insertions(+) create mode 100644 examples/view_bench.params.json diff --git a/examples/view_bench.params.json b/examples/view_bench.params.json new file mode 100644 index 00000000..be01ad2b --- /dev/null +++ b/examples/view_bench.params.json @@ -0,0 +1,1449 @@ +{ + "data": { + "tag": "v0.99.5", + "commit": "3857284f7b892f855edffc5b9c196a0dd74b1b7d", + "format": "pack", + "path": "target/tmp/view_bench/data/git-v0.99.5.pack", + "sha256": "812723cd7e7c540101e3cb297a7dc4f14f18f407cd68d76452e97321e2992003", + "h_max": 1231, + "git_version": "git version 2.54.0 (Apple Git-157)" + }, + "views_config": { + "batch_size": 1000 + }, + "views": [ + ":/Documentation", + ":exclude[::t/]" + ], + "probes": [ + { + "in_view": [ + { + "path": "Documentation/view-bench-in_view-v0-00.txt", + "content": "view benchmark in_view view 0 sample 0\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-01.txt", + "content": "view benchmark in_view view 0 sample 1\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-02.txt", + "content": "view benchmark in_view view 0 sample 2\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-03.txt", + "content": "view benchmark in_view view 0 sample 3\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-04.txt", + "content": "view benchmark in_view view 0 sample 4\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-05.txt", + "content": "view benchmark in_view view 0 sample 5\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-06.txt", + "content": "view benchmark in_view view 0 sample 6\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-07.txt", + "content": "view benchmark in_view view 0 sample 7\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-08.txt", + "content": "view benchmark in_view view 0 sample 8\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-09.txt", + "content": "view benchmark in_view view 0 sample 9\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-10.txt", + "content": "view benchmark in_view view 0 sample 10\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-11.txt", + "content": "view benchmark in_view view 0 sample 11\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-12.txt", + "content": "view benchmark in_view view 0 sample 12\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-13.txt", + "content": "view benchmark in_view view 0 sample 13\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-14.txt", + "content": "view benchmark in_view view 0 sample 14\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-15.txt", + "content": "view benchmark in_view view 0 sample 15\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-16.txt", + "content": "view benchmark in_view view 0 sample 16\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-17.txt", + "content": "view benchmark in_view view 0 sample 17\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-18.txt", + "content": "view benchmark in_view view 0 sample 18\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-19.txt", + "content": "view benchmark in_view view 0 sample 19\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-20.txt", + "content": "view benchmark in_view view 0 sample 20\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-21.txt", + "content": "view benchmark in_view view 0 sample 21\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-22.txt", + "content": "view benchmark in_view view 0 sample 22\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-23.txt", + "content": "view benchmark in_view view 0 sample 23\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-24.txt", + "content": "view benchmark in_view view 0 sample 24\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-25.txt", + "content": "view benchmark in_view view 0 sample 25\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-26.txt", + "content": "view benchmark in_view view 0 sample 26\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-27.txt", + "content": "view benchmark in_view view 0 sample 27\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-28.txt", + "content": "view benchmark in_view view 0 sample 28\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-29.txt", + "content": "view benchmark in_view view 0 sample 29\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-30.txt", + "content": "view benchmark in_view view 0 sample 30\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-31.txt", + "content": "view benchmark in_view view 0 sample 31\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-32.txt", + "content": "view benchmark in_view view 0 sample 32\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-33.txt", + "content": "view benchmark in_view view 0 sample 33\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-34.txt", + "content": "view benchmark in_view view 0 sample 34\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-35.txt", + "content": "view benchmark in_view view 0 sample 35\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-36.txt", + "content": "view benchmark in_view view 0 sample 36\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-37.txt", + "content": "view benchmark in_view view 0 sample 37\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-38.txt", + "content": "view benchmark in_view view 0 sample 38\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-39.txt", + "content": "view benchmark in_view view 0 sample 39\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-40.txt", + "content": "view benchmark in_view view 0 sample 40\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-41.txt", + "content": "view benchmark in_view view 0 sample 41\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-42.txt", + "content": "view benchmark in_view view 0 sample 42\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-43.txt", + "content": "view benchmark in_view view 0 sample 43\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-44.txt", + "content": "view benchmark in_view view 0 sample 44\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-45.txt", + "content": "view benchmark in_view view 0 sample 45\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-46.txt", + "content": "view benchmark in_view view 0 sample 46\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-47.txt", + "content": "view benchmark in_view view 0 sample 47\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-48.txt", + "content": "view benchmark in_view view 0 sample 48\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-49.txt", + "content": "view benchmark in_view view 0 sample 49\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-50.txt", + "content": "view benchmark in_view view 0 sample 50\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-51.txt", + "content": "view benchmark in_view view 0 sample 51\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-52.txt", + "content": "view benchmark in_view view 0 sample 52\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-53.txt", + "content": "view benchmark in_view view 0 sample 53\n" + }, + { + "path": "Documentation/view-bench-in_view-v0-54.txt", + "content": "view benchmark in_view view 0 sample 54\n" + } + ], + "outside": [ + { + "path": "t/view-bench-outside-v0-00.txt", + "content": "view benchmark outside view 0 sample 0\n" + }, + { + "path": "t/view-bench-outside-v0-01.txt", + "content": "view benchmark outside view 0 sample 1\n" + }, + { + "path": "t/view-bench-outside-v0-02.txt", + "content": "view benchmark outside view 0 sample 2\n" + }, + { + "path": "t/view-bench-outside-v0-03.txt", + "content": "view benchmark outside view 0 sample 3\n" + }, + { + "path": "t/view-bench-outside-v0-04.txt", + "content": "view benchmark outside view 0 sample 4\n" + }, + { + "path": "t/view-bench-outside-v0-05.txt", + "content": "view benchmark outside view 0 sample 5\n" + }, + { + "path": "t/view-bench-outside-v0-06.txt", + "content": "view benchmark outside view 0 sample 6\n" + }, + { + "path": "t/view-bench-outside-v0-07.txt", + "content": "view benchmark outside view 0 sample 7\n" + }, + { + "path": "t/view-bench-outside-v0-08.txt", + "content": "view benchmark outside view 0 sample 8\n" + }, + { + "path": "t/view-bench-outside-v0-09.txt", + "content": "view benchmark outside view 0 sample 9\n" + }, + { + "path": "t/view-bench-outside-v0-10.txt", + "content": "view benchmark outside view 0 sample 10\n" + }, + { + "path": "t/view-bench-outside-v0-11.txt", + "content": "view benchmark outside view 0 sample 11\n" + }, + { + "path": "t/view-bench-outside-v0-12.txt", + "content": "view benchmark outside view 0 sample 12\n" + }, + { + "path": "t/view-bench-outside-v0-13.txt", + "content": "view benchmark outside view 0 sample 13\n" + }, + { + "path": "t/view-bench-outside-v0-14.txt", + "content": "view benchmark outside view 0 sample 14\n" + }, + { + "path": "t/view-bench-outside-v0-15.txt", + "content": "view benchmark outside view 0 sample 15\n" + }, + { + "path": "t/view-bench-outside-v0-16.txt", + "content": "view benchmark outside view 0 sample 16\n" + }, + { + "path": "t/view-bench-outside-v0-17.txt", + "content": "view benchmark outside view 0 sample 17\n" + }, + { + "path": "t/view-bench-outside-v0-18.txt", + "content": "view benchmark outside view 0 sample 18\n" + }, + { + "path": "t/view-bench-outside-v0-19.txt", + "content": "view benchmark outside view 0 sample 19\n" + }, + { + "path": "t/view-bench-outside-v0-20.txt", + "content": "view benchmark outside view 0 sample 20\n" + }, + { + "path": "t/view-bench-outside-v0-21.txt", + "content": "view benchmark outside view 0 sample 21\n" + }, + { + "path": "t/view-bench-outside-v0-22.txt", + "content": "view benchmark outside view 0 sample 22\n" + }, + { + "path": "t/view-bench-outside-v0-23.txt", + "content": "view benchmark outside view 0 sample 23\n" + }, + { + "path": "t/view-bench-outside-v0-24.txt", + "content": "view benchmark outside view 0 sample 24\n" + }, + { + "path": "t/view-bench-outside-v0-25.txt", + "content": "view benchmark outside view 0 sample 25\n" + }, + { + "path": "t/view-bench-outside-v0-26.txt", + "content": "view benchmark outside view 0 sample 26\n" + }, + { + "path": "t/view-bench-outside-v0-27.txt", + "content": "view benchmark outside view 0 sample 27\n" + }, + { + "path": "t/view-bench-outside-v0-28.txt", + "content": "view benchmark outside view 0 sample 28\n" + }, + { + "path": "t/view-bench-outside-v0-29.txt", + "content": "view benchmark outside view 0 sample 29\n" + }, + { + "path": "t/view-bench-outside-v0-30.txt", + "content": "view benchmark outside view 0 sample 30\n" + }, + { + "path": "t/view-bench-outside-v0-31.txt", + "content": "view benchmark outside view 0 sample 31\n" + }, + { + "path": "t/view-bench-outside-v0-32.txt", + "content": "view benchmark outside view 0 sample 32\n" + }, + { + "path": "t/view-bench-outside-v0-33.txt", + "content": "view benchmark outside view 0 sample 33\n" + }, + { + "path": "t/view-bench-outside-v0-34.txt", + "content": "view benchmark outside view 0 sample 34\n" + }, + { + "path": "t/view-bench-outside-v0-35.txt", + "content": "view benchmark outside view 0 sample 35\n" + }, + { + "path": "t/view-bench-outside-v0-36.txt", + "content": "view benchmark outside view 0 sample 36\n" + }, + { + "path": "t/view-bench-outside-v0-37.txt", + "content": "view benchmark outside view 0 sample 37\n" + }, + { + "path": "t/view-bench-outside-v0-38.txt", + "content": "view benchmark outside view 0 sample 38\n" + }, + { + "path": "t/view-bench-outside-v0-39.txt", + "content": "view benchmark outside view 0 sample 39\n" + }, + { + "path": "t/view-bench-outside-v0-40.txt", + "content": "view benchmark outside view 0 sample 40\n" + }, + { + "path": "t/view-bench-outside-v0-41.txt", + "content": "view benchmark outside view 0 sample 41\n" + }, + { + "path": "t/view-bench-outside-v0-42.txt", + "content": "view benchmark outside view 0 sample 42\n" + }, + { + "path": "t/view-bench-outside-v0-43.txt", + "content": "view benchmark outside view 0 sample 43\n" + }, + { + "path": "t/view-bench-outside-v0-44.txt", + "content": "view benchmark outside view 0 sample 44\n" + }, + { + "path": "t/view-bench-outside-v0-45.txt", + "content": "view benchmark outside view 0 sample 45\n" + }, + { + "path": "t/view-bench-outside-v0-46.txt", + "content": "view benchmark outside view 0 sample 46\n" + }, + { + "path": "t/view-bench-outside-v0-47.txt", + "content": "view benchmark outside view 0 sample 47\n" + }, + { + "path": "t/view-bench-outside-v0-48.txt", + "content": "view benchmark outside view 0 sample 48\n" + }, + { + "path": "t/view-bench-outside-v0-49.txt", + "content": "view benchmark outside view 0 sample 49\n" + }, + { + "path": "t/view-bench-outside-v0-50.txt", + "content": "view benchmark outside view 0 sample 50\n" + }, + { + "path": "t/view-bench-outside-v0-51.txt", + "content": "view benchmark outside view 0 sample 51\n" + }, + { + "path": "t/view-bench-outside-v0-52.txt", + "content": "view benchmark outside view 0 sample 52\n" + }, + { + "path": "t/view-bench-outside-v0-53.txt", + "content": "view benchmark outside view 0 sample 53\n" + }, + { + "path": "t/view-bench-outside-v0-54.txt", + "content": "view benchmark outside view 0 sample 54\n" + } + ], + "t_only": [ + { + "path": "t/view-bench-t_only-v0-00.txt", + "content": "view benchmark t_only view 0 sample 0\n" + }, + { + "path": "t/view-bench-t_only-v0-01.txt", + "content": "view benchmark t_only view 0 sample 1\n" + }, + { + "path": "t/view-bench-t_only-v0-02.txt", + "content": "view benchmark t_only view 0 sample 2\n" + }, + { + "path": "t/view-bench-t_only-v0-03.txt", + "content": "view benchmark t_only view 0 sample 3\n" + }, + { + "path": "t/view-bench-t_only-v0-04.txt", + "content": "view benchmark t_only view 0 sample 4\n" + }, + { + "path": "t/view-bench-t_only-v0-05.txt", + "content": "view benchmark t_only view 0 sample 5\n" + }, + { + "path": "t/view-bench-t_only-v0-06.txt", + "content": "view benchmark t_only view 0 sample 6\n" + }, + { + "path": "t/view-bench-t_only-v0-07.txt", + "content": "view benchmark t_only view 0 sample 7\n" + }, + { + "path": "t/view-bench-t_only-v0-08.txt", + "content": "view benchmark t_only view 0 sample 8\n" + }, + { + "path": "t/view-bench-t_only-v0-09.txt", + "content": "view benchmark t_only view 0 sample 9\n" + }, + { + "path": "t/view-bench-t_only-v0-10.txt", + "content": "view benchmark t_only view 0 sample 10\n" + }, + { + "path": "t/view-bench-t_only-v0-11.txt", + "content": "view benchmark t_only view 0 sample 11\n" + }, + { + "path": "t/view-bench-t_only-v0-12.txt", + "content": "view benchmark t_only view 0 sample 12\n" + }, + { + "path": "t/view-bench-t_only-v0-13.txt", + "content": "view benchmark t_only view 0 sample 13\n" + }, + { + "path": "t/view-bench-t_only-v0-14.txt", + "content": "view benchmark t_only view 0 sample 14\n" + }, + { + "path": "t/view-bench-t_only-v0-15.txt", + "content": "view benchmark t_only view 0 sample 15\n" + }, + { + "path": "t/view-bench-t_only-v0-16.txt", + "content": "view benchmark t_only view 0 sample 16\n" + }, + { + "path": "t/view-bench-t_only-v0-17.txt", + "content": "view benchmark t_only view 0 sample 17\n" + }, + { + "path": "t/view-bench-t_only-v0-18.txt", + "content": "view benchmark t_only view 0 sample 18\n" + }, + { + "path": "t/view-bench-t_only-v0-19.txt", + "content": "view benchmark t_only view 0 sample 19\n" + }, + { + "path": "t/view-bench-t_only-v0-20.txt", + "content": "view benchmark t_only view 0 sample 20\n" + }, + { + "path": "t/view-bench-t_only-v0-21.txt", + "content": "view benchmark t_only view 0 sample 21\n" + }, + { + "path": "t/view-bench-t_only-v0-22.txt", + "content": "view benchmark t_only view 0 sample 22\n" + }, + { + "path": "t/view-bench-t_only-v0-23.txt", + "content": "view benchmark t_only view 0 sample 23\n" + }, + { + "path": "t/view-bench-t_only-v0-24.txt", + "content": "view benchmark t_only view 0 sample 24\n" + }, + { + "path": "t/view-bench-t_only-v0-25.txt", + "content": "view benchmark t_only view 0 sample 25\n" + }, + { + "path": "t/view-bench-t_only-v0-26.txt", + "content": "view benchmark t_only view 0 sample 26\n" + }, + { + "path": "t/view-bench-t_only-v0-27.txt", + "content": "view benchmark t_only view 0 sample 27\n" + }, + { + "path": "t/view-bench-t_only-v0-28.txt", + "content": "view benchmark t_only view 0 sample 28\n" + }, + { + "path": "t/view-bench-t_only-v0-29.txt", + "content": "view benchmark t_only view 0 sample 29\n" + }, + { + "path": "t/view-bench-t_only-v0-30.txt", + "content": "view benchmark t_only view 0 sample 30\n" + }, + { + "path": "t/view-bench-t_only-v0-31.txt", + "content": "view benchmark t_only view 0 sample 31\n" + }, + { + "path": "t/view-bench-t_only-v0-32.txt", + "content": "view benchmark t_only view 0 sample 32\n" + }, + { + "path": "t/view-bench-t_only-v0-33.txt", + "content": "view benchmark t_only view 0 sample 33\n" + }, + { + "path": "t/view-bench-t_only-v0-34.txt", + "content": "view benchmark t_only view 0 sample 34\n" + }, + { + "path": "t/view-bench-t_only-v0-35.txt", + "content": "view benchmark t_only view 0 sample 35\n" + }, + { + "path": "t/view-bench-t_only-v0-36.txt", + "content": "view benchmark t_only view 0 sample 36\n" + }, + { + "path": "t/view-bench-t_only-v0-37.txt", + "content": "view benchmark t_only view 0 sample 37\n" + }, + { + "path": "t/view-bench-t_only-v0-38.txt", + "content": "view benchmark t_only view 0 sample 38\n" + }, + { + "path": "t/view-bench-t_only-v0-39.txt", + "content": "view benchmark t_only view 0 sample 39\n" + }, + { + "path": "t/view-bench-t_only-v0-40.txt", + "content": "view benchmark t_only view 0 sample 40\n" + }, + { + "path": "t/view-bench-t_only-v0-41.txt", + "content": "view benchmark t_only view 0 sample 41\n" + }, + { + "path": "t/view-bench-t_only-v0-42.txt", + "content": "view benchmark t_only view 0 sample 42\n" + }, + { + "path": "t/view-bench-t_only-v0-43.txt", + "content": "view benchmark t_only view 0 sample 43\n" + }, + { + "path": "t/view-bench-t_only-v0-44.txt", + "content": "view benchmark t_only view 0 sample 44\n" + }, + { + "path": "t/view-bench-t_only-v0-45.txt", + "content": "view benchmark t_only view 0 sample 45\n" + }, + { + "path": "t/view-bench-t_only-v0-46.txt", + "content": "view benchmark t_only view 0 sample 46\n" + }, + { + "path": "t/view-bench-t_only-v0-47.txt", + "content": "view benchmark t_only view 0 sample 47\n" + }, + { + "path": "t/view-bench-t_only-v0-48.txt", + "content": "view benchmark t_only view 0 sample 48\n" + }, + { + "path": "t/view-bench-t_only-v0-49.txt", + "content": "view benchmark t_only view 0 sample 49\n" + }, + { + "path": "t/view-bench-t_only-v0-50.txt", + "content": "view benchmark t_only view 0 sample 50\n" + }, + { + "path": "t/view-bench-t_only-v0-51.txt", + "content": "view benchmark t_only view 0 sample 51\n" + }, + { + "path": "t/view-bench-t_only-v0-52.txt", + "content": "view benchmark t_only view 0 sample 52\n" + }, + { + "path": "t/view-bench-t_only-v0-53.txt", + "content": "view benchmark t_only view 0 sample 53\n" + }, + { + "path": "t/view-bench-t_only-v0-54.txt", + "content": "view benchmark t_only view 0 sample 54\n" + } + ] + }, + { + "in_view": [ + { + "path": "Documentation/view-bench-in_view-v1-00.txt", + "content": "view benchmark in_view view 1 sample 0\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-01.txt", + "content": "view benchmark in_view view 1 sample 1\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-02.txt", + "content": "view benchmark in_view view 1 sample 2\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-03.txt", + "content": "view benchmark in_view view 1 sample 3\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-04.txt", + "content": "view benchmark in_view view 1 sample 4\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-05.txt", + "content": "view benchmark in_view view 1 sample 5\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-06.txt", + "content": "view benchmark in_view view 1 sample 6\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-07.txt", + "content": "view benchmark in_view view 1 sample 7\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-08.txt", + "content": "view benchmark in_view view 1 sample 8\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-09.txt", + "content": "view benchmark in_view view 1 sample 9\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-10.txt", + "content": "view benchmark in_view view 1 sample 10\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-11.txt", + "content": "view benchmark in_view view 1 sample 11\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-12.txt", + "content": "view benchmark in_view view 1 sample 12\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-13.txt", + "content": "view benchmark in_view view 1 sample 13\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-14.txt", + "content": "view benchmark in_view view 1 sample 14\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-15.txt", + "content": "view benchmark in_view view 1 sample 15\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-16.txt", + "content": "view benchmark in_view view 1 sample 16\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-17.txt", + "content": "view benchmark in_view view 1 sample 17\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-18.txt", + "content": "view benchmark in_view view 1 sample 18\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-19.txt", + "content": "view benchmark in_view view 1 sample 19\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-20.txt", + "content": "view benchmark in_view view 1 sample 20\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-21.txt", + "content": "view benchmark in_view view 1 sample 21\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-22.txt", + "content": "view benchmark in_view view 1 sample 22\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-23.txt", + "content": "view benchmark in_view view 1 sample 23\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-24.txt", + "content": "view benchmark in_view view 1 sample 24\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-25.txt", + "content": "view benchmark in_view view 1 sample 25\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-26.txt", + "content": "view benchmark in_view view 1 sample 26\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-27.txt", + "content": "view benchmark in_view view 1 sample 27\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-28.txt", + "content": "view benchmark in_view view 1 sample 28\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-29.txt", + "content": "view benchmark in_view view 1 sample 29\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-30.txt", + "content": "view benchmark in_view view 1 sample 30\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-31.txt", + "content": "view benchmark in_view view 1 sample 31\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-32.txt", + "content": "view benchmark in_view view 1 sample 32\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-33.txt", + "content": "view benchmark in_view view 1 sample 33\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-34.txt", + "content": "view benchmark in_view view 1 sample 34\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-35.txt", + "content": "view benchmark in_view view 1 sample 35\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-36.txt", + "content": "view benchmark in_view view 1 sample 36\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-37.txt", + "content": "view benchmark in_view view 1 sample 37\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-38.txt", + "content": "view benchmark in_view view 1 sample 38\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-39.txt", + "content": "view benchmark in_view view 1 sample 39\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-40.txt", + "content": "view benchmark in_view view 1 sample 40\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-41.txt", + "content": "view benchmark in_view view 1 sample 41\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-42.txt", + "content": "view benchmark in_view view 1 sample 42\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-43.txt", + "content": "view benchmark in_view view 1 sample 43\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-44.txt", + "content": "view benchmark in_view view 1 sample 44\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-45.txt", + "content": "view benchmark in_view view 1 sample 45\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-46.txt", + "content": "view benchmark in_view view 1 sample 46\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-47.txt", + "content": "view benchmark in_view view 1 sample 47\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-48.txt", + "content": "view benchmark in_view view 1 sample 48\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-49.txt", + "content": "view benchmark in_view view 1 sample 49\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-50.txt", + "content": "view benchmark in_view view 1 sample 50\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-51.txt", + "content": "view benchmark in_view view 1 sample 51\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-52.txt", + "content": "view benchmark in_view view 1 sample 52\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-53.txt", + "content": "view benchmark in_view view 1 sample 53\n" + }, + { + "path": "Documentation/view-bench-in_view-v1-54.txt", + "content": "view benchmark in_view view 1 sample 54\n" + } + ], + "outside": [ + { + "path": "t/view-bench-outside-v1-00.txt", + "content": "view benchmark outside view 1 sample 0\n" + }, + { + "path": "t/view-bench-outside-v1-01.txt", + "content": "view benchmark outside view 1 sample 1\n" + }, + { + "path": "t/view-bench-outside-v1-02.txt", + "content": "view benchmark outside view 1 sample 2\n" + }, + { + "path": "t/view-bench-outside-v1-03.txt", + "content": "view benchmark outside view 1 sample 3\n" + }, + { + "path": "t/view-bench-outside-v1-04.txt", + "content": "view benchmark outside view 1 sample 4\n" + }, + { + "path": "t/view-bench-outside-v1-05.txt", + "content": "view benchmark outside view 1 sample 5\n" + }, + { + "path": "t/view-bench-outside-v1-06.txt", + "content": "view benchmark outside view 1 sample 6\n" + }, + { + "path": "t/view-bench-outside-v1-07.txt", + "content": "view benchmark outside view 1 sample 7\n" + }, + { + "path": "t/view-bench-outside-v1-08.txt", + "content": "view benchmark outside view 1 sample 8\n" + }, + { + "path": "t/view-bench-outside-v1-09.txt", + "content": "view benchmark outside view 1 sample 9\n" + }, + { + "path": "t/view-bench-outside-v1-10.txt", + "content": "view benchmark outside view 1 sample 10\n" + }, + { + "path": "t/view-bench-outside-v1-11.txt", + "content": "view benchmark outside view 1 sample 11\n" + }, + { + "path": "t/view-bench-outside-v1-12.txt", + "content": "view benchmark outside view 1 sample 12\n" + }, + { + "path": "t/view-bench-outside-v1-13.txt", + "content": "view benchmark outside view 1 sample 13\n" + }, + { + "path": "t/view-bench-outside-v1-14.txt", + "content": "view benchmark outside view 1 sample 14\n" + }, + { + "path": "t/view-bench-outside-v1-15.txt", + "content": "view benchmark outside view 1 sample 15\n" + }, + { + "path": "t/view-bench-outside-v1-16.txt", + "content": "view benchmark outside view 1 sample 16\n" + }, + { + "path": "t/view-bench-outside-v1-17.txt", + "content": "view benchmark outside view 1 sample 17\n" + }, + { + "path": "t/view-bench-outside-v1-18.txt", + "content": "view benchmark outside view 1 sample 18\n" + }, + { + "path": "t/view-bench-outside-v1-19.txt", + "content": "view benchmark outside view 1 sample 19\n" + }, + { + "path": "t/view-bench-outside-v1-20.txt", + "content": "view benchmark outside view 1 sample 20\n" + }, + { + "path": "t/view-bench-outside-v1-21.txt", + "content": "view benchmark outside view 1 sample 21\n" + }, + { + "path": "t/view-bench-outside-v1-22.txt", + "content": "view benchmark outside view 1 sample 22\n" + }, + { + "path": "t/view-bench-outside-v1-23.txt", + "content": "view benchmark outside view 1 sample 23\n" + }, + { + "path": "t/view-bench-outside-v1-24.txt", + "content": "view benchmark outside view 1 sample 24\n" + }, + { + "path": "t/view-bench-outside-v1-25.txt", + "content": "view benchmark outside view 1 sample 25\n" + }, + { + "path": "t/view-bench-outside-v1-26.txt", + "content": "view benchmark outside view 1 sample 26\n" + }, + { + "path": "t/view-bench-outside-v1-27.txt", + "content": "view benchmark outside view 1 sample 27\n" + }, + { + "path": "t/view-bench-outside-v1-28.txt", + "content": "view benchmark outside view 1 sample 28\n" + }, + { + "path": "t/view-bench-outside-v1-29.txt", + "content": "view benchmark outside view 1 sample 29\n" + }, + { + "path": "t/view-bench-outside-v1-30.txt", + "content": "view benchmark outside view 1 sample 30\n" + }, + { + "path": "t/view-bench-outside-v1-31.txt", + "content": "view benchmark outside view 1 sample 31\n" + }, + { + "path": "t/view-bench-outside-v1-32.txt", + "content": "view benchmark outside view 1 sample 32\n" + }, + { + "path": "t/view-bench-outside-v1-33.txt", + "content": "view benchmark outside view 1 sample 33\n" + }, + { + "path": "t/view-bench-outside-v1-34.txt", + "content": "view benchmark outside view 1 sample 34\n" + }, + { + "path": "t/view-bench-outside-v1-35.txt", + "content": "view benchmark outside view 1 sample 35\n" + }, + { + "path": "t/view-bench-outside-v1-36.txt", + "content": "view benchmark outside view 1 sample 36\n" + }, + { + "path": "t/view-bench-outside-v1-37.txt", + "content": "view benchmark outside view 1 sample 37\n" + }, + { + "path": "t/view-bench-outside-v1-38.txt", + "content": "view benchmark outside view 1 sample 38\n" + }, + { + "path": "t/view-bench-outside-v1-39.txt", + "content": "view benchmark outside view 1 sample 39\n" + }, + { + "path": "t/view-bench-outside-v1-40.txt", + "content": "view benchmark outside view 1 sample 40\n" + }, + { + "path": "t/view-bench-outside-v1-41.txt", + "content": "view benchmark outside view 1 sample 41\n" + }, + { + "path": "t/view-bench-outside-v1-42.txt", + "content": "view benchmark outside view 1 sample 42\n" + }, + { + "path": "t/view-bench-outside-v1-43.txt", + "content": "view benchmark outside view 1 sample 43\n" + }, + { + "path": "t/view-bench-outside-v1-44.txt", + "content": "view benchmark outside view 1 sample 44\n" + }, + { + "path": "t/view-bench-outside-v1-45.txt", + "content": "view benchmark outside view 1 sample 45\n" + }, + { + "path": "t/view-bench-outside-v1-46.txt", + "content": "view benchmark outside view 1 sample 46\n" + }, + { + "path": "t/view-bench-outside-v1-47.txt", + "content": "view benchmark outside view 1 sample 47\n" + }, + { + "path": "t/view-bench-outside-v1-48.txt", + "content": "view benchmark outside view 1 sample 48\n" + }, + { + "path": "t/view-bench-outside-v1-49.txt", + "content": "view benchmark outside view 1 sample 49\n" + }, + { + "path": "t/view-bench-outside-v1-50.txt", + "content": "view benchmark outside view 1 sample 50\n" + }, + { + "path": "t/view-bench-outside-v1-51.txt", + "content": "view benchmark outside view 1 sample 51\n" + }, + { + "path": "t/view-bench-outside-v1-52.txt", + "content": "view benchmark outside view 1 sample 52\n" + }, + { + "path": "t/view-bench-outside-v1-53.txt", + "content": "view benchmark outside view 1 sample 53\n" + }, + { + "path": "t/view-bench-outside-v1-54.txt", + "content": "view benchmark outside view 1 sample 54\n" + } + ], + "t_only": [ + { + "path": "t/view-bench-t_only-v1-00.txt", + "content": "view benchmark t_only view 1 sample 0\n" + }, + { + "path": "t/view-bench-t_only-v1-01.txt", + "content": "view benchmark t_only view 1 sample 1\n" + }, + { + "path": "t/view-bench-t_only-v1-02.txt", + "content": "view benchmark t_only view 1 sample 2\n" + }, + { + "path": "t/view-bench-t_only-v1-03.txt", + "content": "view benchmark t_only view 1 sample 3\n" + }, + { + "path": "t/view-bench-t_only-v1-04.txt", + "content": "view benchmark t_only view 1 sample 4\n" + }, + { + "path": "t/view-bench-t_only-v1-05.txt", + "content": "view benchmark t_only view 1 sample 5\n" + }, + { + "path": "t/view-bench-t_only-v1-06.txt", + "content": "view benchmark t_only view 1 sample 6\n" + }, + { + "path": "t/view-bench-t_only-v1-07.txt", + "content": "view benchmark t_only view 1 sample 7\n" + }, + { + "path": "t/view-bench-t_only-v1-08.txt", + "content": "view benchmark t_only view 1 sample 8\n" + }, + { + "path": "t/view-bench-t_only-v1-09.txt", + "content": "view benchmark t_only view 1 sample 9\n" + }, + { + "path": "t/view-bench-t_only-v1-10.txt", + "content": "view benchmark t_only view 1 sample 10\n" + }, + { + "path": "t/view-bench-t_only-v1-11.txt", + "content": "view benchmark t_only view 1 sample 11\n" + }, + { + "path": "t/view-bench-t_only-v1-12.txt", + "content": "view benchmark t_only view 1 sample 12\n" + }, + { + "path": "t/view-bench-t_only-v1-13.txt", + "content": "view benchmark t_only view 1 sample 13\n" + }, + { + "path": "t/view-bench-t_only-v1-14.txt", + "content": "view benchmark t_only view 1 sample 14\n" + }, + { + "path": "t/view-bench-t_only-v1-15.txt", + "content": "view benchmark t_only view 1 sample 15\n" + }, + { + "path": "t/view-bench-t_only-v1-16.txt", + "content": "view benchmark t_only view 1 sample 16\n" + }, + { + "path": "t/view-bench-t_only-v1-17.txt", + "content": "view benchmark t_only view 1 sample 17\n" + }, + { + "path": "t/view-bench-t_only-v1-18.txt", + "content": "view benchmark t_only view 1 sample 18\n" + }, + { + "path": "t/view-bench-t_only-v1-19.txt", + "content": "view benchmark t_only view 1 sample 19\n" + }, + { + "path": "t/view-bench-t_only-v1-20.txt", + "content": "view benchmark t_only view 1 sample 20\n" + }, + { + "path": "t/view-bench-t_only-v1-21.txt", + "content": "view benchmark t_only view 1 sample 21\n" + }, + { + "path": "t/view-bench-t_only-v1-22.txt", + "content": "view benchmark t_only view 1 sample 22\n" + }, + { + "path": "t/view-bench-t_only-v1-23.txt", + "content": "view benchmark t_only view 1 sample 23\n" + }, + { + "path": "t/view-bench-t_only-v1-24.txt", + "content": "view benchmark t_only view 1 sample 24\n" + }, + { + "path": "t/view-bench-t_only-v1-25.txt", + "content": "view benchmark t_only view 1 sample 25\n" + }, + { + "path": "t/view-bench-t_only-v1-26.txt", + "content": "view benchmark t_only view 1 sample 26\n" + }, + { + "path": "t/view-bench-t_only-v1-27.txt", + "content": "view benchmark t_only view 1 sample 27\n" + }, + { + "path": "t/view-bench-t_only-v1-28.txt", + "content": "view benchmark t_only view 1 sample 28\n" + }, + { + "path": "t/view-bench-t_only-v1-29.txt", + "content": "view benchmark t_only view 1 sample 29\n" + }, + { + "path": "t/view-bench-t_only-v1-30.txt", + "content": "view benchmark t_only view 1 sample 30\n" + }, + { + "path": "t/view-bench-t_only-v1-31.txt", + "content": "view benchmark t_only view 1 sample 31\n" + }, + { + "path": "t/view-bench-t_only-v1-32.txt", + "content": "view benchmark t_only view 1 sample 32\n" + }, + { + "path": "t/view-bench-t_only-v1-33.txt", + "content": "view benchmark t_only view 1 sample 33\n" + }, + { + "path": "t/view-bench-t_only-v1-34.txt", + "content": "view benchmark t_only view 1 sample 34\n" + }, + { + "path": "t/view-bench-t_only-v1-35.txt", + "content": "view benchmark t_only view 1 sample 35\n" + }, + { + "path": "t/view-bench-t_only-v1-36.txt", + "content": "view benchmark t_only view 1 sample 36\n" + }, + { + "path": "t/view-bench-t_only-v1-37.txt", + "content": "view benchmark t_only view 1 sample 37\n" + }, + { + "path": "t/view-bench-t_only-v1-38.txt", + "content": "view benchmark t_only view 1 sample 38\n" + }, + { + "path": "t/view-bench-t_only-v1-39.txt", + "content": "view benchmark t_only view 1 sample 39\n" + }, + { + "path": "t/view-bench-t_only-v1-40.txt", + "content": "view benchmark t_only view 1 sample 40\n" + }, + { + "path": "t/view-bench-t_only-v1-41.txt", + "content": "view benchmark t_only view 1 sample 41\n" + }, + { + "path": "t/view-bench-t_only-v1-42.txt", + "content": "view benchmark t_only view 1 sample 42\n" + }, + { + "path": "t/view-bench-t_only-v1-43.txt", + "content": "view benchmark t_only view 1 sample 43\n" + }, + { + "path": "t/view-bench-t_only-v1-44.txt", + "content": "view benchmark t_only view 1 sample 44\n" + }, + { + "path": "t/view-bench-t_only-v1-45.txt", + "content": "view benchmark t_only view 1 sample 45\n" + }, + { + "path": "t/view-bench-t_only-v1-46.txt", + "content": "view benchmark t_only view 1 sample 46\n" + }, + { + "path": "t/view-bench-t_only-v1-47.txt", + "content": "view benchmark t_only view 1 sample 47\n" + }, + { + "path": "t/view-bench-t_only-v1-48.txt", + "content": "view benchmark t_only view 1 sample 48\n" + }, + { + "path": "t/view-bench-t_only-v1-49.txt", + "content": "view benchmark t_only view 1 sample 49\n" + }, + { + "path": "t/view-bench-t_only-v1-50.txt", + "content": "view benchmark t_only view 1 sample 50\n" + }, + { + "path": "t/view-bench-t_only-v1-51.txt", + "content": "view benchmark t_only view 1 sample 51\n" + }, + { + "path": "t/view-bench-t_only-v1-52.txt", + "content": "view benchmark t_only view 1 sample 52\n" + }, + { + "path": "t/view-bench-t_only-v1-53.txt", + "content": "view benchmark t_only view 1 sample 53\n" + }, + { + "path": "t/view-bench-t_only-v1-54.txt", + "content": "view benchmark t_only view 1 sample 54\n" + } + ] + } + ], + "runs": { + "w": 5, + "n": 50 + }, + "thresholds": { + "k_h": 1.5, + "k_b": 1.0, + "c_cold": 50 + }, + "scale_limits": { + "caps": { + "compose_members": 64, + "exclude_selectors": 256, + "k": 64 + }, + "filter": ":[:/benchsrc00:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout00,:/benchsrc01:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout01,:/benchsrc02:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout02,:/benchsrc03:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout03,:/benchsrc04:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout04,:/benchsrc05:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout05,:/benchsrc06:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout06,:/benchsrc07:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout07,:/benchsrc08:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout08,:/benchsrc09:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout09,:/benchsrc10:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout10,:/benchsrc11:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout11,:/benchsrc12:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout12,:/benchsrc13:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout13,:/benchsrc14:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout14,:/benchsrc15:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout15,:/benchsrc16:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout16,:/benchsrc17:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout17,:/benchsrc18:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout18,:/benchsrc19:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout19,:/benchsrc20:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout20,:/benchsrc21:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout21,:/benchsrc22:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout22,:/benchsrc23:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout23,:/benchsrc24:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout24,:/benchsrc25:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout25,:/benchsrc26:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout26,:/benchsrc27:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout27,:/benchsrc28:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout28,:/benchsrc29:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout29,:/benchsrc30:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout30,:/benchsrc31:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout31,:/benchsrc32:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout32,:/benchsrc33:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout33,:/benchsrc34:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout34,:/benchsrc35:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout35,:/benchsrc36:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout36,:/benchsrc37:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout37,:/benchsrc38:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout38,:/benchsrc39:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout39,:/benchsrc40:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout40,:/benchsrc41:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout41,:/benchsrc42:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout42,:/benchsrc43:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout43,:/benchsrc44:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout44,:/benchsrc45:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout45,:/benchsrc46:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout46,:/benchsrc47:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout47,:/benchsrc48:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout48,:/benchsrc49:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout49,:/benchsrc50:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout50,:/benchsrc51:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout51,:/benchsrc52:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout52,:/benchsrc53:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout53,:/benchsrc54:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout54,:/benchsrc55:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout55,:/benchsrc56:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout56,:/benchsrc57:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout57,:/benchsrc58:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout58,:/benchsrc59:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout59,:/benchsrc60:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout60,:/benchsrc61:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout61,:/benchsrc62:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout62,:/benchsrc63:exclude[::hide00/,::hide01/,::hide02/,::hide03/]:prefix=benchout63]", + "probe_paths": [ + "benchsrc00/benchmark.txt", + "benchsrc01/benchmark.txt", + "benchsrc02/benchmark.txt", + "benchsrc03/benchmark.txt", + "benchsrc04/benchmark.txt", + "benchsrc05/benchmark.txt", + "benchsrc06/benchmark.txt", + "benchsrc07/benchmark.txt", + "benchsrc08/benchmark.txt", + "benchsrc09/benchmark.txt", + "benchsrc10/benchmark.txt", + "benchsrc11/benchmark.txt", + "benchsrc12/benchmark.txt", + "benchsrc13/benchmark.txt", + "benchsrc14/benchmark.txt", + "benchsrc15/benchmark.txt", + "benchsrc16/benchmark.txt", + "benchsrc17/benchmark.txt", + "benchsrc18/benchmark.txt", + "benchsrc19/benchmark.txt", + "benchsrc20/benchmark.txt", + "benchsrc21/benchmark.txt", + "benchsrc22/benchmark.txt", + "benchsrc23/benchmark.txt", + "benchsrc24/benchmark.txt", + "benchsrc25/benchmark.txt", + "benchsrc26/benchmark.txt", + "benchsrc27/benchmark.txt", + "benchsrc28/benchmark.txt", + "benchsrc29/benchmark.txt", + "benchsrc30/benchmark.txt", + "benchsrc31/benchmark.txt", + "benchsrc32/benchmark.txt", + "benchsrc33/benchmark.txt", + "benchsrc34/benchmark.txt", + "benchsrc35/benchmark.txt", + "benchsrc36/benchmark.txt", + "benchsrc37/benchmark.txt", + "benchsrc38/benchmark.txt", + "benchsrc39/benchmark.txt", + "benchsrc40/benchmark.txt", + "benchsrc41/benchmark.txt", + "benchsrc42/benchmark.txt", + "benchsrc43/benchmark.txt", + "benchsrc44/benchmark.txt", + "benchsrc45/benchmark.txt", + "benchsrc46/benchmark.txt", + "benchsrc47/benchmark.txt", + "benchsrc48/benchmark.txt", + "benchsrc49/benchmark.txt", + "benchsrc50/benchmark.txt", + "benchsrc51/benchmark.txt", + "benchsrc52/benchmark.txt", + "benchsrc53/benchmark.txt", + "benchsrc54/benchmark.txt", + "benchsrc55/benchmark.txt", + "benchsrc56/benchmark.txt", + "benchsrc57/benchmark.txt", + "benchsrc58/benchmark.txt", + "benchsrc59/benchmark.txt", + "benchsrc60/benchmark.txt", + "benchsrc61/benchmark.txt", + "benchsrc62/benchmark.txt", + "benchsrc63/benchmark.txt" + ], + "content": "view benchmark scale sample {i}\n", + "p95_budget_us": 1000000 + }, + "env": { + "cpu": "Apple M4 Max", + "cores": 16, + "memory": "128 GiB", + "storage": "USB SSD", + "os": "macOS 27.0.1", + "postgres_image": "postgres:18.6-alpine3.24" + } +} From 96853378beaee2f2ed79e644f75257e39f01da00 Mon Sep 17 00:00:00 2001 From: Eli Ma Date: Thu, 8 Oct 2026 04:09:30 +0800 Subject: [PATCH 4/5] v0.42.23: FIX-HP-01 batch root chain and finish projection batches Signed-off-by: Eli Ma --- Cargo.lock | 2 +- Cargo.toml | 2 +- docs/plan/plan-20261002.md | 105 ++++++-- docs/refactoring/history-projection.md | 11 +- .../service/view_projection_service.rs | 58 ++++- src/jupiter/storage/view_root_chain.rs | 233 ++++++++++++++++-- 6 files changed, 367 insertions(+), 44 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index 55b411c8..4e990bf7 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4172,7 +4172,7 @@ checksum = "7ebb8d8732c6a6df3d8f032a82911cfc747e00efb95cc46e8d0acd5b5b88570c" [[package]] name = "mega2" -version = "0.42.22" +version = "0.42.23" dependencies = [ "anyhow", "async-recursion", diff --git a/Cargo.toml b/Cargo.toml index 3484527d..e67221d3 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "mega2" -version = "0.42.22" +version = "0.42.23" edition = "2024" license = "MIT OR Apache-2.0" description = "Monorepo engine and Git hosting server for AI agents: trunk-based storage, Git Smart HTTP, Git LFS, OCI registry, and Agent Session Capture" diff --git a/docs/plan/plan-20261002.md b/docs/plan/plan-20261002.md index 6deda447..f9b44650 100644 --- a/docs/plan/plan-20261002.md +++ b/docs/plan/plan-20261002.md @@ -2,7 +2,7 @@ **模板版本：** v2.1；依据 [`plan-template.md`](plan-template.md)。源码核对日期 2026-10-02；核对时 tip 为 `c9ae6df`（`v0.41.34`）。成稿时另一会话已发布 `fb5ac71`（`v0.41.35: BB-43 ART input validation case`），只改了 `Cargo.toml`、`Cargo.lock`、`docs/deploy-trunk.md`、`docs/plan/plan-20261001.md`、`scripts/artifacts_smoke_storage_only.sh`，不影响本文的源码锚点。工作区仍有该会话的未提交改动，本计划不触碰；每张卡开工时按 ER-01、ER-12 重新读取权威版本。 -**计划状态：** **执行中**。主链已完成 HP-32 与 HP-34；HP-30 承载发布为 `v0.42.7`；HP-13 承载发布为 `v0.42.8`；HP-31 承载发布为 `v0.42.9`；HP-14 由 `v0.42.10` 承载；HP-16 由 `v0.42.11` 承载；HP-17 由 `v0.42.12` 承载；HP-34 由 `v0.42.13` 承载；HP-32 由 `v0.42.14` 承载；HP-15 由 `v0.42.15` 承载；HP-33 由 `v0.42.16` 承载；HP-18 由 `v0.42.17` 承载；HP-19 由 `v0.42.18` 承载；HP-20 由 `v0.42.19` 承载；HP-21 由 `v0.42.20` 承载；HP-22 由 `v0.42.21` 承载；HP-23 由 `v0.42.22` 承载。按剩余依赖顺序从 HP-24 继续。Docker Build / Actions 按发布者指令不等待、不查询。 +**计划状态：** **执行中**。主链已完成 HP-32 与 HP-34；HP-30 承载发布为 `v0.42.7`；HP-13 承载发布为 `v0.42.8`；HP-31 承载发布为 `v0.42.9`；HP-14 由 `v0.42.10` 承载；HP-16 由 `v0.42.11` 承载；HP-17 由 `v0.42.12` 承载；HP-34 由 `v0.42.13` 承载；HP-32 由 `v0.42.14` 承载；HP-15 由 `v0.42.15` 承载；HP-33 由 `v0.42.16` 承载；HP-18 由 `v0.42.17` 承载；HP-19 由 `v0.42.18` 承载；HP-20 由 `v0.42.19` 承载；HP-21 由 `v0.42.20` 承载；HP-22 由 `v0.42.21` 承载；HP-23 由 `v0.42.22` 承载。HP-24 冻结参数提交为 `7acc2b5`，首次有效运行判据 (c)(d) FAIL，现为 blocked；先执行 FIX-HP-01。Docker Build / Actions 按发布者指令不等待、不查询。 **设计契约：** [`docs/refactoring/history-projection.md`](../refactoring/history-projection.md)（下称「设计」，v0.2，经 Codex 五轮审计，见设计附录 F）。本文用「设计 §x.y」引用其条款；设计节号冻结（ADR-HP-01）。 @@ -36,7 +36,7 @@ - 机器接口：视图 URL 的错误按设计 §6.1 错误契约应答（404 / 403 / 503 + `Retry-After` / 200 + ERR；SSH ERR + 退出码 1 / 75）；注册 API 的 400 / 429 + `Retry-After` / 404（关闭时）语义固定；`[views]` 配置的缺省值、取值范围与热加载类别与设计 §6.9 一致。 - 数据状态：清表重建、两次冷启动、两个 worker 并发之后，所有视图 tip 的哈希逐字节相同（设计 §7.5 P0 验收 2）；根链不连续、前提校验失败与缺对象都 fail-closed 并告警，不发布空视图或改写已发布的映射。 - 证据：每张会推送的卡完成 ER-04 的 C 组与 D 组；HP-24 基准按冻结协议满足设计 §7.5 P0 验收 11 的四项判据；HP-26 收口时 `plan-long.md` PT-14 与本计划状态同步。 -- 计划完成：全部 35 张卡（HP-01…HP-35）`Lifecycle=done` 且 `Acceptance=complete`，「完成判据」全部勾选。 +- 计划完成：HP-01…HP-35 与 ER-10 增补的 FIX-HP-01 均 `Lifecycle=done` 且 `Acceptance=complete`，「完成判据」全部勾选。 ## 事实基线 @@ -114,7 +114,7 @@ | 维度 | 结论 | 修订动作 | |---|---|---| | 合理性 | P0 补上设计 §0.6 的缺口 a–d，且不改变 `/.git`。审计结论是 P0 已闭合、可以拆卡（设计附录 F「审计后的结论」） | 只拆 P0；P1/P2/P3 用 `DEFER-HP-*` 登记 | -| 可行性 | 35 张卡均为 S 或 M，按模板 G-03 诚实计数后单卡 AC ≤ 8、VER ≤ 8（docs 卡 ≤ 20）；纯计算、配置、迁移、协议四条链可以并行 | 从「数据 / 状态轴 → 表面轴 → 协议轴」切分（模板「推荐拆分维度」）；R1 后按诚实计数拆出 HP-27…HP-34；R3 从 HP-03 拆出 HP-35 | +| 可行性 | 36 张卡均为 S 或 M，按模板 G-03 诚实计数后单卡 AC ≤ 8、VER ≤ 8（docs 卡 ≤ 20）；纯计算、配置、迁移、协议四条链可以并行 | 从「数据 / 状态轴 → 表面轴 → 协议轴」切分（模板「推荐拆分维度」）；R1 后按诚实计数拆出 HP-27…HP-34；R3 从 HP-03 拆出 HP-35 | | 任务卡粒度 | 没有 L/XL 卡，没有 EX。catch_up 按恢复轴拆成「可失败读取」（HP-29）、「核心 + 整批回滚」（HP-11）、「部分推进停止」（HP-12）与「停服运维」（HP-30）；根链拆成「扩展」（HP-10）与「停追判定」（HP-28）；准入拆成「总数与名额」（HP-13）与「速率」（HP-31）；worker 拆成「补偿」（HP-14）与「信号」（HP-32）；注册 API 拆成「写」（HP-15）与「查询」（HP-33）；错误传输拆成 HTTP 与钩子（HP-17）、SSH（HP-34）；tree 拆成原语（HP-04）与求值（HP-27）；ViewRepo 拆成宣告与 want（HP-19）、打包（HP-22）、预检（HP-23）；入口拆成第 1、2 层（HP-18）与第 3 层（HP-20）；读者触发的重新预热独立为 HP-21；输出路径从可注册判定拆出（HP-03 → HP-35） | 每张卡都写了 Granularity 摘要行，汇总见审计表 | | 依赖与顺序 | DAG 无环；相交的写集都由顺序边串行化 | 「实施顺序」中的写集边单独标注 | | 完整性 | 每张发布卡都含测试与文档；迁移为 forward-only；恢复路径见故障恢复矩阵 | ADR-HP-08 规定停服运维步骤 | @@ -185,6 +185,8 @@ | 2026-10-07 | HP-21 设计阶段归属校准 | 设计 §6.1 第 3 层的读者触发重新预热未标（P1），而 §3.3、§7.5 与 §7.4 有指向 P1 的描述；本计划按 §6.1 与 ADR-HP-08 在 P0 实施。设计「事实校准」第 15 条登记该取舍，正文算法与错误契约不变。 | 无拆卡 | HP-21、设计「事实校准」、ADR-HP-01 | | 2026-10-07 | HP-21 实施、验收与三轮 Claude CLI 复审 | 回收态读请求经准入触发预热与 worker 信号，本次仍按原状态返回暂不可用；四种准入结果、拒绝原因、取消锁等待、热加载、重启恢复和入口闸门均有用例。Claude R1/R2 的测试证据缺口已关闭，R3 `VERDICT: PASS`；Codex 复核通过。8 条精确验证、ER-04 A 组只读 smoke 8/8、nightly 格式、全目标全特性 Clippy、完整 `cargo test --all` 和两个构建门均通过，专用链接参数下构建 0 警告。Docker Build / Actions 按发布者指令不等待、不查询。 | 无拆卡 | HP-21、`deploy-trunk.md`、设计「事实校准」、Review log、ER-04 C/D | | 2026-10-07 | HP-22 打包、验收与三轮 Claude CLI 复审 | ViewRepo 提供只读视图 URL 的 HTTP/SSH v0/v2 clone 与 fetch；Josh 固定期望与本项目增量扩展按 ER-03 分开登记。Claude R1 的闭包与夹具证据缺口已修订，R2/R3 `VERDICT: PASS`；Codex 复核通过。8 条精确验证、ER-04 A 组四项本地只读 smoke、nightly 格式、全目标全特性 Clippy、受控并发的完整 `cargo test --all` 及两个构建门均通过，专用链接参数下构建 0 警告。首次完整测试的一个现有进程级用例遇到回环连接重置，单独复跑与受控并发的第二次完整运行通过。Docker Build / Actions 按发布者指令不等待、不查询。 | 无拆卡 | HP-22、`protocol.md`、双语用户指南与架构、Review log、ER-04 C/D | +| 2026-10-08 | HP-23 发布 | HP-23 已完成并由 `v0.42.22` 承载；发布后从 HP-24 继续。 | 无拆卡 | HP-23、发布窗口、Review log | +| 2026-10-08 | HP-24 冻结基准 FAIL 与 ER-10 修复卡 | 参数冻结提交 `7acc2b5` 已推送；ER-03 冻结前因 v2.0.0 首父链存在非 UTF-8 作者而改选 git.git v0.99.5，已在 HP-24 Performance budget 登记范围与局限；首次有效运行的 (c)(d) FAIL，随后诊断把增量耗时拆为根链与视图两段；HP-24 置 blocked，新增 FIX-HP-01 并立即加前置依赖；先修订设计 §3.3、§4.4，再实施根链批写、无预算单提交快路与末批终态。所有 `run-*.json` 与 `failed-*.json` 保留。 | HP-24 → FIX-HP-01、HP-24 | 实施顺序、发布顺序、共享文件、ER-05、HP-24、审计表、设计 §3.3/§4.4、Review log | ## 已决议设计决策 @@ -202,7 +204,7 @@ ### ADR-HP-02: 模块落点与写集切分 - **Status:** Accepted -- **Context:** 35 张卡里有多条可并行的链，需要事先确定落点，才能声明互不相交的实现写集（G-10）。 +- **Context:** 成稿时的 35 张卡里有多条可并行的链，需要事先确定落点，才能声明互不相交的实现写集（G-10）。 - **Decision:** - 纯计算内核放在 `src/ceres/view/`：`filter/`（语法、规范化、filter_id、语义）、`tree_source.rs`（tree 读取原语与建树）、`tree.rs`（过滤器在 tree 上的求值）、`commit.rs`（字节级重写）、`project.rs`（线性投影规则）。这一层不访问数据库，tree 读取经 `TreeSource` trait 注入；可以使用 `crate::jupiter::utils::converter` 中的纯函数（如 `sort_git_tree_items`），不得使用 `jupiter::storage` 与 `jupiter::service`。 - 存储层放在 `src/jupiter/storage/` 的 `view_*.rs` 文件中，由 `AppService` 聚合并提供 `view_storage()` 访问器（先例：`src/jupiter/storage/mod.rs:73-93`、`:502`、`:518`）。各卡分文件：视图与过滤器行 `view_storage.rs`，根链 `view_root_chain.rs`，数据库 TreeSource `view_tree_source.rs`，投影写入 `view_projection_storage.rs`，准入 `view_admission.rs`，运维 SQL 测试 `view_ops_tests.rs`（`#[cfg(test)]`）。 @@ -224,7 +226,7 @@ ### ADR-HP-04: 默认关闭的开关作为中间发布的安全阀 - **Status:** Accepted -- **Context:** P0 共 35 张卡，每张独立发布（G-07）。中间版本的视图功能不完整，例如 HP-18 发布后、HP-20 之前，合法的视图定位符还没有解析层。 +- **Context:** 成稿时 P0 共 35 张卡，每张独立发布（G-07）。中间版本的视图功能不完整，例如 HP-18 发布后、HP-20 之前，合法的视图定位符还没有解析层。 - **Decision:** `[views].enabled` 缺省为 `false`。关闭时，视图 URL 一律返回 404，注册 API 不挂载，worker 不启动。中间版本的未完成行为都只在 `enabled = true` 时可见，并且每张卡在 `Out of scope` 中写明当时对外的应答（例如 HP-18：合法视图定位符一律 404）。只有三类变化与开关无关、从发布起即生效：保留名（HP-08）；v2 `ls-refs` 零 tip 修正（HP-16）；视图前缀的定位与早期拒绝（HP-18）——`enabled = false` 时 `/.view/…`、`/.filter/…` 的 git URL 与 LFS 请求返回 404，过去它们会落入 Monorepo 的空路径。三者都按各自的兼容说明发布。 - **Alternatives considered:** 用 `REL-*` 家族卡把 P0 合成一次发布。这样窗口太长，无法逐卡回退，也违反 G-07 的默认规则。 - **Consequences:** HP-22 之前，不得在用户文档中宣称视图 URL 可用（GC-HP-08）。 @@ -368,7 +370,7 @@ 7. 打标签：`libra tag -m "v: " v`，再 `libra push origin refs/tags/v`。 8. 发布：人工撰写 release note 后执行 `gh release create v -R gitmono-dev/mega2 --title "v" --notes-file `，禁止 `--generate-notes`。 - **ER-04 D 组：** 本仓只剩 `.github/workflows/docker.yml`：`docker` job，`push: tags: v*`，无 `paths:` 过滤。每次实际 bump 推送 tag 之后，必须取得该 job 的绿灯。`Version increment = N/A` 的卡写 `N/A`。不存在其他远端门。 -- **ER-05：** 每张卡在本地验收后做代码 review，结论必须是 `PASS`。涉及锁、事务或协议的卡要另做一次独立视角的复核（Codex 或第二个 Claude 视角，二者择一），在 Review log 登记。这些卡是 HP-10、HP-28、HP-29、HP-11、HP-12、HP-13、HP-31、HP-14、HP-32、HP-16、HP-17、HP-34、HP-18、HP-19、HP-20、HP-21、HP-22、HP-23、HP-33。HP-16 改 v2 应答；HP-19 在锁内宣告；HP-21、HP-22 走视图协议读路径；HP-33 在准入提交后发信号并读就绪状态。 +- **ER-05：** 每张卡在本地验收后做代码 review，结论必须是 `PASS`。涉及锁、事务或协议的卡要另做一次独立视角的复核（Codex 或第二个 Claude 视角，二者择一），在 Review log 登记。这些卡是 HP-10、HP-28、HP-29、HP-11、HP-12、HP-13、HP-31、HP-14、HP-32、HP-16、HP-17、HP-34、HP-18、HP-19、HP-20、HP-21、HP-22、HP-23、HP-33、FIX-HP-01。HP-16 改 v2 应答；HP-19 在锁内宣告；HP-21、HP-22 走视图协议读路径；HP-33 在准入提交后发信号并读就绪状态；FIX-HP-01 改根链锁内事务与追赶末批终态。 - **ER-07：** 提交用 `libra commit -s`。提交后跑 `libra cat-file -p HEAD`，确认含 `Signed-off-by` trailer；有 `gpgsig` 时一并记录。 - **ER-10 本地化：** 与 Postgres、Redis 测试栈相关的暂时性错误，最多重试 5 次，总时长不超过 30 分钟。另一会话同时在跑 `cargo test` 时，等它结束，不与它并发跑 C 组。实现中发现需要越出本卡写集的修复（例如 HP-24 判定规模上限超出预算），按 ER-10 新建 `FIX-HP-NN` 卡，加一条 `FIX-HP-NN -> 当前卡` 的依赖边，并把当前卡置为 `blocked`。 - **ER-13：** 视图测试在 `test_db_connection` 建的独立 schema 中写库。语句计数与同一 schema 上的多连接池按 GC-HP-10 改用 `test_db_config`，守卫持有到用例结束。进程级黑盒测试按 ADR-HP-06 直接写服务所用的测试库，那是每个用例独立创建的数据库（`tests/integration_git_ssh.rs:31-40` 的 `TestDatabase` 先例），结束时清理。基准（HP-24）自建 schema，并在退出路径（含出错）上删除。 @@ -440,6 +442,10 @@ - `HP-19 -> HP-22` - `HP-22 -> HP-23` - `HP-21 -> HP-23`（写集：`tests/integration_views.rs`） +- `HP-10 -> FIX-HP-01`（根链实现） +- `HP-28 -> FIX-HP-01`（根链停追语义与相交写集） +- `HP-12 -> FIX-HP-01`（追赶的部分推进与相交写集） +- `FIX-HP-01 -> HP-24`（ER-10：冻结基准判据 (c)(d) FAIL） - `HP-12 -> HP-24` - `HP-28 -> HP-24` - `HP-23 -> HP-25` @@ -478,7 +484,7 @@ **发布窗口顺序:** 按依赖拓扑序逐张发布： -`HP-02 → HP-03 → HP-35 → HP-04 → HP-27 → HP-05 → HP-06 → HP-07 → HP-08 → HP-09 → HP-16 → HP-17 → HP-34 → HP-10 → HP-28 → HP-29 → HP-11 → HP-12 → HP-30 → HP-13 → HP-31 → HP-14 → HP-32 → HP-15 → HP-33 → HP-18 → HP-19 → HP-20 → HP-21 → HP-22 → HP-23 → HP-24 → HP-25` +`HP-02 → HP-03 → HP-35 → HP-04 → HP-27 → HP-05 → HP-06 → HP-07 → HP-08 → HP-09 → HP-16 → HP-17 → HP-34 → HP-10 → HP-28 → HP-29 → HP-11 → HP-12 → HP-30 → HP-13 → HP-31 → HP-14 → HP-32 → HP-15 → HP-33 → HP-18 → HP-19 → HP-20 → HP-21 → HP-22 → HP-23 → FIX-HP-01 → HP-24 → HP-25` - 相邻、互不依赖的卡可以按完成先后互换。 - HP-01 随 HP-02 的提交进入仓库；HP-26 是计划收口的 docs 提交（M7）。 @@ -489,7 +495,7 @@ - **协调写集（发布者独占，不进任何卡的实现写集，也不参与并发判定）：** - `docs/plan/plan-20261002.md`：各卡的 Lifecycle、修订历史、Review log，由发布者在该卡的发布窗口内修改； - - `docs/refactoring/history-projection.md`：设计修订（ADR-HP-01），以及 HP-01、HP-24、HP-26 对它的内容改动，也只在该卡的发布窗口内由发布者修改。 + - `docs/refactoring/history-projection.md`：设计修订（ADR-HP-01），以及 HP-01、FIX-HP-01、HP-24、HP-26 对它的内容改动，也只在该卡的发布窗口内由发布者修改。 - **共享文件：** 下列文件由多张卡依次修改，写入顺序已由依赖边固定，禁止并行修改： | 文件 | 写入顺序 | @@ -498,6 +504,8 @@ | `src/jupiter/storage/mod.rs` | HP-08 → HP-10 → HP-29 → HP-11 → HP-30 → HP-13 → HP-14 → HP-32 | | `src/jupiter/tests.rs`（测试辅助） | HP-10 → HP-14 → HP-32 | | `src/jupiter/storage/view_storage.rs` | HP-10 → HP-33 → HP-20 | + | `src/jupiter/storage/view_root_chain.rs` | HP-10 → HP-28 → FIX-HP-01 | + | `src/jupiter/service/view_projection_service.rs` | HP-11 → HP-12 → FIX-HP-01 | | `src/jupiter/storage/view_projection_storage.rs` | HP-11 → HP-12 → HP-19 → HP-22 → HP-23 | | `src/jupiter/service/view_metrics.rs` | HP-11 → HP-12 → HP-23 | | `src/jupiter/service/view_worker.rs` | HP-14 → HP-32 | @@ -609,15 +617,15 @@ **目标:** 基准按冻结协议满足设计 §7.5 P0 验收 11；Libra 的视图 smoke 通过（设计 §7.4 错误契约第 7 项）。 -**卡:** HP-24、HP-25。 +**卡:** FIX-HP-01、HP-24、HP-25。 **进入条件:** -- HP-12、HP-28 完成（HP-24）；HP-23、HP-30 完成且 `DEP-HP-01` 可用（HP-25）。 +- HP-12、HP-28 完成（FIX-HP-01）；FIX-HP-01 完成后 HP-24 才恢复；HP-23、HP-30 完成且 `DEP-HP-01` 可用（HP-25）。 **退出条件:** -- 两张卡均 `done`。 +- 三张卡均 `done`。 ### Phase 7: 收口 @@ -5198,11 +5206,67 @@ ER-04 强制门（不计入 VER）：本卡改 `src/contract/git_protocol/mod.rs **Granularity:** `type=implementation; axis=发包前 fail-closed 预检及其线上应答; recovery=前滚新 patch（降级：[views].enabled=false 并重启，视图 URL 全部 404）; complete=yes; self-contained=yes; AC=8/8; VER=7/8; landing=3; prod-files=3; scope=M; deps=HP-22,HP-21; writeset=序列化于 HP-22,HP-21; release=independent; split-from=N/A; exception=N/A` +### Task FIX-HP-01: 根链扩展与视图追赶的性能修复 + +**Task type:** `implementation` + +**Lifecycle / Acceptance:** `in-progress` / 空 + +**Description:** HP-24 的冻结基准首次有效运行 `run-20261007T192312Z.json` 在判据 (c)、(d) 失败。根链冷启动 H=1231 用 2484 条 SQL；代码逐行写入暂存表与根链表，是 (d) 的主要原因。单提交增量 p95 为 92–100 ms，而 H_max 档的视图冷启动耗时除以名义批数为 33–106 ms，`:/Documentation` 的三类探针违反 (c)。诊断运行 `run-20261007T193500Z.json` 将 H_max 档视图 0 的总计时分段：根链段固定 19 语句，三类样本 p95 为 53–58 ms；视图段 20/24 语句，p95 为 43–49 ms。因此修复同时涉及两段：设计 §3.3 允许批量写入与无预算单提交快路；设计 §4.4 允许末批在水位和就绪已同事务写入后直接返回终态，不再开一笔空的终止检查事务。目标是 H=1231、B=1000 的根链冷启动段 ≤59 语句、单提交快路至多一笔事务，视图末批不再额外开终止事务；(c) 的实际时间仍须在本地提交后、推送前做非判定诊断，因为改动也会缩短冷启动分母，不能从语句数保证通过。实现限于 `view_root_chain.rs` 与 `view_projection_service.rs`，维持锁、预算、冲突、部分推进及停止语义。不得调整冻结参数或放宽阈值。诊断使用临时计时探针，运行后已恢复 `examples/view_bench.rs` 的原 sha256 `a1464857…`；全部失败与运行证据原样保留。 + +**Out of scope:** 过滤器上限与 HP-24 冻结基准本身。本卡发布后若 (c)(d) 仍失败，按 ER-10 新建 FIX-HP-02，HP-24 保持 blocked；不得修改本卡的不可变发布。 + +**Current evidence:** `target/tmp/view_bench/run-20261007T192312Z.json`：H=1231 根链段 2484 语句、视图段 38/41 语句；(c) 三项违反，(d) 2522/2525 > 100；规模上限 PASS。`run-20261007T193500Z.json` 的诊断日志在 `target/tmp/view_bench/diagnostic.log`，三个探针类各 50 个计量样本，分段 p95 见 Description。`src/jupiter/storage/view_root_chain.rs` 的 `insert_scan_rows` 和 `insert_segment` 逐行执行 SQL；`extend_root_chain_inner` 对单提交追加仍经过起扫、回走、接入三笔事务。`ViewProjectionService::catch_up` 在末批已调用 `mark_ready_if_covered` 后仍以 `Advanced` 开下一笔空终止事务。 + +**Acceptance criteria:** +- [ ] lib 级语句计数用例在 H=64 与 H=128、B=1000 的两档证明根链冷启动 SQL 条数相等，且均 ≤29；按每增加一批和每增加一个 1000 行 SQL 分块至多各增加 15 条的上界，推得 H=1231、B=1000 的根链段 ≤59 条，满足 (d) 两视图约 38/41 条的 100 条总预算。 +- [ ] 无预算单提交合法追加时，根链在同一持锁事务内接入并返回 CaughtUp。 +- [ ] 带预算追加的结果、消耗与暂存状态与旧算法相同。 +- [ ] 冲突及不连续输入保留原有 fail-closed 结果与持久证据。 +- [ ] `catch_up` 末批已覆盖 `main@/` 时不再开空终止事务，并返回 Ready。 +- [ ] 末批未覆盖 `main@/` 时返回 MainNotCovered；停止与部分推进结果不变。 +- [ ] 本卡的根链与投影服务定向用例通过，包含语句计数、快路、冲突和终态。 +- [ ] 设计 §3.3 与 §4.4 写明优化的前提、边界和回退路径。 + +**Verification:** +- [ ] `source .env.test && cargo test -p mega2 --lib 'jupiter::storage::view_root_chain::tests::'`：全部通过，含新增批量语句数与快路性质用例。 +- [ ] `source .env.test && cargo test -p mega2 --lib 'jupiter::service::view_projection_service::tests::'`：全部通过，含新增末批终态与语句数用例。 +- [ ] `rg -n '单提交快路|批量 SQL' docs/refactoring/history-projection.md`：§3.3 有命中。 +- [ ] `rg -n '末批终态' docs/refactoring/history-projection.md`：§4.4 有命中。 + +ER-04 C 组五条精确命令和两个零警告构建门另执行，不计入本卡 VER。提交前后分别记录验收树与提交树；HP-24 未提交的接缝与 example 不进入本卡提交，在以本卡提交为 HEAD 的隔离 worktree 中重跑 C 组，确认 exact tree 后再推送。在同一隔离 worktree 中复制 HP-24 的接缝与 example（仅这两处允许出现在 benchmark 的工作树状态），并放入冻结 pack 的独立副本；于推送前运行同一冻结参数的非判定诊断。该 worktree 的 `target/tmp/view_bench/` 与主工作树分离，不把诊断的 `run-*.json` 复制到 HP-24 的判定目录。若 (c)(d) 预计不通过，先继续修订本卡实现与设计，不发布。ER-05 要求 Claude PASS 与独立 Codex PASS。Docker Build Action 按发布者指令不等待、不查询。 + +**Dependencies:** `HP-10`（根链写入）、`HP-28`（停追与冲突语义）、`HP-12`（投影服务的部分推进与停止）；`FIX-HP-01 -> HP-24` 即刻生效。 + +**Deliverables:** `N/A` + +**Implementation write set:** `src/jupiter/storage/view_root_chain.rs`、`src/jupiter/service/view_projection_service.rs`；协调写集 `docs/refactoring/history-projection.md` 与本计划。 + +**Release write set:** `Inherited` + +**Docs and compatibility impact:** 设计 §3.3、§4.4 补充等价快路、批写与末批终态的正确性条件；外部 API 不变。 + +**Rollback mode:** `immutable-release` + +**Security and privacy:** 不新增外部输入、连接串或日志；所有写入仍限于持根链 advisory 锁的事务。 + +**Performance budget:** 只沿用 HP-24 的冻结参数与阈值；不重新冻结。 + +**Estimated scope:** `M` + +**Version increment:** `patch` + +**Release boundary:** `independent` + +**C/D coverage from:** `self`（D 组按发布者指令不等待、不查询）。 + +**Granularity:** `type=implementation; axis=根链与视图追赶性能修复; recovery=前滚 patch; complete=yes; self-contained=yes; AC=8/8; VER=4/8; landing=2; prod-files=2; scope=M; deps=HP-10,HP-28,HP-12; writeset=序列化于 HP-10,HP-28,HP-12; release=independent; split-from=N/A; exception=N/A` + ### Task HP-24: 投影基准门 **Task type:** `implementation` -**Lifecycle / Acceptance:** `pending` / 空 +**Lifecycle / Acceptance:** `blocked` / 空（冻结运行的判据 (c)(d) FAIL；待 `FIX-HP-01` 发布后以同一参数重跑） **Description:** 新增 opt-in 基准 `examples/view_bench.rs` 与冻结参数文件 `examples/view_bench.params.json`（ADR-HP-07）。基准按设计 §7.4「基准」的冻结协议，计量根链扩展与 `catch_up` 的冷启动和单提交增量，判定设计 §7.5 P0 验收 11 的四项判据，并复核设计 §6.5 的过滤器规模上限。唯一的行为轴是「P0 性能门」，G-02 允许独立的性能门单独成卡。范围： - 接缝：`src/lib.rs` 新增 `#[doc(hidden)] pub mod view_bench_ops`，只做转发、包装与装配，不新增写库代码，也不改被测代码。它提供以下能力： @@ -5360,7 +5424,7 @@ ER-04 强制门（不计入 VER）：本卡改 `src/contract/git_protocol/mod.rs - [ ] `source .env.test && test "$(docker compose -p mega2-it -f docker/docker-compose.test.yml exec -T postgres psql -U mega2 -d mega2 -Atc "select count(*) from pg_namespace where starts_with(nspname, 'view_bench_')")" = 0`：退出码 0（AC 8）。第 6 行之后没有残留的 `view_bench_` schema。`psql` 在容器内运行，不依赖宿主客户端，连接串不进证据。 - [ ] `source .env.test && rg -U --pcre2 -n '^### 7\.2 规模估算（纯算术推导，P0 实测见本节末条）$(?:\n(?!### ).*)*\n- \*\*P0 基准实测（HP-24）。\*\*.*(?:\n(?!### |- ).*)*\n### ' docs/refactoring/history-projection.md`：有命中，退出码 0。本行核对 §7.2 的标题已改，实测条目位于 §7.2 内且是该节最后一个顶层条目；`rg` 执行失败时退出码大于 1，同样不通过。由发布者在发布窗口写入设计 §7.2 之后执行。 -**Dependencies:** `HP-12`（消费 `catch_up` 的定型版本：批事务、部分推进与「停止」结果，基准把「已就绪」以外的结果当作出错，以 2 退出；经其上游链，消费 `ViewProjectionService::new(storage, metrics)` 与 `ViewMetrics`、`catch_up` 的结果枚举、`ViewsConfig` 的 `batch_size`、过滤器的解析与规范化（canonical AST、canonical 文本与 `filter_id`）、注册期校验与 `RegistrationCheck`、规模上限常量 `REGISTER_SCALE_LIMITS`，以及 `mega_view_filter` 实体）、`HP-28`（`src/jupiter/storage/view_root_chain.rs` 在 HP-28 之后定型，HP-28 可能就地修正根链扩展在不连续之后的性质；基准计量定型后的 `extend_root_chain` 及其三态结果；经其上游链，消费 `budget = None` 与试锁模式，以及视图表迁移）、`DEP-HP-02`（git.git 数据：只用「归档文件 + sha256」方式，数据文件为原始 pack；开工时写入参数文件的 tag、peeled 提交 id、路径、sha256 与生成所用的 `git --version`） +**Dependencies:** `HP-12`（消费 `catch_up` 的定型版本：批事务、部分推进与「停止」结果，基准把「已就绪」以外的结果当作出错，以 2 退出；经其上游链，消费 `ViewProjectionService::new(storage, metrics)` 与 `ViewMetrics`、`catch_up` 的结果枚举、`ViewsConfig` 的 `batch_size`、过滤器的解析与规范化（canonical AST、canonical 文本与 `filter_id`）、注册期校验与 `RegistrationCheck`、规模上限常量 `REGISTER_SCALE_LIMITS`，以及 `mega_view_filter` 实体）、`HP-28`（`src/jupiter/storage/view_root_chain.rs` 在 HP-28 之后定型，HP-28 可能就地修正根链扩展在不连续之后的性质；基准计量定型后的 `extend_root_chain` 及其三态结果；经其上游链，消费 `budget = None` 与试锁模式，以及视图表迁移）、`FIX-HP-01`（ER-10 前置：冻结运行的 (c)(d) FAIL；修复根链批写、单提交快路与追赶末批终态后，本卡才能用同一冻结参数重跑）、`DEP-HP-02`（git.git 数据：只用「归档文件 + sha256」方式，数据文件为原始 pack；开工时写入参数文件的 tag、peeled 提交 id、路径、sha256 与生成所用的 `git --version`） **Deliverables:** `N/A` @@ -5391,6 +5455,8 @@ ER-04 强制门（不计入 VER）：本卡改 `src/contract/git_protocol/mod.rs **Performance budget:** +**ER-03 冻结前的数据取舍（2026-10-08）。** 数据选 git.git v0.99.5 的 peeled 提交 3857284f7b892f855edffc5b9c196a0dd74b1b7d，首父链 H_max = 1231，三档为 307、615、1231，B = 1000。曾预检 v2.0.0（H_max = 13236），但其首父链第 1326 个提交 7eb93c89651c47c8095d476251f2e4314656b292 的作者字段含非 UTF-8 字节；现有 git-internal 的 Commit/Signature 入口不能在保持作者原字节不变的前提下构造该提交。按本卡的线性化保真要求，选择该提交之前的发布 tag，不改被测代码或解码器。此选择的局限是前两档各仅一批、H_max 档两批；H = 307 档的根 tree 中尚无 Documentation/ 与 t/，Documentation/ 首次出现于首父链第 322 个提交，因此该档只有视图 0（:/Documentation）的冷启动为空投影，其吞吐不纳入冷启动吞吐结论；视图 1（:exclude[::t/]）仍投影根目录的可见内容，但尚无 t/；该档的常规探针在预热期才建出这两个目录，判据 (a)(b) 的跨档比较混有目录形态差异，不能解释为只由 H 引起。顶格过滤器的 64 个源目录及其 256 个排除选择器在这份 git.git 历史中也未命中，规模复核的结果偏向空视图场景。判据 (a)–(d) 只覆盖此规模，不得据此断言更长历史的无关性或把冷启动吞吐外推到生产大库。阈值、探针次数及判定方式均不放宽；DEFER-HP-12 使用本卡吞吐记录时须保留这一局限。冻结提交和实测结果在本卡发布窗口补入下表与执行记录。 + 冻结协议。开工复核（ER-03）时，执行者在第一次运行基准之前写定下表的待定项，与已定项一起写入 `examples/view_bench.params.json`。 - 冻结提交：发布者随即单独提交这一个文件。提交之前，在包含该文件、其余被测代码与当时 HEAD 相同的树上跑三门：`cargo +nightly fmt --all --check`、`cargo clippy --all-targets --all-features -- -D warnings`、`source .env.test && cargo test --all`。三门都通过后才 `libra add examples/view_bench.params.json`，再 `libra commit -s -m "HP-24: freeze view_bench params"`，按 ER-07 核对 trailer。这次提交不 bump、不打 tag。文件扩展名不免除这三门。推送前 `libra fetch`；分叉时 rebase 到 `origin/main`，在 rebase 后的树上重跑三门，通过后再 `libra push origin main`。推送成功之后才运行基准（ADR-HP-07「开工时提交」）。 - 冻结以这次提交中的文件为准，它就是可复核的冻结记录。Verification 第 2 行核对参数文件已被跟踪且未改动，并核对每份证据的 `params` 都与它相同。 @@ -5625,7 +5691,7 @@ ER-04 强制门（A 组，不计入 VER，每条退出 0 即通过）： **Lifecycle / Acceptance:** `pending` / 空 -**Description:** 其余 34 张卡（按「收口写法」第 7 项整卡延后的卡除外）全部 `done` / `complete` 之后，把本计划、计划索引、长期能力 PT-14 与设计速览表同步到收口状态，并核对四项 outgoing 移交的接手情况。唯一的行为轴是「计划收口与移交登记」。范围： +**Description:** 其余 35 张卡（按「收口写法」第 7 项整卡延后的卡除外，包含 FIX-HP-01）全部 `done` / `complete` 之后，把本计划、计划索引、长期能力 PT-14 与设计速览表同步到收口状态，并核对四项 outgoing 移交的接手情况。唯一的行为轴是「计划收口与移交登记」。范围： - 本计划（协调文件，由发布者写入，ER-12）：「计划状态」行改为「已完成」，写出收口日期与每张发布卡的版本、`docker` run id；「完成判据」与「兼容与文档收口」逐项勾选，「完成判据」各项附证据；依赖登记表 `DEP-HP-04`…`DEP-HP-07` 四条 outgoing 行写入收口核对；`DEP-HP-05` 的 Owner 列不再是「待立项」。 - `docs/plan/README.md`：本计划行的状态。 @@ -5641,7 +5707,7 @@ ER-04 强制门（A 组，不计入 VER，每条退出 0 即通过）： 4. `DEP-HP-05` 的 Owner：接手时写接收方的日期计划文件（按第 3 项写成相对链接）或新的 PT 编号；未接手时写「未接手」，回落按该行「超时与失败策略」执行。 5. P0 交付证据：PT-14 详细章节「审计证据、真实缺口与提升条件」小节中的一条，以 `- **P0 交付证据**：` 开头，含指向本计划的相对链接（目标为 `plan-20261002.md`）与 P0 最后一个发布版本号 `v`；链接与版本号的先后不限。「最后一个发布版本号」指「计划状态」行各条发布证据（第 2 项的写法）中按版本号排序最大的一个。 6. 引用写法与核对范围（由 Verification 之后的「ER-04 B 组门」执行）：四个产物文件中的全部相对链接（外部 URL 与行内代码中的文本除外）都解析到当前 checkout 中的文件。相对开工基线新增或改写的行中，`路径:行号` / `路径:起-止` 锚点按仓库相对路径解析（文件存在，行号不超过文件行数）；以 `src/`、`tests/`、`examples/`、`scripts/`、`config/`、`docs/` 开头的路径都存在；不写只有 `:行号` 的续写锚点。外部仓库（Josh、Libra 等）的位置写符号名、提交或 URL，不写成 `路径:行号`。 -7. 整卡延后：指「非目标与延后项」中某一 `DEFER-HP-*` 行的「延后内容」列以 `整卡延后：<卡号>` 开头时列出的卡（多张卡用「、」连写，如 `整卡延后：HP-21、FIX-HP-01`；同一格中其后出现的卡号不算）。这类行只能经「修订历史」登记的规范性变更新增（G-09），核对日没有这样的行；其它写法的延后行不使任何卡豁免。新增这类行会缩小 P0 的交付范围，所以新增它的那次修订必须在同一变更中一并改写下列各处，缺一处就不得新增：「成功定义」中「全部 35 张卡」一句（扣除整卡延后的卡）；被延后卡所在 Phase 的退出条件；追溯表「设计 §7.5 P0 验收与卡的对应」中由被延后卡承接的验收；本卡 AC 第 15、16 条与 Verification 第 17、18 条，使被延后卡的 P0 组件所在的速览表行按 `general.md:96` 的词表如实写作「部分完成」或「未实现」，并从这两组判据中豁免该行。AC 第 2、4 条与 Verification 第 4、6 条已按本项豁免，不需要改写。 +7. 整卡延后：指「非目标与延后项」中某一 `DEFER-HP-*` 行的「延后内容」列以 `整卡延后：<卡号>` 开头时列出的卡（多张卡用「、」连写，如 `整卡延后：HP-21、FIX-HP-01`；同一格中其后出现的卡号不算）。这类行只能经「修订历史」登记的规范性变更新增（G-09），核对日没有这样的行；其它写法的延后行不使任何卡豁免。新增这类行会缩小 P0 的交付范围，所以新增它的那次修订必须在同一变更中一并改写下列各处，缺一处就不得新增：「成功定义」中「HP-01…HP-35 与 ER-10 增补的 FIX-HP-01」一句（扣除整卡延后的卡）；被延后卡所在 Phase 的退出条件；追溯表「设计 §7.5 P0 验收与卡的对应」中由被延后卡承接的验收；本卡 AC 第 15、16 条与 Verification 第 17、18 条，使被延后卡的 P0 组件所在的速览表行按 `general.md:96` 的词表如实写作「部分完成」或「未实现」，并从这两组判据中豁免该行。AC 第 2、4 条与 Verification 第 4、6 条已按本项豁免，不需要改写。 开工步骤（ER-01 之后、修改任何文件之前执行一次；这是准备步骤，不是验证门，不计入 VER）。先做前置检查，再保存四个产物文件与 `libra status --short` 的基线，供「ER-04 B 组门」第 2、3 条与 M7 第 4 步的暂存前复核使用：`source .env.test && bash -c 'set -eo pipefail; t=target/tmp/hp26; mkdir -p $t/before; libra status --short > $t/status-before.raw; if rg -n -e " docs/plan/(plan-long|README)\.md$" -e " docs/refactoring/history-projection\.md$" $t/status-before.raw; then echo "BLOCKED: shared file has uncommitted changes"; exit 1; else rc=$?; [ "$rc" -eq 1 ] || exit "$rc"; fi; for f in docs/plan/plan-20261002.md docs/plan/README.md docs/plan/plan-long.md docs/refactoring/history-projection.md; do cp "$f" "$t/before/$(printf %s "$f" | tr / _)"; done; LC_ALL=C sort $t/status-before.raw > $t/status-before.txt; echo OK'`，输出 `OK`。 @@ -5721,7 +5787,7 @@ ER-04 强制门（A 组，不计入 VER，每条退出 0 即通过）： - [ ] `` source .env.test && rg -n '^\| \[`plan-20261002\.md`\]\(plan-20261002\.md\) \| PT-14[^|]* \| (\*\*)?已完成' docs/plan/plan-long.md ``：至少 1 行命中（AC 第 1 条的 `plan-long.md` 一处）。 - [ ] `source .env.test && bash -c 'set -eo pipefail; f=docs/plan/plan-20261002.md; s=$(rg -m1 "^\*\*计划状态：\*\*" "$f"); dfr=$(awk -F"|" "/^## 非目标与延后项/{g=1;next} /^## /{g=0} g && \$2 ~ /DEFER-HP-/ {print \$3}" "$f" | { rg -oP "^ 整卡延后：(?:(?:FIX-)?HP-[0-9]+、)*(?:FIX-)?HP-[0-9]+" || [ $? -eq 1 ]; } | { rg -oP "(? target/tmp/hp26/runs.txt; test -s target/tmp/hp26/runs.txt; while read -r v id; do r=$(gh run view "$id" -R gitmono-dev/mega2 --json headBranch,event,conclusion,workflowName -q "[.headBranch,.event,.conclusion,.workflowName]|join(\" \")" < /dev/null); [ "$r" = "$v push success docker-build-push" ] || { echo "BAD $v $id: $r"; exit 1; }; done < target/tmp/hp26/runs.txt; echo OK'`：输出 `OK`；逐条比对「收口写法」第 2 项的四个字段（AC 第 3 条）。 -- [ ] `source .env.test && bash -c 'set -eo pipefail; f=docs/plan/plan-20261002.md; t=target/tmp/hp26; mkdir -p $t; awk -F"|" "/^## 非目标与延后项/{g=1;next} /^## /{g=0} g && \$2 ~ /DEFER-HP-/ {print \$3}" "$f" | { rg -oP "^ 整卡延后：(?:(?:FIX-)?HP-[0-9]+、)*(?:FIX-)?HP-[0-9]+" || [ $? -eq 1 ]; } | { rg -oP "(? $t/deferred-cards.txt; awk "/^### Task /{id=\$3; sub(/:\$/, \"\", id)} /^[*][*]Lifecycle \/ Acceptance:[*][*]/{n++; if (\$0 !~ /:[*][*] .done. \/ .complete./) print id} END{if (n < 35) print \"COUNT\"}" "$f" > $t/open-cards.txt; rc=0; while read -r id; do [ "$id" = HP-26 ] && continue; rg -q -x -F "$id" $t/deferred-cards.txt || { echo "OPEN $id"; rc=1; }; done < $t/open-cards.txt; [ "$rc" -eq 0 ]; echo OK'`：输出 `OK`；任务卡少于 35 张时打印 `OPEN COUNT`，除本卡与整卡延后的卡（「收口写法」第 7 项）外有卡不是 `done` / `complete` 时打印 `OPEN <卡号>`，两种情况都失败（AC 第 4 条）。 +- [ ] `source .env.test && bash -c 'set -eo pipefail; f=docs/plan/plan-20261002.md; t=target/tmp/hp26; mkdir -p $t; awk -F"|" "/^## 非目标与延后项/{g=1;next} /^## /{g=0} g && \$2 ~ /DEFER-HP-/ {print \$3}" "$f" | { rg -oP "^ 整卡延后：(?:(?:FIX-)?HP-[0-9]+、)*(?:FIX-)?HP-[0-9]+" || [ $? -eq 1 ]; } | { rg -oP "(? $t/deferred-cards.txt; awk "/^### Task /{id=\$3; sub(/:\$/, \"\", id)} /^[*][*]Lifecycle \/ Acceptance:[*][*]/{n++; if (\$0 !~ /:[*][*] .done. \/ .complete./) print id} END{if (n < 36) print \"COUNT\"}" "$f" > $t/open-cards.txt; rc=0; while read -r id; do [ "$id" = HP-26 ] && continue; rg -q -x -F "$id" $t/deferred-cards.txt || { echo "OPEN $id"; rc=1; }; done < $t/open-cards.txt; [ "$rc" -eq 0 ]; echo OK'`：输出 `OK`；任务卡少于 36 张时打印 `OPEN COUNT`，除本卡与整卡延后的卡（「收口写法」第 7 项）外有卡不是 `done` / `complete` 时打印 `OPEN <卡号>`，两种情况都失败（AC 第 4 条）。 - [ ] `source .env.test && bash -c 'set -eo pipefail; mkdir -p target/tmp/hp26; awk "/^## (完成判据|兼容与文档收口)\$/{f=1;next} /^## /{f=0} f && /^- \\[/ && !/计划完成门（M7）/" docs/plan/plan-20261002.md > target/tmp/hp26/checks.txt; test "$(wc -l < target/tmp/hp26/checks.txt)" -ge 1; if rg -n -v "^- \[x\] " target/tmp/hp26/checks.txt; then echo "FAIL: unchecked item"; exit 1; else rc=$?; [ "$rc" -eq 1 ] || exit "$rc"; echo OK; fi'`：输出 `OK`；行数下限 1 只用于确认找到了这两节（AC 第 5 条）。 - [ ] `source .env.test && bash -c 'set -eo pipefail; mkdir -p target/tmp/hp26; awk "/^## 完成判据\$/{f=1;next} /^## /{f=0} f && /^- \\[x\\]/" docs/plan/plan-20261002.md > target/tmp/hp26/checked.txt; test "$(wc -l < target/tmp/hp26/checked.txt)" -ge 1; if rg -n -v "证据：" target/tmp/hp26/checked.txt; then echo "FAIL: item without evidence"; exit 1; else rc=$?; [ "$rc" -eq 1 ] || exit "$rc"; echo OK; fi'`：输出 `OK`；行数下限 1 只用于确认找到了该节（AC 第 6 条）。 - [ ] `source .env.test && awk -F'|' '$2 ~ /^ DEP-HP-0[4-7] $/ {n++; if (NF == 10 && $8 ~ /收口核对（[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]）：(已接手|未接手)/) m++} END {if (n == 4 && m == 4) print "OK"; else {print "FAIL " m+0 "/" n+0; exit 1}}' docs/plan/plan-20261002.md`：输出 `OK`；只读「证据」列（按 `|` 切分的第 8 段），四行中任一行缺收口核对、收口核对写在其它列或行数不是 4 都失败（AC 第 7 条）。 @@ -5819,7 +5885,8 @@ ER-04 强制门（A 组，不计入 VER，每条退出 0 即通过）： | HP-21 | implementation | 回收态视图经读者访问触发 Rewarm 准入回到预热（应答恒为暂不可用，不计速率） | 前滚新 patch（降级为 [views].enabled=false 并重启，视图入口 404、不再触发准入；被触发预热的视图可在停服后用 recycle-filter 运维 SQL 退回回收态） | yes | yes | 8/8 | 8/8 | 1 / 1 | S | HP-20,HP-13 | 序列化于 HP-20（src/contract/git_protocol/mod.rs、tests/integration_views.rs），docs/deploy-trunk.md 经 HP-20 的前置链晚于 HP-30、HP-14、HP-33 | independent | N/A | N/A | | HP-22 | implementation | 视图 URL 的 clone 与 fetch（ViewRepo 打包） | 前滚新 patch（降级：[views].enabled = false 后重启，视图 URL 一律 404） | yes | yes | 8/8 | 8/8（第 6、7 行为前缀门，分别要求 passed ≥ 3、passed ≥ 2，每行一条命令、各计 1 条） | 2 / 2 | M | HP-20,HP-19,DEP-HP-03 | 序列化于 HP-19,HP-20（view_repo.rs、view_projection_storage.rs 经 HP-19；integration_git_cli.rs、integration_git_ssh.rs、protocol.md 经 HP-20），docs/architecture.md / .zh.md 经上游链晚于 HP-07，docs/user-guide.md / .zh.md 经上游链晚于 HP-08 | independent | N/A | N/A | | HP-23 | implementation | 发包前 fail-closed 预检及其线上应答 | 前滚新 patch（降级：[views].enabled=false 并重启，视图 URL 全部 404） | yes | yes | 8/8 | 7/8 | 3 / 3 | M | HP-22,HP-21 | 序列化于 HP-22,HP-21 | independent | N/A | N/A | -| HP-24 | implementation | P0 性能门（冻结协议下的投影计量与判定） | 前滚新 patch（example 不进 mega2 二进制，接缝只转发、包装与装配。判据或预算不满足时按 ER-10 新建 FIX-HP-*：修复不改 REGISTER_SCALE_LIMITS 时用同一份冻结参数重跑，下调该常量时本卡经 ER-03 修订、在新上限处重新冻结并把旧证据移入 superseded-*） | yes | yes | 8/8 | 8/8 | 2 / 3 | M | HP-12,HP-28,DEP-HP-02 | no-overlap | independent | N/A | N/A | +| FIX-HP-01 | implementation | 根链与视图追赶的性能修复 | 前滚 patch；未达标时由新 FIX 卡承接 | yes | yes | 8/8 | 4/8 | 2 / 2 | M | HP-10,HP-28,HP-12 | 序列化于 HP-10,HP-28,HP-12 | independent | N/A | N/A | +| HP-24 | implementation | P0 性能门（冻结协议下的投影计量与判定） | 前滚新 patch（example 不进 mega2 二进制，接缝只转发、包装与装配。判据或预算不满足时按 ER-10 新建 FIX-HP-*：修复不改 REGISTER_SCALE_LIMITS 时用同一份冻结参数重跑，下调该常量时本卡经 ER-03 修订、在新上限处重新冻结并把旧证据移入 superseded-*） | yes | yes | 8/8 | 8/8 | 2 / 3 | M | HP-12,HP-28,FIX-HP-01,DEP-HP-02 | no-overlap | independent | N/A | N/A | | HP-25 | implementation | Libra 与视图 URL 的互通（clone / fetch 成功路径与设计 §7.4 错误契约第 7 项，在隔离的 views 模式栈上判定） | 前滚新 patch（只有脚本、compose overlay 与运维文档，mega2 二进制不变） | yes | yes | 8/8 | 8/8 | 2 / 4 | M | HP-23,HP-30,DEP-HP-01 | docs/deploy-trunk.md 经 HP-23 的前置链晚于 HP-30、HP-14、HP-33、HP-21（本卡为最后一个写入者）；scripts/bb_optin_run.sh 按 DEP-HP-01 的协调规则与 plan-20261001 串行 | independent | N/A | N/A | | HP-26 | docs | 计划收口与移交登记 | revert（推送前只撤回本卡在四个文件中的改动；推送后以一次文档提交撤回） | yes | yes | 16/20 | 18/20 | 0 / 0 | S | HP-24,HP-25 | 序列化于 HP-01（docs/plan/README.md、docs/plan/plan-long.md，经依赖链传递） | no-release | N/A | N/A | @@ -5875,6 +5942,7 @@ ER-04 强制门（A 组，不计入 VER，每条退出 0 即通过）： | HP-21 | 设计 §4.6「视图删除即回收」、§6.1 第 3 层 | `src/contract/git_protocol/mod.rs` | `deploy-trunk.md` | `--test integration_views rewarm_` | | HP-22 | 设计 §6.3「打包」、§7.4「协议（P0）」「能力」；P0 验收 1、3、7 | `src/ceres/pack/view_repo.rs` | 双语 `user-guide`、`architecture`；`protocol.md` | `--test integration_git_cli integration_git_cli_view_` | | HP-23 | 设计 §6.3「L0 tree 校验」「want 校验」、§7.4 错误契约第 5、6 项；P0 验收 7 | `src/ceres/pack/view_repo.rs` | `protocol.md` | `--test integration_views precheck_` | +| FIX-HP-01 | 设计 §3.3 与 §4.4；HP-24 首次冻结运行的 P0 验收 11(c)(d) 失败 | `src/jupiter/storage/view_root_chain.rs`、`src/jupiter/service/view_projection_service.rs` | 设计 §3.3/§4.4 | 两个模块的 lib 定向测试与提交树 C 组 | | HP-24 | 设计 §7.4「基准」；P0 验收 11 | `examples/view_bench.rs`、`src/lib.rs` | 设计 §7.2 | `cargo run --release --example view_bench` | | HP-25 | 设计 §7.4 错误契约第 7 项、§8「plan-20261001」 | `scripts/libra_view_smoke.sh`、`scripts/bb_optin_run.sh` | 无 | `scripts/bb_optin_run.sh views …` | | HP-26 | 「完成判据」 | `docs/plan/`、设计 | PT-14 | 结构与链接门 | @@ -6110,6 +6178,9 @@ Result 只允许 `PASS` 或 `FAIL`。`FAIL` 必须列出 P0/P1 条目，并在 | HP23-R2 | HP-23 首轮修订，Claude CLI 只读复审 | FAIL | P1：夹具修订后 nightly 格式门未重跑；R3 前已执行格式化、格式检查及 Clippy | 测试专用提交读取、`:/e` 告警断言、缺 tree 路径唯一性与 HTTP 四视图夹具已收紧；have 侧旧树验证偏严格由发布者（Codex）作为 fail-closed 残余行为接受 | `target/tmp/hp23/claude-r2.txt` | | HP23-R3 | HP-23 最终源码、八项验收与七条验证，Claude CLI 只读复审 | PASS | P0 0；P1 0 | 三项非阻断意见：have 侧旧树也校验、同层多个不自洽 tree 的首个 id 由数据库行序决定、SSH 原始请求的 stderr 管道；发布者（Codex）接受为本卡残余风险，均不改变既定错误契约 | `target/tmp/hp23/claude-r3.txt`；`VERDICT: PASS`；7/7 定向验证、nightly 格式、Clippy、ER-04 A 组通过 | | HP23-R4 | HP-23 代码、测试、协议应答、文档及发布门，Codex 独立复核 | PASS | P0 0；P1 0 | 七条精确验证全通过；ER-04 A 组只读 smoke 8 passed、0 failed（既有 LFS 指针使用 `GIT_LFS_SKIP_SMUDGE=1`）；最终版本树的完整测试 19 个目标均 0 失败，lib 2038 通过、1 项既有忽略；nightly 格式、全目标全特性 Clippy、两个构建命令退出 0，定向 macOS 链接参数下 0 警告；保留 R3 的三项 P2 残余行为 | `target/tmp/hp23/codex-r4.md`、`target/tmp/hp23/full-test-final.log`、`target/tmp/hp23/claude-r3.txt`、`libra diff --check` | +| FIXHP01-PLAN-R1 | FIX-HP-01 初稿与设计 §3.3，Claude CLI 只读审查 | FAIL | P1：判据 (c) 未分段诊断；AC 与发布闭环；依赖边和顺序未登记；模板字段及验证命令不合规；修订历史未登记。已用分段诊断并修订 §3.3/§4.4、卡片、DAG 与表格，待 R2 复审 | P2：预算语义、冲突证据、参数上限、提交树验证均纳入修订 | `target/tmp/view_bench/preflight/fix-hp01-plan-review.txt`、`diagnostic.log` | +| FIXHP01-PLAN-R2 | FIX-HP-01 与设计 §3.3/§4.4 修订，Claude CLI 只读复审 | FAIL | P1：发布前缺少判据 (c) 的实际诊断与量化预期；AC 1 依赖发布后的基准；Phase 6 未登记 FIX；HP-26 对原 35 张卡的文字引用过期。已补推送前隔离 worktree 诊断、lib 语句上界、Phase 6 与 HP-26 文本，待 R3 复审 | P2：设计已明确快路 `ON CONFLICT DO NOTHING` 的冲突回退，诊断与正式证据目录分离 | `target/tmp/view_bench/preflight/fix-hp01-plan-review-r2.txt` | +| FIXHP01-PLAN-R3 | FIX-HP-01 可执行计划与设计，Claude CLI 只读复审 | PASS | P0 0；P1 0 | P2：多余空行待清理；AC 1 对增加一批的 +15 上界可再加一档 H=128、B=64 实测，现已在定向用例实施 | `target/tmp/view_bench/preflight/fix-hp01-plan-review-r3.txt`；`VERDICT: PASS` | ## 非目标与延后项 diff --git a/docs/refactoring/history-projection.md b/docs/refactoring/history-projection.md index 1b912d1e..7edfb06f 100644 --- a/docs/refactoring/history-projection.md +++ b/docs/refactoring/history-projection.md @@ -39,6 +39,7 @@ 13. 执行计划「测试」行把 `GIT_CLI_UNAVAILABLE` 标在 `tests/common/git_cli.rs` 约 L20、runner 不可用分支标在约 L440–448；端口分配改造后，常量在 `tests/common/git_cli.rs:21`，该分支在 `tests/common/git_cli.rs:482-537`，仍会在未启用宿主 opt-in 且容器不可用时 panic。 14. 执行计划「文档」行的 `docs/refactoring/integration.md` 迁移覆盖锚点原为 L129–150；增加 compose 黑盒覆盖表后，迁移覆盖章节从 `docs/refactoring/integration.md:143-164` 开始，原行号不再定位该节。 15. §6.1 第 3 层的读者触发重新预热未标（P1），但 §3.3、§7.5 与 §7.4 的阶段描述指向 P1。按执行计划 HP-21 与 ADR-HP-08，P0 实现读者触发的准入；本条只校准阶段归属，不改设计正文的算法与错误契约。 +16. 2026-10-08 的 HP-24 冻结运行在 §7.5 P0 验收 11(c)(d) 失败，执行计划按 ER-10 新增 FIX-HP-01。§3.3 与 §4.4 的批量写入、无预算单提交快路和末批终态是性能修复的执行契约；预算调用、根链不连续、投影停止及部分推进仍按原语义。冻结参数和阈值不变，修复发布后再重跑 HP-24。 ## 当前实现状态速览表 @@ -585,6 +586,7 @@ T_mono_new = overlay(lifted, stripped) # overlay 冲突时 lifted 胜出 - advertise 与 B0 抢锁失败时，用带 `lock_timeout` 的阻塞锁等待，超时按"未追上"处理。 2. **起扫。** - 暂存表为空时，在事务内读取 `main@/`，记为 h0。h0 等于链尾就返回"已追平"，否则写入 pos=1 的行。 + - 暂存表为空、调用方不设预算、链尾存在、h0 恰有一个父提交且该父提交等于当前链尾时，可在这笔持锁事务内直接以 `seq(链尾)+1` 接入 h0，随后提交、清除断链告警并返回"已追平"。h0 与链尾都在取锁之后的同一事务内读取。这条单提交快路同时核对父链与链尾，不根据无条件的 `max(seq)+1` 猜测接入位置；插入的 `SELECT` 同时检查链尾提交行仍在 `mega_commit`，并使用 `ON CONFLICT DO NOTHING`。因父行缺失或唯一键冲突使影响行数为 0 时，在同一事务内继续写 pos=1 的暂存行，由原算法判定并保留暂存证据，不能把不一致的行视为成功。带预算的调用保持原有回走与接入计数，不走快路；暂存表非空、多父、缺父、回滚及分叉仍走原算法。 - 暂存表非空时，从 pos 最大的行接着扫，不重读 `main@/`。扫描期间根的前进，留给下一次扫描。 3. **判定。** 取暂存表中 pos 最大的行 c，即回走到的最早提交： - c 在根链表中，且等于链尾：**连上**，转第 5 步。 @@ -594,12 +596,14 @@ T_mono_new = overlay(lifted, stripped) # overlay 冲突时 lifted 胜出 - c 的首父在 `mega_commit` 中查不到：**不连续**（数据缺失）。 - 其余情形，即 c 不在表中、恰有一个父提交：未终止，转第 4 步。 4. **分批回走。** 从 c 的首父起，用一条递归 CTE 沿 `mega_commit.parents_id->>0` 往回取，至多 `views.batch_size` 个提交。遇到第一个"在根链表中、无父、多父或首父缺失"的提交就停下，这个提交也写入暂存表，然后回到第 3 步。 + - 同一批暂存行以每条至多 1000 行的批量 SQL 写入，按 pos 顺序保留 `ON CONFLICT DO NOTHING` 的原有幂等语义；每行 5 个参数，因此每条最多 5000 个参数，低于 Postgres 的 65535 上限。即使内部调用绕过配置层的 `views.batch_size ≤ 10000` 校验，分块仍保证参数上限。 - 预算按本次调用处理的提交数计算，回走与第 5 步的接入合计。advertise 与 B0 的预算取 `views.max_append_walk`。 - 预算用尽就返回"未追上"。这只表示本次的工作量用完了，**不是**不连续判定：暂存行保留，由下一次调用或后台 worker 接着扫。 - 后台 worker 不设预算，一批接一批扫到终止，或者扫到进程退出为止。 5. **接入。** 按 pos 从大到小（即 seq 递增）把暂存行插入根链表。每段不超过 10 000 行，每段一个事务。根链锁是事务级锁（第 1 步），段与段之间会释放，期间进程可能退出，其他调用方也可能取得锁，所以每个段事务都自成一体，不沿用上一段的判定： - **重新判定。** 段事务按第 1 步取根链锁，重读链尾，取暂存表中 pos 最大的行作为锚点 c，在本事务内重做第 3 步。只有结论为"连上"才继续，其余结论按第 3 步处理。"连上"本身就保证待插入的第一行的首父等于链尾；冷启动时，它保证根链表为空。 - **插入。** 设 c 的 pos 为 P。 + - 一段内的行以每条至多 1000 行的批量 SQL 写入，每行 3 个参数。部分行因唯一键冲突未插入时，按原有规则核对已存在行的 seq 与 commit_id；不一致仍为"不连续（RowConflict）"，整段事务回滚。段至多 10000 行，冲突核对的 2 × 10000 个参数也低于 Postgres 上限。 - 接在链尾之后时（c 就是链尾），本段从 pos = P − 1 起向下插入，pos = p 的行得到 seq = seq(c) + (P − p)。 - 冷启动时，本段从 c 自身起插入，pos = p 的行得到 seq = P − p + 1，c 得到 seq = 1。 - **换锚点。** 设本段插入的最小 pos 为 p_min。在**同一事务内**删除暂存表中 pos > p_min 的全部行，包括原锚点 c（冷启动时 c 已在本段接入，同样删除），只留下 pos = p_min 的行，即本段的新链尾；p_min = 1 时清空暂存表。插入与删除必须一起提交，不得拆成两个事务。 @@ -614,7 +618,7 @@ T_mono_new = overlay(lifted, stripped) # overlay 冲突时 lifted 胜出 所以"在根链表中但不是链尾"仍然只表示真实的回滚或分叉，不会因为段间退出或段间换手而出现。不持根链锁的读者（5.2 第 2 步的单条 SQL、4.4 的 catch_up）按快照读根链表，只会看到某个段事务提交之前或之后的状态。两种状态都满足首父闭包，回走遇到的第一个表内提交都是当时的链尾，所以也不会误判。 全部接入后返回"已追平"。如果这时 `main@/` 又前进了，下一次调用从新的 h0 开始扫描。 - 6. **只按 seq 递增插入。** 禁止用 `max(seq)+1` 追加单个提交。 + 6. **只按 seq 递增插入。** 禁止不核对首父与链尾就用 `max(seq)+1` 追加单个提交；第 2 步的快路满足首父闭包并持同一把根链锁。 - **调用方如何处理"未追上"。** 只读且已就绪的视图，照常广告已投影到的最新 tip，并记录滞后（4.4）；可推送视图的 advertise 与 B0 返回可重试错误。三者都发 4.4 的进程内信号，唤醒本副本的后台 worker 从暂存表接着扫描（worker 不受预算限制）。 - **为什么不用走回根就能判定祖先关系。** 根链表是首父闭包的，所以从 h0 回走遇到的第一个表内提交 c，就是 h0 与链尾的最近公共祖先：c 等于链尾，当且仅当链尾是 h0 的祖先。回走长度就是 h0 到这个公共祖先的距离。 - 合法滞后时，它等于需要接入的提交数，这部分工作本来就要做； @@ -837,7 +841,9 @@ fn catch_up(F): last = if let Some((s, _)) = stop { s − 1 } else { rows.last.seq } F.projected_seq = GREATEST(F.projected_seq, last) if let Some(st) = stop: alert(st); return # 本事务照常提交；不原地重试 - if last == tip: mark_ready_if_covered(F, tip) # 与推进水位同一事务 + if last == tip: # 与推进水位同一事务 + if mark_ready_if_covered(F, tip): return Ready + return MainNotCovered fn mark_ready_if_covered(F, tip) -> bool: # 只在持 L_V(F) 与 L_G 共享锁的批事务内调用 root = read main@/ @@ -846,6 +852,7 @@ fn mark_ready_if_covered(F, tip) -> bool: # 只在持 return true ``` - **往返次数。** 每批 O(1) 条提交查询，再加 O(深度) 条按层的 tree 查询；不存在"每个提交一次往返"。 +- **末批终态（FIX-HP-01）。** 最后一批在持有 L_V(F) 与 L_G 共享锁的同一事务内，已写入 `projected_seq` 并执行 `mark_ready_if_covered`。该事务提交成功后可直接返回 Ready 或 MainNotCovered，不再打开一笔无内容的终止检查事务。若该事务提交失败，仍返回错误而不是终态；未到 tip 的批、停止和部分推进仍按原循环处理。这里不跳过就绪校验，也不提前释放锁。只读视图的同步追赶收到 MainNotCovered 时会按 4.6 唤醒 worker；直接返回终态使这个唤醒早于原先的空终止事务，行为仍可重试。 - **可失败批量读取的语句数（HP-29）。** 一层有 k 个待查 tree id、其中 r 个不在 `mega_view_object` 中时，先查询 `mega_view_object`、再查询剩余 `mega_tree`，至多执行 ⌈k/1000⌉ + ⌈r/1000⌉ 条语句；每条最多 1000 个 id。伪代码的「每层 1 条 SQL」和 4.1 的「同一层只发一次查询」按此分块规则理解：一层 id 不超过 1000 时至多两条。每层语句数只取决于该层 id 数，与批内提交数无关，因此 O(深度) 不变。提交的 `get_commits_by_hashes_fallible` 也按 1000 分块：请求 m 个不同提交 id 时执行 ⌈m/1000⌉ 条；若按伪代码传入完整 rows，m 至多 B + 1，默认 B = 1000 且 s0 ≥ 1 的满批为 1001 个 id、两条查询，B ≤ 10 000 时至多 11 条。第 s0 行只提供父 tree 时可从 `mega_view_root_chain.tree_id` 取得它，不读取该提交则 m ≤ B；提交读取的「1 条 SQL」按此理解，仍为每批 O(1)。 - **批的前提【决策】。** `s0 < tip` 时，本批必须恰好取到 seq 从 `max(s0,1)` 到 `min(s0+B, tip)` 的连续行，因此至少包含 `seq = s0+1` 这一行，水位每批至少前进 1。 - 取不到这些行，说明派生状态或配置已经损坏，原因可能是根链表出现空洞，或者 B < 1 绕过了 6.9 的校验。 diff --git a/src/jupiter/service/view_projection_service.rs b/src/jupiter/service/view_projection_service.rs index b3982af5..404241b1 100644 --- a/src/jupiter/service/view_projection_service.rs +++ b/src/jupiter/service/view_projection_service.rs @@ -354,14 +354,27 @@ impl ViewProjectionService { view_storage .write_projection_batch(txn, filter_pk, &segments, &written_outputs, projected_seq) .await?; - if stop.is_none() && last == tip { - view_storage + let outcome = if let Some(stop) = stop { + CatchUpOutcome::Stopped(stop) + } else if last == tip { + #[cfg(test)] + self.test_hooks + .terminal_checks + .fetch_add(1, Ordering::Relaxed); + if view_storage .mark_ready_if_covered(txn, filter_pk, tip) - .await?; - } + .await? + { + CatchUpOutcome::Ready + } else { + CatchUpOutcome::MainNotCovered + } + } else { + CatchUpOutcome::Advanced + }; #[cfg(test)] self.test_hooks.record_advanced_projected_seq(projected_seq); - Ok(stop.map_or(CatchUpOutcome::Advanced, CatchUpOutcome::Stopped)) + Ok(outcome) } async fn prefetch_filter_outputs( @@ -2815,7 +2828,11 @@ mod tests { .catch_up_one_batch(filter_pk, &storage.config(), 1) .await .unwrap(), - CatchUpOutcome::Advanced + if expected_seq == 5 { + CatchUpOutcome::Ready + } else { + CatchUpOutcome::Advanced + } ); let row = mega_view_filter::Entity::find_by_id(filter_pk) .one(db) @@ -3463,6 +3480,35 @@ mod tests { assert!(!captured.contains(":[:/a:prefix=x,:/b:prefix=x]")); } + #[tokio::test] + async fn final_batch_returns_ready_without_a_terminal_transaction() { + async fn count(one_batch: bool) -> usize { + let (_temp, _schema, storage, counter) = counted_storage_with_history( + vec![vec![("a/file".to_owned(), b"one".to_vec())]], + 1000, + ) + .await; + let filter_pk = insert_warming_filter(&storage, 1, ":/a").await; + let service = ViewProjectionService::new(storage.clone(), ViewMetrics::default()); + counter.store(0, Ordering::Relaxed); + let outcome = if one_batch { + service + .catch_up_one_batch(filter_pk, &storage.config(), 1000) + .await + } else { + service.catch_up(filter_pk).await + }; + assert_eq!(outcome.unwrap(), CatchUpOutcome::Ready); + assert_eq!( + service.test_hooks.terminal_checks.load(Ordering::Relaxed), + 1 + ); + counter.load(Ordering::Relaxed) + } + + assert_eq!(count(true).await, count(false).await); + } + #[tokio::test] async fn statement_count_independent_of_batch() { fn roots(batch_size: u64) -> Vec)>> { diff --git a/src/jupiter/storage/view_root_chain.rs b/src/jupiter/storage/view_root_chain.rs index 15b8f2c5..27c2a9cd 100644 --- a/src/jupiter/storage/view_root_chain.rs +++ b/src/jupiter/storage/view_root_chain.rs @@ -12,6 +12,7 @@ use crate::{ }; const ROOT_CHAIN_SEGMENT_ROWS: usize = 10_000; +const ROOT_CHAIN_SQL_CHUNK_ROWS: usize = 1_000; /// A self-contained expression readers can embed in their snapshot query. /// It preserves the ordered root-chain decision: an anchored scan row is @@ -172,6 +173,31 @@ impl ViewStorage { txn.commit().await?; return Ok(self.finish_outcome(RootChainOutcome::CaughtUp)); } + if remaining.is_none() + && root.parent_count == 1 + && let Some(tail) = tail.as_ref() + && root.first_parent.as_deref() == Some(tail.commit_id.as_str()) + { + let inserted = txn + .execute_raw(Statement::from_sql_and_values( + DbBackend::Postgres, + "INSERT INTO mega_view_root_chain (seq, commit_id, tree_id) \ + SELECT $1, $2, $3 WHERE EXISTS ( \ + SELECT 1 FROM mega_commit WHERE commit_id = $4 \ + ) ON CONFLICT DO NOTHING", + [ + Value::from(tail.seq + 1), + Value::from(root.commit_id.clone()), + Value::from(root.tree_id.clone()), + Value::from(tail.commit_id.clone()), + ], + )) + .await?; + if inserted.rows_affected() == 1 { + txn.commit().await?; + return Ok(self.finish_outcome(RootChainOutcome::CaughtUp)); + } + } if exhausted(remaining) { txn.rollback().await?; return Ok(RootChainOutcome::NotCaughtUp); @@ -471,19 +497,35 @@ async fn insert_scan_rows( first_pos: i64, rows: &[ScanRow], ) -> Result<(), MegaError> { - for (offset, row) in rows.iter().enumerate() { - txn.execute_raw(Statement::from_sql_and_values( - DbBackend::Postgres, - "INSERT INTO mega_view_root_chain_scan \ - (pos, commit_id, tree_id, parent_count, first_parent) \ - VALUES ($1, $2, $3, $4, $5) ON CONFLICT DO NOTHING", - [ - Value::from(first_pos + offset as i64), + for (chunk_index, chunk) in rows.chunks(ROOT_CHAIN_SQL_CHUNK_ROWS).enumerate() { + let mut placeholders = Vec::with_capacity(chunk.len()); + let mut values = Vec::with_capacity(chunk.len() * 5); + for (offset, row) in chunk.iter().enumerate() { + let parameter = offset * 5 + 1; + placeholders.push(format!( + "(${parameter}, ${}, ${}, ${}, ${})", + parameter + 1, + parameter + 2, + parameter + 3, + parameter + 4 + )); + values.extend([ + Value::from(first_pos + (chunk_index * ROOT_CHAIN_SQL_CHUNK_ROWS + offset) as i64), Value::from(row.commit_id.clone()), Value::from(row.tree_id.clone()), Value::from(row.parent_count), Value::from(row.first_parent.clone()), - ], + ]); + } + txn.execute_raw(Statement::from_sql_and_values( + DbBackend::Postgres, + format!( + "INSERT INTO mega_view_root_chain_scan \ + (pos, commit_id, tree_id, parent_count, first_parent) \ + VALUES {} ON CONFLICT DO NOTHING", + placeholders.join(", ") + ), + values, )) .await?; } @@ -529,17 +571,31 @@ async fn insert_segment( rows: &[RootChainRow], ) -> Result, MegaError> { let mut inserted = 0; - for row in rows { + for chunk in rows.chunks(ROOT_CHAIN_SQL_CHUNK_ROWS) { + let mut placeholders = Vec::with_capacity(chunk.len()); + let mut values = Vec::with_capacity(chunk.len() * 3); + for (index, row) in chunk.iter().enumerate() { + let parameter = index * 3 + 1; + placeholders.push(format!( + "(${parameter}, ${}, ${})", + parameter + 1, + parameter + 2 + )); + values.extend([ + Value::from(row.seq), + Value::from(row.commit_id.clone()), + Value::from(row.tree_id.clone()), + ]); + } let result = txn .execute_raw(Statement::from_sql_and_values( DbBackend::Postgres, - "INSERT INTO mega_view_root_chain (seq, commit_id, tree_id) \ - VALUES ($1, $2, $3) ON CONFLICT DO NOTHING", - [ - Value::from(row.seq), - Value::from(row.commit_id.clone()), - Value::from(row.tree_id.clone()), - ], + format!( + "INSERT INTO mega_view_root_chain (seq, commit_id, tree_id) \ + VALUES {} ON CONFLICT DO NOTHING", + placeholders.join(", ") + ), + values, )) .await?; inserted += result.rows_affected() as usize; @@ -912,6 +968,149 @@ mod tests { assert_eq!(root_chain(db).await, expected_root_chain(commits)); } + #[tokio::test] + async fn cold_chain_statement_count_depends_on_batches() { + async fn count(commits: usize, batch_size: usize) -> usize { + let (_config, _schema, db, counter) = counted_empty().await; + let history = seed_linear_root_history(db.as_ref(), commits).await; + let storage = new_view_storage(db.clone()); + counter.store(0, Ordering::Relaxed); + assert_eq!( + storage + .extend_root_chain(None, batch_size, ViewLockMode::Try) + .await + .unwrap(), + RootChainOutcome::CaughtUp + ); + let statements = counter.load(Ordering::Relaxed); + assert_root_chain_matches(db.as_ref(), &history).await; + statements + } + + let one_batch_64 = count(64, 1000).await; + let one_batch_128 = count(128, 1000).await; + let two_batches_128 = count(128, 64).await; + assert_eq!(one_batch_64, one_batch_128); + assert!(one_batch_128 <= 29, "{one_batch_128}"); + assert!(two_batches_128 >= one_batch_128); + assert!(two_batches_128 - one_batch_128 <= 15); + } + + #[tokio::test] + async fn unbudgeted_single_append_uses_direct_chain_insert() { + let (_config, _schema, db, counter) = counted_empty().await; + let history = seed_linear_root_history(db.as_ref(), 6).await; + let storage = new_view_storage(db.clone()); + assert_eq!( + storage + .extend_root_chain(None, 1000, ViewLockMode::Try) + .await + .unwrap(), + RootChainOutcome::CaughtUp + ); + let next = seed_single_parent_root_commit_with_tree( + db.as_ref(), + HashKind::Sha1, + fixture_tree("direct-append"), + history.last().unwrap(), + "direct append", + ) + .await; + assert!(cas_fixture_main(db.as_ref(), history.last().unwrap(), &next).await); + counter.store(0, Ordering::Relaxed); + assert_eq!( + storage + .extend_root_chain(None, 1000, ViewLockMode::Try) + .await + .unwrap(), + RootChainOutcome::CaughtUp + ); + let statements = counter.load(Ordering::Relaxed); + assert!(statements <= 10, "{statements}"); + assert!(scan_chain(db.as_ref()).await.is_empty()); + let mut expected = history; + expected.push(next); + assert_root_chain_matches(db.as_ref(), &expected).await; + } + + #[tokio::test] + async fn direct_append_conflict_keeps_scan_evidence() { + let (db, storage, history) = view_storage_with_history(6).await; + assert_eq!( + storage + .extend_root_chain(None, 1000, ViewLockMode::Try) + .await + .unwrap(), + RootChainOutcome::CaughtUp + ); + let next = seed_single_parent_root_commit_with_tree( + &db, + HashKind::Sha1, + fixture_tree("direct-conflict"), + history.last().unwrap(), + "direct conflict", + ) + .await; + assert!(cas_fixture_main(&db, history.last().unwrap(), &next).await); + db.execute_raw(Statement::from_sql_and_values( + DbBackend::Postgres, + "UPDATE mega_view_root_chain SET commit_id = $1 WHERE seq = 2", + [Value::from(next.commit.id.to_string())], + )) + .await + .unwrap(); + let roots_before = root_chain(&db).await; + assert_eq!( + storage + .extend_root_chain(None, 1000, ViewLockMode::Try) + .await + .unwrap(), + RootChainOutcome::Discontinuous(DiscontinuityReason::RolledBack) + ); + assert_eq!(root_chain(&db).await, roots_before); + assert_eq!(scan_chain(&db).await[0].1, next.commit.id.to_string()); + assert!(storage.root_chain_halted().await.unwrap()); + } + + #[tokio::test] + async fn direct_append_missing_tail_commit_keeps_scan_evidence() { + let (db, storage, history) = view_storage_with_history(6).await; + assert_eq!( + storage + .extend_root_chain(None, 1000, ViewLockMode::Try) + .await + .unwrap(), + RootChainOutcome::CaughtUp + ); + let next = seed_single_parent_root_commit_with_tree( + &db, + HashKind::Sha1, + fixture_tree("direct-missing-tail"), + history.last().unwrap(), + "direct missing tail", + ) + .await; + assert!(cas_fixture_main(&db, history.last().unwrap(), &next).await); + db.execute_raw(Statement::from_sql_and_values( + DbBackend::Postgres, + "DELETE FROM mega_commit WHERE commit_id = $1", + [Value::from(history.last().unwrap().commit.id.to_string())], + )) + .await + .unwrap(); + let roots_before = root_chain(&db).await; + assert_eq!( + storage + .extend_root_chain(None, 1000, ViewLockMode::Try) + .await + .unwrap(), + RootChainOutcome::Discontinuous(DiscontinuityReason::MissingFirstParent) + ); + assert_eq!(root_chain(&db).await, roots_before); + assert_eq!(scan_chain(&db).await[0].1, next.commit.id.to_string()); + assert!(storage.root_chain_halted().await.unwrap()); + } + async fn assert_scan_anchor_is_tail(db: &DatabaseConnection) { let scan = scan_chain(db).await; if let Some((_, commit_id, _, _, _)) = scan.last() { From d2ac0beb0effa5fa1744d0c4cf1d8865ad39a5a7 Mon Sep 17 00:00:00 2001 From: Eli Ma Date: Thu, 8 Oct 2026 05:25:00 +0800 Subject: [PATCH 5/5] docs(plan): record FIX-HP-01 completion Signed-off-by: Eli Ma --- docs/plan/plan-20261002.md | 34 ++++++++++++++++++++-------------- 1 file changed, 20 insertions(+), 14 deletions(-) diff --git a/docs/plan/plan-20261002.md b/docs/plan/plan-20261002.md index f9b44650..9cce7846 100644 --- a/docs/plan/plan-20261002.md +++ b/docs/plan/plan-20261002.md @@ -187,6 +187,7 @@ | 2026-10-07 | HP-22 打包、验收与三轮 Claude CLI 复审 | ViewRepo 提供只读视图 URL 的 HTTP/SSH v0/v2 clone 与 fetch；Josh 固定期望与本项目增量扩展按 ER-03 分开登记。Claude R1 的闭包与夹具证据缺口已修订，R2/R3 `VERDICT: PASS`；Codex 复核通过。8 条精确验证、ER-04 A 组四项本地只读 smoke、nightly 格式、全目标全特性 Clippy、受控并发的完整 `cargo test --all` 及两个构建门均通过，专用链接参数下构建 0 警告。首次完整测试的一个现有进程级用例遇到回环连接重置，单独复跑与受控并发的第二次完整运行通过。Docker Build / Actions 按发布者指令不等待、不查询。 | 无拆卡 | HP-22、`protocol.md`、双语用户指南与架构、Review log、ER-04 C/D | | 2026-10-08 | HP-23 发布 | HP-23 已完成并由 `v0.42.22` 承载；发布后从 HP-24 继续。 | 无拆卡 | HP-23、发布窗口、Review log | | 2026-10-08 | HP-24 冻结基准 FAIL 与 ER-10 修复卡 | 参数冻结提交 `7acc2b5` 已推送；ER-03 冻结前因 v2.0.0 首父链存在非 UTF-8 作者而改选 git.git v0.99.5，已在 HP-24 Performance budget 登记范围与局限；首次有效运行的 (c)(d) FAIL，随后诊断把增量耗时拆为根链与视图两段；HP-24 置 blocked，新增 FIX-HP-01 并立即加前置依赖；先修订设计 §3.3、§4.4，再实施根链批写、无预算单提交快路与末批终态。所有 `run-*.json` 与 `failed-*.json` 保留。 | HP-24 → FIX-HP-01、HP-24 | 实施顺序、发布顺序、共享文件、ER-05、HP-24、审计表、设计 §3.3/§4.4、Review log | +| 2026-10-08 | FIX-HP-01 实施、双审与发布 | 根链批写、无预算单提交快路和追赶末批终态由 `v0.42.23` 的 `9685337` 承载；Claude 与独立 Codex 均 `VERDICT: PASS`。定向测试、nightly 格式、全目标全特性 Clippy、第二轮完整测试 19 个目标与两个零警告构建门通过；首轮完整测试中既有端口占用用例的日志时序波动，精确复跑和第二轮完整运行通过。推送前三次隔离冻结诊断的 (a)–(d) 与规模上限均 PASS；标签与 GitHub Release 已发布。Docker Build Action 按发布者指令不等待、不查询。 | 无拆卡 | FIX-HP-01、设计 §3.3/§4.4、Review log、ER-04 C/D | ## 已决议设计决策 @@ -5210,29 +5211,31 @@ ER-04 强制门（不计入 VER）：本卡改 `src/contract/git_protocol/mod.rs **Task type:** `implementation` -**Lifecycle / Acceptance:** `in-progress` / 空 +**Lifecycle / Acceptance:** `done` / `complete`（8/8 验收、4/4 定向验证、Claude 与 Codex 双审 PASS；提交树 nightly 格式、Clippy、完整测试 19 个目标、两个零警告构建门通过；`v0.42.23` 的提交、标签与 GitHub Release 已发布；Docker Build Action 按发布者指令不等待、不查询） **Description:** HP-24 的冻结基准首次有效运行 `run-20261007T192312Z.json` 在判据 (c)、(d) 失败。根链冷启动 H=1231 用 2484 条 SQL；代码逐行写入暂存表与根链表，是 (d) 的主要原因。单提交增量 p95 为 92–100 ms，而 H_max 档的视图冷启动耗时除以名义批数为 33–106 ms，`:/Documentation` 的三类探针违反 (c)。诊断运行 `run-20261007T193500Z.json` 将 H_max 档视图 0 的总计时分段：根链段固定 19 语句，三类样本 p95 为 53–58 ms；视图段 20/24 语句，p95 为 43–49 ms。因此修复同时涉及两段：设计 §3.3 允许批量写入与无预算单提交快路；设计 §4.4 允许末批在水位和就绪已同事务写入后直接返回终态，不再开一笔空的终止检查事务。目标是 H=1231、B=1000 的根链冷启动段 ≤59 语句、单提交快路至多一笔事务，视图末批不再额外开终止事务；(c) 的实际时间仍须在本地提交后、推送前做非判定诊断，因为改动也会缩短冷启动分母，不能从语句数保证通过。实现限于 `view_root_chain.rs` 与 `view_projection_service.rs`，维持锁、预算、冲突、部分推进及停止语义。不得调整冻结参数或放宽阈值。诊断使用临时计时探针，运行后已恢复 `examples/view_bench.rs` 的原 sha256 `a1464857…`；全部失败与运行证据原样保留。 **Out of scope:** 过滤器上限与 HP-24 冻结基准本身。本卡发布后若 (c)(d) 仍失败，按 ER-10 新建 FIX-HP-02，HP-24 保持 blocked；不得修改本卡的不可变发布。 -**Current evidence:** `target/tmp/view_bench/run-20261007T192312Z.json`：H=1231 根链段 2484 语句、视图段 38/41 语句；(c) 三项违反，(d) 2522/2525 > 100；规模上限 PASS。`run-20261007T193500Z.json` 的诊断日志在 `target/tmp/view_bench/diagnostic.log`，三个探针类各 50 个计量样本，分段 p95 见 Description。`src/jupiter/storage/view_root_chain.rs` 的 `insert_scan_rows` 和 `insert_segment` 逐行执行 SQL；`extend_root_chain_inner` 对单提交追加仍经过起扫、回走、接入三笔事务。`ViewProjectionService::catch_up` 在末批已调用 `mark_ready_if_covered` 后仍以 `Advanced` 开下一笔空终止事务。 +**Current evidence:** 以下为修复前证据。`target/tmp/view_bench/run-20261007T192312Z.json`：H=1231 根链段 2484 语句、视图段 38/41 语句；(c) 三项违反，(d) 2522/2525 > 100；规模上限 PASS。`run-20261007T193500Z.json` 的诊断日志在 `target/tmp/view_bench/diagnostic.log`，三个探针类各 50 个计量样本，分段 p95 见 Description。修复前 `src/jupiter/storage/view_root_chain.rs` 的 `insert_scan_rows` 和 `insert_segment` 逐行执行 SQL；`extend_root_chain_inner` 对单提交追加仍经过起扫、回走、接入三笔事务。修复前 `ViewProjectionService::catch_up` 在末批已调用 `mark_ready_if_covered` 后仍以 `Advanced` 开下一笔空终止事务。 + +**推送前隔离诊断（2026-10-08）：** 在 FIX 提交 `9685337` 的独立 worktree 中，只复制 HP-24 的 `src/lib.rs` 接缝、`examples/view_bench.rs` 与冻结 pack，连续运行三次同一参数。三份诊断均为 (a)–(d) 与规模上限 PASS；H=1231 根链段均为 27 条，根链加两个视图的语句数分别为 58、61 条，(c) 最大比值为 0.8705–0.9328（阈值 1.0）。诊断文件保留在该 worktree 的 `target/tmp/view_bench/`，未复制到 HP-24 正式判定目录。 **Acceptance criteria:** -- [ ] lib 级语句计数用例在 H=64 与 H=128、B=1000 的两档证明根链冷启动 SQL 条数相等，且均 ≤29；按每增加一批和每增加一个 1000 行 SQL 分块至多各增加 15 条的上界，推得 H=1231、B=1000 的根链段 ≤59 条，满足 (d) 两视图约 38/41 条的 100 条总预算。 -- [ ] 无预算单提交合法追加时，根链在同一持锁事务内接入并返回 CaughtUp。 -- [ ] 带预算追加的结果、消耗与暂存状态与旧算法相同。 -- [ ] 冲突及不连续输入保留原有 fail-closed 结果与持久证据。 -- [ ] `catch_up` 末批已覆盖 `main@/` 时不再开空终止事务，并返回 Ready。 -- [ ] 末批未覆盖 `main@/` 时返回 MainNotCovered；停止与部分推进结果不变。 -- [ ] 本卡的根链与投影服务定向用例通过，包含语句计数、快路、冲突和终态。 -- [ ] 设计 §3.3 与 §4.4 写明优化的前提、边界和回退路径。 +- [x] lib 级语句计数用例在 H=64 与 H=128、B=1000 的两档证明根链冷启动 SQL 条数相等，且均 ≤29；按每增加一批和每增加一个 1000 行 SQL 分块至多各增加 15 条的上界，推得 H=1231、B=1000 的根链段 ≤59 条，满足 (d) 两视图约 38/41 条的 100 条总预算。 +- [x] 无预算单提交合法追加时，根链在同一持锁事务内接入并返回 CaughtUp。 +- [x] 带预算追加的结果、消耗与暂存状态与旧算法相同。 +- [x] 冲突及不连续输入保留原有 fail-closed 结果与持久证据。 +- [x] `catch_up` 末批已覆盖 `main@/` 时不再开空终止事务，并返回 Ready。 +- [x] 末批未覆盖 `main@/` 时返回 MainNotCovered；停止与部分推进结果不变。 +- [x] 本卡的根链与投影服务定向用例通过，包含语句计数、快路、冲突和终态。 +- [x] 设计 §3.3 与 §4.4 写明优化的前提、边界和回退路径。 **Verification:** -- [ ] `source .env.test && cargo test -p mega2 --lib 'jupiter::storage::view_root_chain::tests::'`：全部通过，含新增批量语句数与快路性质用例。 -- [ ] `source .env.test && cargo test -p mega2 --lib 'jupiter::service::view_projection_service::tests::'`：全部通过，含新增末批终态与语句数用例。 -- [ ] `rg -n '单提交快路|批量 SQL' docs/refactoring/history-projection.md`：§3.3 有命中。 -- [ ] `rg -n '末批终态' docs/refactoring/history-projection.md`：§4.4 有命中。 +- [x] `source .env.test && cargo test -p mega2 --lib 'jupiter::storage::view_root_chain::tests::'`：全部通过，含新增批量语句数与快路性质用例。 +- [x] `source .env.test && cargo test -p mega2 --lib 'jupiter::service::view_projection_service::tests::'`：全部通过，含新增末批终态与语句数用例。 +- [x] `rg -n '单提交快路|批量 SQL' docs/refactoring/history-projection.md`：§3.3 有命中。 +- [x] `rg -n '末批终态' docs/refactoring/history-projection.md`：§4.4 有命中。 ER-04 C 组五条精确命令和两个零警告构建门另执行，不计入本卡 VER。提交前后分别记录验收树与提交树；HP-24 未提交的接缝与 example 不进入本卡提交，在以本卡提交为 HEAD 的隔离 worktree 中重跑 C 组，确认 exact tree 后再推送。在同一隔离 worktree 中复制 HP-24 的接缝与 example（仅这两处允许出现在 benchmark 的工作树状态），并放入冻结 pack 的独立副本；于推送前运行同一冻结参数的非判定诊断。该 worktree 的 `target/tmp/view_bench/` 与主工作树分离，不把诊断的 `run-*.json` 复制到 HP-24 的判定目录。若 (c)(d) 预计不通过，先继续修订本卡实现与设计，不发布。ER-05 要求 Claude PASS 与独立 Codex PASS。Docker Build Action 按发布者指令不等待、不查询。 @@ -6181,6 +6184,9 @@ Result 只允许 `PASS` 或 `FAIL`。`FAIL` 必须列出 P0/P1 条目，并在 | FIXHP01-PLAN-R1 | FIX-HP-01 初稿与设计 §3.3，Claude CLI 只读审查 | FAIL | P1：判据 (c) 未分段诊断；AC 与发布闭环；依赖边和顺序未登记；模板字段及验证命令不合规；修订历史未登记。已用分段诊断并修订 §3.3/§4.4、卡片、DAG 与表格，待 R2 复审 | P2：预算语义、冲突证据、参数上限、提交树验证均纳入修订 | `target/tmp/view_bench/preflight/fix-hp01-plan-review.txt`、`diagnostic.log` | | FIXHP01-PLAN-R2 | FIX-HP-01 与设计 §3.3/§4.4 修订，Claude CLI 只读复审 | FAIL | P1：发布前缺少判据 (c) 的实际诊断与量化预期；AC 1 依赖发布后的基准；Phase 6 未登记 FIX；HP-26 对原 35 张卡的文字引用过期。已补推送前隔离 worktree 诊断、lib 语句上界、Phase 6 与 HP-26 文本，待 R3 复审 | P2：设计已明确快路 `ON CONFLICT DO NOTHING` 的冲突回退，诊断与正式证据目录分离 | `target/tmp/view_bench/preflight/fix-hp01-plan-review-r2.txt` | | FIXHP01-PLAN-R3 | FIX-HP-01 可执行计划与设计，Claude CLI 只读复审 | PASS | P0 0；P1 0 | P2：多余空行待清理；AC 1 对增加一批的 +15 上界可再加一档 H=128、B=64 实测，现已在定向用例实施 | `target/tmp/view_bench/preflight/fix-hp01-plan-review-r3.txt`；`VERDICT: PASS` | +| FIXHP01-CODE-R1 | FIX-HP-01 根链批写、单提交快路与追赶末批终态，Claude CLI 只读审查 | PASS | P0 0；P1 0 | 指出缺失链尾提交行时快路可能掩盖 MissingFirstParent；已加 EXISTS 条件、回退用例与设计说明，R2 复核关闭 | `target/tmp/view_bench/preflight/fix-hp01-code-review.txt` | +| FIXHP01-CODE-R2 | FIX-HP-01 修订实现，Claude CLI 只读复审 | PASS | P0 0；P1 0 | 缺失链尾提交行的回退与暂存证据已核对；根链 16/16、投影 15/15 定向测试在最终源码上通过 | `target/tmp/view_bench/preflight/fix-hp01-code-review-r2.txt`、`fix-root-tests-r3.log`、`fix-projection-tests-final.log` | +| FIXHP01-CODE-R3 | FIX-HP-01 提交 `9685337`，Codex CLI 独立只读复核 | PASS | P0 0；P1 0 | 锁内快路、预算保留、批写冲突回退和末批提交后返回均无阻断问题；第二 Claude 视角也 PASS | `target/tmp/view_bench/preflight/fix-codex-review.txt`、`fix-second-review.txt`；`VERDICT: PASS` | ## 非目标与延后项