diff --git a/docs/architecture/rfcs/loopx-overall-roadmap-v0.md b/docs/architecture/rfcs/loopx-overall-roadmap-v0.md index 685ce4662b..f67b47a571 100644 --- a/docs/architecture/rfcs/loopx-overall-roadmap-v0.md +++ b/docs/architecture/rfcs/loopx-overall-roadmap-v0.md @@ -96,7 +96,7 @@ These directly determine whether a long-running team is usable. A directory or R | [Goal Vision/replan](../../reference/protocols/goal-vision-replan-contract-v0.md), [work graph](../../reference/protocols/task-graph-projection-v0.md), [peer runtime](../../reference/protocols/peer-agent-runtime-v1.md), [supervisor](../../reference/protocols/peer-supervisor-v0.md) | S2/S3 | Exercise dependencies, replanning, acceptance and handoff in one real case; aggregate closeout consumes acceptance facts | | [Quota](../../quota-allocation.md), [cadence](../../operations/long-task-cadence-policy.md), [attention](../../operations/attention-queue.md) | S5/S7 | Budget exhaustion, deferral and blocking expose next triggers/readback; scale without frequent full-state polling | | [Material lifecycle](../../reference/protocols/material-lifecycle-architecture-v0.md), [material frontier](../../reference/protocols/agent-material-frontier-v0.md), [authority registration](../../operations/authority-source-registration.md) | S6 | Agents discover roadmap/RFC revisions and record reads; reading grants neither agreement nor authority; archival preserves raw-source ownership | -| [Decision Context](../../../loopx/capabilities/decision_context/README.md), [Reward Memory](../../../loopx/capabilities/reward_memory/README.md), [Semantic Preference](../../../loopx/capabilities/semantic_preference/README.md), [Turn Recall](../../../loopx/capabilities/agent_turn_recall/README.md) | S6/S11 | Distinguish facts/preferences/advice/attribution/authority; scoped recall, expiry and outcome-feedback counterexamples first | +| [Decision Context](../../../loopx/capabilities/decision_context/README.md), [Reward Memory](../../../loopx/capabilities/reward_memory/README.md), [Semantic Preference](../../../loopx/capabilities/semantic_preference/README.md), [Turn Recall](../../../loopx/capabilities/agent_turn_recall/README.md) | S6/S11 | Distinguish facts/preferences/advice/attribution/authority; scoped recall, expiry and outcome-feedback counterexamples first. Start review learning with the [#5944 Git-owned experience](../../../loopx/capabilities/pr_review_queue/experiences/README.md): preserve machine and maintainer-directed frames, deliver qualified advice to the explicitly opted-in review Agent, then verify current-head adoption and useful review outcomes. Context delivery closes no quality/utility gate; held-out false-blocker, cost and attention controls follow [the existing utility RFC](post-outcome-memory-utility-attribution-v0.md#91-review-learning-pilot-repository-experience-before-utility) | | [Issue Fix](../../../loopx/capabilities/issue_fix/README.md), [PR Review](../../../loopx/capabilities/pr_review_queue/README.md), [Change Quality](../../../loopx/capabilities/change_quality/README.md), [Integration Branch](../../../loopx/capabilities/integration_branch/README.md), [Change Window](../../../loopx/capabilities/repository_change_window/README.md) | S8/S12 | Engineering caller: issue→implementation→independent review→validation→authorized delivery; source-head drift invalidates old evidence | | [Explore](../../../loopx/capabilities/explore/README.md), [Benchmark Toolkit](../../../loopx/capabilities/benchmark_toolkit/README.md), [auto-research](../../product/use-cases/auto-research/README.md) | S8/S11 | Second collaboration journey: question/hypothesis→parallel experiments→independent interpretation→successor; no uplift/failure remain valid outcomes | | [Content Operations](../../../loopx/capabilities/content_ops/README.md), [Periodic Report](../../../loopx/capabilities/periodic_report/README.md), [office operations](../../product/use-cases/office-operations/README.md), [domain packs](../../product/domain-capability-packs.md) | S5/S8 | Public-safe projections feed sinks; separate drafts/review/publish authority/outcomes; generic sinks do not depend on project-private documents | diff --git a/docs/architecture/rfcs/loopx-overall-roadmap-v0.zh-CN.md b/docs/architecture/rfcs/loopx-overall-roadmap-v0.zh-CN.md index bfc5861654..15628d42d8 100644 --- a/docs/architecture/rfcs/loopx-overall-roadmap-v0.zh-CN.md +++ b/docs/architecture/rfcs/loopx-overall-roadmap-v0.zh-CN.md @@ -82,7 +82,7 @@ managed 与 attached 的工作对话都应能持续在 LoopX 中进行：沿用 | [Goal Vision/replan](../../reference/protocols/goal-vision-replan-contract-v0.md)、[work graph](../../reference/protocols/task-graph-projection-v0.md)、[peer runtime](../../reference/protocols/peer-agent-runtime-v1.md)、[监督](../../reference/protocols/peer-supervisor-v0.md) | S2/S3 | 将跨工作依赖、重规划、验收与 handoff 放进同一个真实案例；aggregate closeout 必须消费验收事实 | | [quota](../../quota-allocation.md)、[cadence](../../operations/long-task-cadence-policy.md)、[attention](../../operations/attention-queue.md) | S5/S7 | 预算耗尽/延期/被阻塞时有明确下一次触发及用户回读；百 Agent 不靠高频全文轮询 | | [材料生命周期](../../reference/protocols/material-lifecycle-architecture-v0.zh-CN.md)、[材料 frontier](../../reference/protocols/agent-material-frontier-v0.md)、[authority 注册](../../operations/authority-source-registration.md) | S6 | 路线/RFC 更新能由 Agent 按 revision 发现并登记阅读；read receipt 不表示同意或获得权限；归档不丢原始来源 | -| [Decision Context](../../../loopx/capabilities/decision_context/README.md)、[Reward Memory](../../../loopx/capabilities/reward_memory/README.md)、[Semantic Preference](../../../loopx/capabilities/semantic_preference/README.md)、[Turn Recall](../../../loopx/capabilities/agent_turn_recall/README.md) | S6/S11 | 区分事实、偏好、建议、归因和权威；同一 scope 的回忆/失效/结果反馈负例先行 | +| [Decision Context](../../../loopx/capabilities/decision_context/README.md)、[Reward Memory](../../../loopx/capabilities/reward_memory/README.md)、[Semantic Preference](../../../loopx/capabilities/semantic_preference/README.md)、[Turn Recall](../../../loopx/capabilities/agent_turn_recall/README.md) | S6/S11 | 区分事实、偏好、建议、归因和权威；同一 scope 的回忆/失效/结果反馈负例先行。以 [#5944 的 Git 经验](../../../loopx/capabilities/pr_review_queue/experiences/README.zh-CN.md)起步：保留机器与维护者指引下的口径，将合格建议投递到明确 opt-in 的 review Agent，再验证当前 head 的采用与有用评审结果。投递不关闭质量/utility gate；held-out 误阻断、成本和注意力对照遵循[现有 utility RFC](post-outcome-memory-utility-attribution-v0.zh-CN.md#91-评审学习试点先沉淀仓库经验再证明效用) | | [Issue Fix](../../../loopx/capabilities/issue_fix/README.md)、[PR Review](../../../loopx/capabilities/pr_review_queue/README.md)、[Change Quality](../../../loopx/capabilities/change_quality/README.md)、[Integration Branch](../../../loopx/capabilities/integration_branch/README.md)、[Change Window](../../../loopx/capabilities/repository_change_window/README.md) | S8/S12 | 作为工程团队的端到端 caller：问题→实现→独立 review→验证→批准范围内交付；head 漂移不复用旧证据 | | [Explore](../../../loopx/capabilities/explore/README.md)、[Benchmark Toolkit](../../../loopx/capabilities/benchmark_toolkit/README.md)、[auto-research](../../product/use-cases/auto-research/README.md) | S8/S11 | 作为第二类协作旅程：问题/假设→并行实验→独立解释→后继；无提升/失败也形成有效结果 | | [Content Operations](../../../loopx/capabilities/content_ops/README.md)、[Periodic Report](../../../loopx/capabilities/periodic_report/README.md)、[office operations](../../product/use-cases/office-operations/README.md)、[domain packs](../../product/domain-capability-packs.md) | S5/S8 | 公共投影驱动显示；草稿、review、发布权限与结果分离；不将 generic sink 绑定项目私有文档 | diff --git a/docs/architecture/rfcs/post-outcome-memory-utility-attribution-v0.md b/docs/architecture/rfcs/post-outcome-memory-utility-attribution-v0.md index 38dd304c85..8360f386ef 100644 --- a/docs/architecture/rfcs/post-outcome-memory-utility-attribution-v0.md +++ b/docs/architecture/rfcs/post-outcome-memory-utility-attribution-v0.md @@ -1,6 +1,6 @@ # RFC: Post-Outcome Memory Utility Attribution v0 -- Status: Accepted; Stage 1 shipped, Stage 2 delivered by this PR, Stage 3+ remain proposed +- Status: Accepted; Stage 1 and Stage 2 implemented, Stage 3+ remain proposed - Supersedes / closes: none - Date: 2026-08-15 - Tracking issues: [#3214](https://github.com/huangruiteng/loopx/issues/3214), [#3824](https://github.com/huangruiteng/loopx/issues/3824) @@ -368,6 +368,48 @@ bounded rank-prior effect plus readback receipt. Stage 4 runs a bounded pilot with held-out or counterfactual evaluation before any default-on ranking influence. +### 9.1 Review-learning pilot: repository experience before utility + +[PR #5944](https://github.com/loopx-project/loopx/pull/5944) supplies the first +[repository-owned comparison and experience](../../../loopx/capabilities/pr_review_queue/experiences/README.md). +Earlier machine approvals covered technical recovery design; a later +maintainer-directed model review requested a first useful recovered Goal, +earlier packaged interaction and useful post-recovery results. Preserve each +frame, actor provenance, API target commit, body-declared head, evidence and +unresolved outcome. Different frames and source heads prevent treating this +as a controlled experiment or automatic gold label. + +The first pilot phase stores public distilled evidence in Git using the existing +`procedural_experience_contract_v0`. `pull-request-review` owns the source and +read-only file adapter; `reward_memory` retains qualification, recall and the +TypeScript context-delivery decision. Self-repair points to the same case and +repairs its owning workflow. No second store, shared decision owner, utility +score or provider write is introduced. + +A registered Agent needs qualified Reward Memory enablement, automatic recall +and an explicit `pull_request_review.review` surface. Its actionable PR packet +can deliver at most three repository-matched, exact-readback experiences. +Historical verdict labels stay outside retrieval inputs. Disabled, other-surface, +unqualified and unrelated-repository calls preserve their base behavior; +malformed/changed sources fail open. Native public advice extends only this +opt-in caller boundary; provider ranking, configuration, action authority and +exact-head work selection are unchanged. + +This phase accepts a versioned case, qualified experience, real source readback +and JSON/Markdown delivery at the production CLI, plus off-parity and failure +controls. Context delivery does not establish semantic adoption or better +reviews. Explain adoption, rejection or irrelevance in the existing +`problem_context` judgment with current-head evidence. + +The next phase pins one common review frame, artifact revision, model and budget +across baseline/experience arms on held-out cases, including justified internal +prerequisites and false-blocker controls. Retain disagreement; measure actionable +findings, incorrect blockers, review cost and human attention per accepted review. +A maintainer correction is evidence, not an unconditional reward. Only verified +application and work-outcome lineage can feed the existing utility observation +and reducer. The original case has no such lineage, so its utility is unproven. +This pilot closes no Stage 3/4 ranking or general review-quality qualification. + ## 10. Validation criteria Focused fixtures must prove: diff --git a/docs/architecture/rfcs/post-outcome-memory-utility-attribution-v0.zh-CN.md b/docs/architecture/rfcs/post-outcome-memory-utility-attribution-v0.zh-CN.md index b735e83be5..8d1d03216c 100644 --- a/docs/architecture/rfcs/post-outcome-memory-utility-attribution-v0.zh-CN.md +++ b/docs/architecture/rfcs/post-outcome-memory-utility-attribution-v0.zh-CN.md @@ -1,6 +1,6 @@ # RFC：结果后记忆效用归因 v0 -- 状态：已接受；Stage 1 已交付，Stage 2 由本 PR 交付，Stage 3+ 仍为提案 +- 状态：已接受；Stage 1 与 Stage 2 已实现，Stage 3+ 仍为提案 - 替代 / 关闭：无 - 日期：2026-08-15 - 跟踪 issue：[#3214](https://github.com/huangruiteng/loopx/issues/3214)、[#3824](https://github.com/huangruiteng/loopx/issues/3824) @@ -282,6 +282,38 @@ Stage 3 增加 OpenViking readback；只有 provider 协议真实存在时，才 Stage 4 先做有界 pilot、held-out 或 counterfactual evaluation，再考虑让排序影响 default-on。 +### 9.1 评审学习试点：先沉淀仓库经验，再证明效用 + +以 [PR #5944](https://github.com/loopx-project/loopx/pull/5944)作为第一条 +[仓库持有的判断对照与经验](../../../loopx/capabilities/pr_review_queue/experiences/README.zh-CN.md)。 +先前机器评审覆盖恢复的技术设计，后续维护者指引下的模型评审要求明确第一个有用的 +恢复 Goal、更早的打包交互和恢复后有用结果。分别保留口径、actor 来源、API target +commit、正文声明的 head、证据和未解决的结果。口径与 source head 差异意味着 +这不是受控实验，也不能自动生成金标准标签。 + +试点第一阶段把公共蒸馏证据存入 Git，复用现有 +`procedural_experience_contract_v0`。`pull-request-review` 持有来源与只读文件 +adapter；`reward_memory` 保留资格化、recall 和 TypeScript 上下文投递判断。 +self-repair 指向同一个案例并修复原有工作流，不新建 store、共享 decision owner、 +utility 分数或 provider 写入。 + +已注册 Agent 须具备资格化的 Reward Memory enablement、自动 recall 和明确的 +`pull_request_review.review` surface。可执行 PR packet 才可投递最多三条仓库匹配、 +精确回读的经验；历史 verdict 标签不进入检索输入。关闭、其他 surface、未资格化 +和无关仓库调用保持原行为；损坏或变化中的来源 fail-open。内置公共建议只扩展此 +opt-in caller 边界，不改变 provider ranking、配置、行动权限或 exact-head 选工。 + +本阶段验收是版本化案例、合格经验、真实来源回读与生产 CLI 的 JSON/Markdown 投递， +加关闭等价和失败对照。上下文投递不证明语义采用或评审质量变好。 +在现有 `problem_context` 判断中，用当前 head 证据说明采用、拒绝或不适用。 + +下一阶段在 held-out 案例上固定共同的评审口径、产物 revision、模型和预算，比较 +baseline/experience 两组，包含合理内部前置阶段与误阻断对照。保留分歧，衡量每份 +被接受评审的可行动发现、错误 blocker、评审成本与人工注意力。 +维护者纠正是证据，不是无条件 reward。只有验证后的采用与工作结果 lineage 才能进入 +现有 utility observation/reducer；原案例没有该 lineage，效用仍未证明。 +此试点不关闭 Stage 3/4 排序资格或通用评审质量资格。 + ## 10. 验证标准 Focused fixture 必须证明： diff --git a/loopx/capabilities/pr_review_queue/README.md b/loopx/capabilities/pr_review_queue/README.md index 9fc11f325e..734cb462cb 100644 --- a/loopx/capabilities/pr_review_queue/README.md +++ b/loopx/capabilities/pr_review_queue/README.md @@ -1275,3 +1275,21 @@ its existing values, and a new namespace uses capability defaults. 审阅优先级），部分修改保留既有目标值，新覆盖使用 capability 默认值。关闭时不 查询、轮询或等待 CI；本地必需验证、当前提交评审、评论及权限检查仍然适用。 GitHub `BLOCKED` 只提示另需管理员授权，不授予合并权限。 + +## Repository experience for opted-in review Agents + +The existing Reward Memory experiment can deliver Git-versioned procedural +advice in actionable review packets. Enable the registered Agent, automatic +recall and the explicit `pull_request_review.review` surface in its existing +configuration. See [operation, readback and disable](experiences/README.md). +The [#5944 comparison](experiences/pr-5944-review-frame.md) is the first stored +case. Historical verdicts are outside retrieval; packet delivery proves neither +adoption nor utility. Base/off paths, queue selection and review authority retain +their existing contracts; the native file reader writes no provider. + +已开启的 review Agent 可通过现有 Reward Memory 实验，在可执行 review packet 中 +收到 Git 版本化的过程经验。须启用已注册 Agent、自动 recall，并在原配置中明确指定 +`pull_request_review.review` surface，参见[操作、回读与关闭](experiences/README.zh-CN.md)。 +[#5944 判断对照](experiences/pr-5944-review-frame.md)是第一条案例。 +历史 verdict 不进入检索，packet 投递不证明采用或效用。关闭路径、选工和评审权限 +仍遵循现有合同；内置文件 reader 不写 provider。 diff --git a/loopx/capabilities/pr_review_queue/experiences/README.md b/loopx/capabilities/pr_review_queue/experiences/README.md new file mode 100644 index 0000000000..2cf15819e3 --- /dev/null +++ b/loopx/capabilities/pr_review_queue/experiences/README.md @@ -0,0 +1,99 @@ +# Repository review experience + +These Git-versioned public sources belong to `pull-request-review`. Each active +JSON file under `experiences///` uses the existing +`procedural_experience_contract_v0`; there is no new memory schema or writable +provider. Git history retains revisions. Replace or remove a source through a +reviewed PR to update or retire it; keep one active revision of a case. + +The [PR #5944 case](pr-5944-review-frame.md) records the machine reviews and the +maintainer-directed model review separately. Its [distilled experience](loopx-project/loopx/pr-5944-v1.json) +contains applicability, observation, attribution, future behavior, limitations +and public evidence refs. Historical verdict labels stay in the case document, +which retrieval does not load. Self-repair points to this same source rather +than maintaining another copy of the lesson. + +## Use in an enabled review Agent + +Use the existing **Reward Memory experiment** editor in the Goal's Capability +Center to enable the registered review Agent and apply its versioned config. +The config must have `automation.automatic_recall=true` and an explicit +`pull_request_review.review` surface, assigned to its existing corpus ids with +a recall profile. Keep the normal provider preflight and exact readback. +See [Reward Memory configuration](../../reward_memory/README.md); no new toggle, +provider write or automatic import is added here. + +Add this surface to the existing private config, substituting the owned corpus +id, and include the surface in that corpus and standing policy's `surface_ids`: + +```json +{ + "surface_id": "pull_request_review.review", + "adapter": "scoped_feedback", + "corpus_ids": ["reviewed_experiences"], + "ingest_corpus_id": "reviewed_experiences", + "recall_profile": { + "profile_id": "pr_review", + "mode": "function_boundary", + "max_queries": 1, + "limit": 3 + } +} +``` + +Apply/read back through the existing Goal editor or CLI: + +```sh +loopx configure-goal --goal-id GOAL \ + --reward-memory-config .loopx/config/reward-memory/experiment.json \ + --reward-memory-agent REVIEWER --execute +``` + +```sh +loopx reward-memory experiment-status --goal-id GOAL --agent-id REVIEWER --format json +loopx pr-review --goal-id GOAL --agent-id REVIEWER --repo loopx-project/loopx \ + --target-exact-head NUMBER@HEAD_OID --format json +``` + +An actionable review row may contain `repository_experience.decision` and +`repository_experience.guidance`. The built-in reader searches only the selected +repository's bundled files, validates the existing experience contract, and +checks exact file readback before delivering at most three matching experiences. +It composes the existing corpus, candidate, recall and TypeScript decision +owners. BM25 orders lexical matches; it does not decide applicability or the +verdict. The SDK `provider_call_count` counts this local reader invocation; +there is no external memory-provider call or write in this path. + +The review skill reads the guidance before judging the current diff. Record +adoption, rejection or irrelevance with current-head evidence in the existing +`problem_context` judgment. `context_delivery_verified=true` proves packet +delivery only: `semantic_disposition=null`, `decision_consumption_complete=false` +and `utility_verified=false` remain until their separate evidence exists. The +ordinary five-block review still returns through the existing conversation; +this internal source reader adds no separate product screen or agent authority. + +This is a disclosed opt-in behavior extension: qualified Agents with automatic +recall and this review surface receive matching repository advice. Disabled, +unbound, other-surface, stale-config, unqualified and different-repository calls +preserve the original packet. Inventory-only and merge-readiness rows receive +no new recall. Invalid or changing assets preserve empty guidance and never +become a user gate. Empty guidance and its decision remain visible in JSON and +Markdown; no failed retrieval is reported as adopted learning. + +Disable with `automation.automatic_recall=false` and reapply the existing config, +or remove the review surface and reapply it. Clear the whole experiment with +`loopx configure-goal --goal-id GOAL --clear-reward-memory-config --execute`. +Downgrading removes this reader; there is no provider or persisted runtime state +to migrate. Repository publication, review/comment, push, merge and cross-Agent +writes retain their existing authorization boundaries. + +## Next evidence + +The current case is a curated experience and a working context-delivery path, +not a measured improvement in review quality. A later pilot must pin the same +review frame, artifacts, model and budget, blind held-out cases to old verdicts, +and retain disagreement, false blockers, cost and human attention. Verified +adoption and outcomes may then enter the existing +[utility-attribution RFC](../../../../docs/architecture/rfcs/post-outcome-memory-utility-attribution-v0.md). + +[中文](README.zh-CN.md) diff --git a/loopx/capabilities/pr_review_queue/experiences/README.zh-CN.md b/loopx/capabilities/pr_review_queue/experiences/README.zh-CN.md new file mode 100644 index 0000000000..0e7ac0b8a5 --- /dev/null +++ b/loopx/capabilities/pr_review_queue/experiences/README.zh-CN.md @@ -0,0 +1,87 @@ +# 仓库中的评审经验 + +这些公共来源随 Git 版本保存，由 `pull-request-review` 持有。 +`experiences///` 下的有效 JSON 使用现有 +`procedural_experience_contract_v0`，不新增记忆 schema 或可写 provider。 +通过经过 review 的 PR 替换或删除来源来更新、退休经验；每个案例只保留一个有效版本， +历史由 Git 保存。 + +[#5944 案例](pr-5944-review-frame.md)分别保存机器评审和维护者指引下的模型评审。 +[蒸馏经验](loopx-project/loopx/pr-5944-v1.json)包含适用条件、观察、归因、未来行为、 +局限与公共证据引用。历史 verdict 标签保留在案例文档中，召回不读取这份文档。 +self-repair 指向同一来源，不另存一份经验正文。 + +## 让已开启的 review Agent 使用 + +在 Goal 能力中心现有 **Reward Memory 实验**编辑器中启用已注册的 review Agent， +应用其版本化配置。配置须包含 `automation.automatic_recall=true`，以及明确的 +`pull_request_review.review` surface，并为它指定现有 corpus ids 和 recall profile。 +保留正常的 provider preflight 和精确回读。参见 +[Reward Memory 配置](../../reward_memory/README.zh-CN.md)；本切片不新增开关、 +provider 写入或自动导入。 + +在原有私有配置中添加如下 surface，将 corpus id 替换成已有 owner 的 id， +并在对应 corpus 和 standing policy 的 `surface_ids` 中包含此 surface： + +```json +{ + "surface_id": "pull_request_review.review", + "adapter": "scoped_feedback", + "corpus_ids": ["reviewed_experiences"], + "ingest_corpus_id": "reviewed_experiences", + "recall_profile": { + "profile_id": "pr_review", + "mode": "function_boundary", + "max_queries": 1, + "limit": 3 + } +} +``` + +通过现有 Goal 编辑器或 CLI 应用并回读： + +```sh +loopx configure-goal --goal-id GOAL \ + --reward-memory-config .loopx/config/reward-memory/experiment.json \ + --reward-memory-agent REVIEWER --execute +``` + +```sh +loopx reward-memory experiment-status --goal-id GOAL --agent-id REVIEWER --format json +loopx pr-review --goal-id GOAL --agent-id REVIEWER --repo loopx-project/loopx \ + --target-exact-head NUMBER@HEAD_OID --format json +``` + +可执行 review 行可能包含 `repository_experience.decision` 和 +`repository_experience.guidance`。内置 reader 只搜索所选仓库的打包文件， +校验现有经验契约，并在投递最多三条匹配经验前精确回读文件。 +它组合现有 corpus、candidate、recall 与 TypeScript decision owner。 +BM25 只排列词汇匹配，不判断适用性或 verdict。SDK 的 `provider_call_count` +计数本地 reader 调用；这条路径不调用或写入外部记忆 provider。 + +review skill 在判断当前 diff 前读取建议，在现有 `problem_context` 判断中， +以当前 head 证据说明采用、拒绝或不适用。`context_delivery_verified=true` 只证明 +packet 投递；`semantic_disposition=null`、`decision_consumption_complete=false` +和 `utility_verified=false` 保留，直到各自有独立证据。 +普通五段式评审仍返回原有会话；此内部来源 reader 不另建产品页面，也不授予 Agent 权限。 + +这是明确披露的 opt-in 行为扩展：已资格化、开启自动 recall 并配置 review surface +的 Agent 会收到匹配的仓库建议。关闭、未绑定、其他 surface、配置过期、未资格化和 +评审其他仓库时保持原 packet。仅 inventory 或 merge-readiness 行不新增 recall。 +无效或读取中变化的文件保留空 guidance，不形成用户 gate。 +JSON 和 Markdown 可回读空结果及其 decision；检索失败不被标成经验已采用。 + +关闭时设 `automation.automatic_recall=false` 并重新应用现有配置，或移除 review +surface 后重新应用。清除整个实验： +`loopx configure-goal --goal-id GOAL --clear-reward-memory-config --execute`。 +降级移除此 reader，无 provider 或持久运行状态需要迁移。仓库发布、review/comment、 +push、merge 与跨 Agent 写入仍遵循原授权边界。 + +## 下一步证据 + +当前案例交付的是已整理经验与可运行的上下文投递路径，尚未证明评审质量提升。 +后续 pilot 必须固定同一评审口径、产物、模型和预算，对 held-out 案例隐藏旧 verdict， +保留分歧、误阻断、成本和人工注意力。验证后的采用与工作结果，才可进入现有 +[效用归因 RFC](../../../../docs/architecture/rfcs/post-outcome-memory-utility-attribution-v0.zh-CN.md)。 + +[English](README.md) diff --git a/loopx/capabilities/pr_review_queue/experiences/loopx-project/loopx/pr-5944-v1.json b/loopx/capabilities/pr_review_queue/experiences/loopx-project/loopx/pr-5944-v1.json new file mode 100644 index 0000000000..d80997d704 --- /dev/null +++ b/loopx/capabilities/pr_review_queue/experiences/loopx-project/loopx/pr-5944-v1.json @@ -0,0 +1,29 @@ +{ + "schema_version": "procedural_experience_contract_v0", + "applicability": [ + "Reviewing a recovery RFC or phased architecture proposal against an accepted product roadmap", + "Technical design coverage passes while the first useful user outcome or packaged journey remains unclear" + ], + "observed_outcome": "The complete-state recovery RFC covered eight technical design criteria. A subsequent maintainer-directed product review requested a bounded first recovered Goal, earlier packaged interaction, and evidence of useful work after recovery. The revised implementation and its user benefit were still unverified when this experience was recorded.", + "attribution": "The earlier review emphasized technical coverage and treated user experience as inapplicable to a documentation change. The later review made product value and delivery order explicit. This is a difference in review frame, not a controlled same-head experiment or proof that one reviewer is always correct. Result checking verifies consistency, not the truth of either judgment.", + "future_behavior": { + "trigger": "An RFC defines delivery stages for recovery or another caller-facing capability, even when its diff contains only documentation.", + "action": "Read the current accepted roadmap and PR frame before the diff. Name the first useful user outcome, owning boundary, minimum packaged journey, and useful result after the operation. Check whether the proposed stages deliver these together. Keep refactors bounded to removing duplicate authority and making the next likely change easier.", + "validation": "Bind findings to the current exact head and roadmap acceptance. Walk one ordinary task through discovery, operation, failure or recovery, and result readback. For a design PR, review the specified acceptance journey and mark runtime evidence pending. Measure attention and maintenance cost per accepted result; do not infer value from test, receipt or PR counts.", + "stop_condition": "Ignore or refute this advice when the current change has a justified internal prerequisite boundary or different accepted scope. Do not require a completed runtime for a design PR, invent unrelated roadmap gates, inherit a historical verdict, or block solely because the human and machine judgments differ." + }, + "limitations": [ + "One maintainer-directed correction is advisory evidence, not a universal reward label, hard policy or additional action authority.", + "A structurally qualified experience and delivered context do not prove semantic adoption or improved review quality.", + "No memory recall or application lineage was recorded for the original review; do not fabricate a memory utility observation from the verdict difference.", + "Review API target commits and the heads named in earlier review bodies differ; preserve both in the companion case and do not claim a same-head counterfactual." + ], + "evidence_refs": [ + "github:loopx-project/loopx:issue:5940", + "github:loopx-project/loopx:pr:5944", + "github:loopx-project/loopx:review:5452444228", + "github:loopx-project/loopx:review:5453719155", + "github:loopx-project/loopx:review:5454100835", + "git:f4c21235dfa6695a7592c8023c9a150268e28f48" + ] +} diff --git a/loopx/capabilities/pr_review_queue/experiences/pr-5944-review-frame.md b/loopx/capabilities/pr_review_queue/experiences/pr-5944-review-frame.md new file mode 100644 index 0000000000..bd204d67c8 --- /dev/null +++ b/loopx/capabilities/pr_review_queue/experiences/pr-5944-review-frame.md @@ -0,0 +1,58 @@ +# PR #5944: technical coverage and product delivery judgment + +Recorded 2026-10-08. Public sources: +[recovery design issue #5940](https://github.com/loopx-project/loopx/issues/5940), +[RFC PR #5944](https://github.com/loopx-project/loopx/pull/5944). +This case is a distilled comparison, not a transcript or a reward label. + +| Published judgment | API target commit | Head declared in the review body | Review frame | +| --- | --- | --- | --- | +| [Machine APPROVE, 06:23:53 UTC](https://github.com/loopx-project/loopx/pull/5944#pullrequestreview-5452444228) | `f4c21235dfa6695a7592c8023c9a150268e28f48` | `4d98703445fb64e413f2cfb274617fc932b851cf` | Eight design criteria; owner authority, inert recovery, fencing and replay safety. User experience was declared inapplicable to a docs-only diff. | +| [Machine APPROVE, 08:18:28 UTC](https://github.com/loopx-project/loopx/pull/5944#pullrequestreview-5453719155) | `f4c21235dfa6695a7592c8023c9a150268e28f48` | `1741770661c990eab7d74814fd85ee2246910182` | Updated design/roadmap alignment and prior validation; the product journey remained outside the judgment. | +| [Maintainer-directed model REQUEST_CHANGES, 08:53:20 UTC](https://github.com/loopx-project/loopx/pull/5944#pullrequestreview-5454100835) | `f4c21235dfa6695a7592c8023c9a150268e28f48` | `f4c21235dfa6695a7592c8023c9a150268e28f48` | First useful recovered Goal, earlier minimum packaged journey, and useful post-recovery result with attention/maintenance evidence. | + +The final review was authored by a model under maintainer direction; it is not +an independent purely human adjudication. Account names alone do not establish +who reasoned about a PR. Preserve declared actor/model provenance in the linked +reviews. API target commits and body-declared heads differ in the approvals; +therefore this is not a controlled same-head baseline/treated comparison. + +The review capability's `check-result` can accept either internally consistent +verdict. Its success certifies packet/body consistency and required structure, +not evidence truth, product value or agreement with a maintainer. The subsequent +checked Request changes therefore does not contradict a successful checker. +It exposes a narrower earlier judgment and a newly explicit product frame. + +The three actionable changes requested were: + +1. Bound the first outcome to one local Goal and explain the user's recovery + problem, rather than treating complete state coverage as sufficient value. +2. Include minimum packaged verification/readback with M1, and recovery/failure + interaction with M3, instead of deferring the user journey to M5. +3. Verify useful work after recovery, human attention and maintenance cost; + reuse existing typed owners and keep refactoring bounded. + +At recording time the revised RFC, implemented recovery journey and resulting +benefit were **unverified**. No recall/application receipt from the earlier +reviews was available. Neither a terminal reward nor a positive/negative memory +utility observation can be derived from this disagreement alone. Subsequent +resolution belongs to a new reviewed revision with new evidence, preserving +these historical facts. + +The [machine-consumable experience](loopx-project/loopx/pr-5944-v1.json) retains +the procedure and its stop condition without historical verdict labels. Its +real consumer is the opted-in PR review packet; see [operation/readback](README.md). +Self-repair reuses `review_outcome_continuity_gap` to repair the missing review +frame. Reward Memory owns reusable advice and its separate utility evidence. + +## 中文判断 + +自动评审主要回答“八项技术设计是否覆盖”，后来的人机评审明确要求回答 +“第一阶段让用户得到什么、如何操作、恢复后是否产生有用结果”。 +`check-result` 通过只证明声明与格式一致，不证明产品判断正确，因此它可以通过 +一份一致的 APPROVE，也可以通过一份一致的 Request changes。 + +经验正文属于现有 reward memory 的 `procedural_experience`；self-repair +修复漏掉完整用户结果的评审工作流。人机判断是新的证据与口径，不自动成为金标准。 +原评审没有记忆采用链路，不能由 verdict 差异倒推出 memory utility。 +当前只确认公共评审分歧、修改要求与可复用过程；RFC 修订后的效果仍待验证。 diff --git a/loopx/capabilities/pr_review_queue/repository_experience.py b/loopx/capabilities/pr_review_queue/repository_experience.py new file mode 100644 index 0000000000..621592b851 --- /dev/null +++ b/loopx/capabilities/pr_review_queue/repository_experience.py @@ -0,0 +1,178 @@ +"""Read-only Git-published review experience through existing Reward Memory. + +Placement: pull-request-review owns these public source files and this file/lexical +adapter. reward_memory owns qualification, scope, recall and the TS delivery +decision. This adapter is not a configurable provider, store or decision owner. +""" +from __future__ import annotations + +from collections.abc import Mapping +from datetime import datetime, timezone +import hashlib +from importlib.resources import files +import json +import re +from typing import Any + +from ...lexical_retrieval import score_bm25 +from ..context_providers.base import ContextProviderItem, ContextProviderRetrieval +from ..reward_memory.application import build_active_reward_memory_record +from ..reward_memory.candidate_review import ( + build_reward_memory_candidate, review_reward_memory_candidate, +) +from ..reward_memory.decision import run_reward_memory_decision +from ..reward_memory.experience_quality import normalize_procedural_experience +from ..reward_memory.experiment import resolve_goal_reward_memory_experiment +from ..reward_memory.registry import normalize_reward_memory_corpus + + +REPOSITORY_REVIEW_SURFACE_ID = "pull_request_review.review" +REPOSITORY_REVIEW_CORPUS_ID = "repository_review_experiences" +REPOSITORY_REVIEW_PROVIDER_ID = "repository_review_experience" + + +class _RepositoryExperienceReader: + """Transient exact-corpus reader; only installed public assets are discovered.""" + + def __init__(self, repository: str, corpus: dict[str, Any]) -> None: + self.root = files(__package__).joinpath("experiences", *repository.split("/")) + self.corpus = corpus + + def retrieve(self, **request: Any) -> ContextProviderRetrieval: + sources = sorted(self.root.iterdir(), key=lambda entry: entry.name) if self.root.is_dir() else [] + if len(sources) > 24: + raise ValueError("repository experience corpus exceeds its read bound") + records = [] + for source in sources: + if not source.name.endswith(".json") or not source.is_file(): + continue + content = source.read_bytes() + if len(content) > 16384: + raise ValueError("repository experience exceeds its read bound") + experience = normalize_procedural_experience(json.loads(content)) + digest = "sha256:" + hashlib.sha256(content).hexdigest() + records.append((source, content, experience, digest)) + scores = score_bm25((json.dumps(row[2]) for row in records), request["query"]) + ranked = sorted(zip(records, scores.documents, strict=True), key=lambda row: -row[1].score) + items = [] + for (source, content, experience, digest), hit in ranked[:request["max_results"]]: + if hit.score <= 0: + continue + # Exact readback guards an asset changing during this invocation. + if source.read_bytes() != content: + raise ValueError("repository experience changed before readback") + candidate = build_reward_memory_candidate({ + "target_class": "procedural_experience", + "content_summary": experience["future_behavior"]["trigger"], + "experience": experience, + "source": {"source_kind": "reviewed_repository_experience", + "source_ref": digest, "actor_ref": self.corpus["owner_ref"], + "actor_role": "repository_source_owner"}, + "scope": self.corpus["scope"], + "reasoning": {"summary": "Git-published advisory experience; utility remains unproven.", + "confidence": "medium"}, + "guard_context": {"source_freshness": "current", "conflict_state": "clear", + "current_artifact_verified": True}, + "requested_action_scopes": [], "raw_content_captured": False, + }) + # Canonical source publication owns curation. This projects that source + # into the existing envelope, without activating or writing a provider. + reviewed = review_reward_memory_candidate(candidate, { + "decision": "accept", "reviewer_ref": self.corpus["owner_ref"], + "review_ref": digest, + "reasoning_summary": "Read-only projection of the repository-owned curated source.", + }) + active = build_active_reward_memory_record(reviewed, self.corpus, + activated_at=request["observed_at"]) + items.append(ContextProviderItem(resource_ref=f"repository:{source.name}:{digest}", + summary=active["content_summary"], content=json.dumps(active), score=hit.score)) + return ContextProviderRetrieval(provider=REPOSITORY_REVIEW_PROVIDER_ID, namespace=request["namespace"], + status="completed", query_summary=request["query_summary"], + observed_at=request["observed_at"], search_performed=True, read_performed=True, + items=tuple(items), requested_limit=request["max_results"]) + + def sync(self, **request: Any) -> Any: + raise ValueError("repository experience is read-only; publication uses repository review") + + +def attach_repository_review_experience( + packet: dict[str, Any], *, goal: Mapping[str, Any] | None, agent_id: str | None, +) -> None: + """Deliver scoped advisory context only to an already enabled review Agent.""" + if goal is None or not agent_id: + return + status, config = resolve_goal_reward_memory_experiment(goal=goal, agent_id=agent_id) + if config is None or status.get("automatic_recall") is not True: + return + # A configured provider or another module's recall route is insufficient. + if REPOSITORY_REVIEW_SURFACE_ID not in config["surfaces"]: + return + repository = str(packet.get("request", {}).get("repository") or "").casefold() + if not re.fullmatch(r"[a-z0-9_.-]+/[a-z0-9_.-]+", repository) or any( + part in {".", ".."} for part in repository.split("/") + ): + return + scope = {"workspace_ref": f"goal:{goal['id']}", "project_ref": f"repository:{repository}", + "peer_ref": f"agent:{agent_id}", "surface_ids": [REPOSITORY_REVIEW_SURFACE_ID]} + corpus = normalize_reward_memory_corpus({ + "corpus_id": REPOSITORY_REVIEW_CORPUS_ID, "class_id": "procedural_experience", "provider_id": REPOSITORY_REVIEW_PROVIDER_ID, + "owner_ref": f"repository:{repository}", "source_of_truth": f"repository:{repository}:experiences", + "read_authority": "actor_scoped", "write_authority": "read_only", "scope": scope, + "freshness": {"mode": "source_truth_bound"}, + "lifecycle": {"state": "active", "supersedes": []}, + "retrieval": {"index_required": False, "readback_required": True, + "application_receipt_required": True}, + "maintenance": {"writeback_triggers": [], "closure_policy": "repository_review", + "retirement_authority": f"repository:{repository}"}, + "privacy": {"visibility": "public_safe", "raw_content_in_registry": False}, + }) + reader = _RepositoryExperienceReader(repository, corpus) + if not reader.root.is_dir(): + return + # Native read-only source: no provider configuration is replaced, extended or + # written. The existing opt-in admits this caller-owned context boundary. + recall_config = { + "automation": {"automatic_recall": True, "automatic_ingest": False, "fail_open": True}, + "corpora": {REPOSITORY_REVIEW_CORPUS_ID: {"corpus": corpus, "standing_policy": {}, "provider_binding": { + "corpus_id": REPOSITORY_REVIEW_CORPUS_ID, "provider_id": REPOSITORY_REVIEW_PROVIDER_ID, "namespace": "reward_memory", + "scope_ref": f"repository:{repository}:experiences", "timeout_seconds": 5}}}, + "surfaces": {REPOSITORY_REVIEW_SURFACE_ID: {"corpus_ids": [REPOSITORY_REVIEW_CORPUS_ID], "ingest_corpus_id": REPOSITORY_REVIEW_CORPUS_ID, + "adapter": "scoped_feedback", "recall_profile": {"profile_id": "repository_review", + "mode": "function_boundary", "max_queries": 1, + "limit": min(3, config["surfaces"][REPOSITORY_REVIEW_SURFACE_ID]["recall_profile"]["limit"])}}}, + } + + def deliver(base: Any, items: tuple[Any, ...]) -> Mapping[str, Any]: + return {"outcome": "applied", "output": [ + {"candidate_ref": item.candidate_ref, "experience_digest": item.experience_digest, + "experience": dict(item.experience)} for item in items], + "memory_refs": [item.memory_ref for item in items], "current_artifact_verified": True, + "reasoning_summary": "Delivered advisory experience into this exact PR review packet; no verdict or utility inferred."} + + for item in packet.get("pull_requests", []): + # Do not change selection, exact-head idempotency or merge-readiness work. + if not item.get("review_action_kind") or item["review_action_kind"] == "qualify_pull_request_merge_readiness": + continue + head = str(item.get("head_oid") or "") + if not re.fullmatch(r"[0-9a-fA-F]{40}|[0-9a-fA-F]{64}", head): + continue + artifact = f"github:{repository}:pr:{item['number']}:{head}" + query = " ".join([str(item.get("title") or ""), *[ + str(entry.get("path") or "") for entry in item.get("key_files", [])]])[:500] + result = run_reward_memory_decision(recall_config, query_ready=bool(query), + application_kind="context_delivery", apply_memory=deliver, + surface_id=REPOSITORY_REVIEW_SURFACE_ID, base_output=[], workspace_ref=scope["workspace_ref"], + project_ref=scope["project_ref"], peer_ref=scope["peer_ref"], revision_ref=f"git:{head}", + queries=[{"query": query, "query_summary": "Current PR title and changed paths; no historical verdict."}], + observed_at=datetime.now(timezone.utc).isoformat(), + freshness_context={"source_truth_current": True, "source_revision": f"git:{head}", "age_seconds": 0}, + conflict_state="clear", application_id=f"review:{item['number']}:{head}", artifact_ref=artifact, + read_authority_checkpoints={REPOSITORY_REVIEW_CORPUS_ID: {"verified": True, "corpus_id": REPOSITORY_REVIEW_CORPUS_ID, + "workspace_ref": scope["workspace_ref"], "project_ref": scope["project_ref"], + "peer_ref": scope["peer_ref"], "surface_id": REPOSITORY_REVIEW_SURFACE_ID, "read_authority": "actor_scoped", + "source_ref": f"registry:{goal['id']}:reward-memory"}}, provider=reader) + if result is not None: + item["repository_experience"] = {"decision": result.public_packet, "guidance": result.output} + if any("repository_experience" in item for item in packet.get("pull_requests", [])): + packet["agent_response_contract"]["required_packet_fields_to_preserve"].append( + "pull_requests[review_action_kind!=null].repository_experience") diff --git a/loopx/capabilities/reward_memory/README.md b/loopx/capabilities/reward_memory/README.md index 47439e1bd7..ffc4c1231b 100644 --- a/loopx/capabilities/reward_memory/README.md +++ b/loopx/capabilities/reward_memory/README.md @@ -715,6 +715,15 @@ never infers an actor identity from an arbitrary target URI. ## Stage 4 evaluation and release gate +The PR-review caller also consumes [Git-owned review experiences](../pr_review_queue/experiences/README.md) +through a read-only native source adapter. Qualified Agent enablement, automatic +recall and an explicit `pull_request_review.review` surface are required; another +module's route does not enable it. Existing corpus/experience qualification and +the TypeScript decision owner bind context delivery to the current exact head. +This adds advisory repository context for that opted-in caller, with no provider +write, new setting, semantic-adoption claim or utility score. The public #5944 +case preserves differing review frames and unresolved outcomes separately. + Stage 4 is one bounded contract suite over the existing shared core. It does not add another evaluator, store, provider, scheduler, or semantic router: diff --git a/loopx/capabilities/reward_memory/README.zh-CN.md b/loopx/capabilities/reward_memory/README.zh-CN.md index ea3b28b10d..d157888410 100644 --- a/loopx/capabilities/reward_memory/README.zh-CN.md +++ b/loopx/capabilities/reward_memory/README.zh-CN.md @@ -512,6 +512,14 @@ actor 身份。 ## Stage 4 评估与发布门禁 +PR-review caller 还通过只读内置来源 adapter 消费 +[Git 持有的评审经验](../pr_review_queue/experiences/README.zh-CN.md)。 +须有资格化的 Agent enablement、自动 recall 和明确的 +`pull_request_review.review` surface；其他模块的 route 不会启用它。 +现有 corpus/experience 资格化与 TypeScript decision owner 把上下文投递绑定到 +当前 exact head。这为该 opt-in caller 增加仓库建议，不写 provider、不新增设置、 +不声明语义采用或 utility 分数；公共 #5944 案例分别保留不同口径和未解决的结果。 + Stage 4 只在现有共享核心之上增加一套受限 contract suite，不新增另一套 evaluator、 store、provider、scheduler 或 semantic router： diff --git a/loopx/cli_commands/pr_review.py b/loopx/cli_commands/pr_review.py index 5c3eb867c5..c4435ff713 100644 --- a/loopx/cli_commands/pr_review.py +++ b/loopx/cli_commands/pr_review.py @@ -608,6 +608,13 @@ def handle_pr_review_command( payload["request"]["readiness_observation_count"] = len( readiness_observations ) + from ..capabilities.pr_review_queue.repository_experience import attach_repository_review_experience + try: + attach_repository_review_experience(payload, goal=goal, agent_id=review_agent_id) + except (OSError, TypeError, ValueError): + # Memory availability must not invalidate the review queue or create + # a user gate. Never expose source/config exception text. + payload["repository_experience_warning"] = "repository_experience_unavailable" if args.autonomous_observation: autonomous_review = build_pull_request_review_queue_observation( repository=repository, diff --git a/loopx/pr_review.py b/loopx/pr_review.py index 27c16ed149..37e9dd6002 100644 --- a/loopx/pr_review.py +++ b/loopx/pr_review.py @@ -1438,6 +1438,11 @@ def append_group(group: dict[str, Any]) -> None: ] ) applicability = _as_dict(review_plan.get("applicability")) + memory = _as_dict(pr.get("repository_experience")) + if memory: + lines.extend(["", "### Repository experience (advisory)", "", + "Compare this experience with the current review frame; context delivery does not establish adoption or utility.", + "", "```json", json.dumps(memory, ensure_ascii=False, indent=2), "```", ""]) if review_plan: lines.append( "- review plan: " diff --git a/pyproject.toml b/pyproject.toml index 7427b6e474..97aed4f6a1 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -69,6 +69,7 @@ include = ["loopx*"] "loopx.control_plane.work_items" = ["*.ts"] "*" = ["README.md", "README.zh-CN.md", "docs/*.md", "docs/protocols/*.md"] "loopx.capabilities.content_ops" = ["templates/*.json", "experiences/*.json"] +"loopx.capabilities.pr_review_queue" = ["experiences/*/*/*.json", "experiences/*.md"] "loopx.capabilities.manager_context" = ["skills/loopx-manager/SKILL.md"] "loopx.extensions.lark" = ["extension.toml", "README.md", "docs/*.md"] "loopx.extensions.openviking_periodic_report" = ["extension.toml"] diff --git a/skills/loopx-pr-review/SKILL.md b/skills/loopx-pr-review/SKILL.md index b12c45f34e..7d75ce59b8 100644 --- a/skills/loopx-pr-review/SKILL.md +++ b/skills/loopx-pr-review/SKILL.md @@ -69,11 +69,11 @@ When `review_action_kind` is null, the row stays in `pull_requests` inventory bu 1. Record the packet's exact head, then follow `review_execution_contract.decision_procedure`: the current goal and `problem_context` delivery judgment first, including on re-review, then `evidence_commands` and repository-native validation. -2. Fill `review_plan.result_template`; preserve missing evidence as `unverified` and never - infer `verified` from metadata or CI. Execute its repository-reuse, default-off, authority - and real-path counterfactuals rather than repeating them as prose. Fill `result.reviewer` - per `review_execution_contract.reviewer_declaration`, open the body with its `body_marker` - line, and read `problem_context.spec_basis`'s specification before the diff. +2. Fill `review_plan.result_template`; missing evidence stays `unverified`, never `verified` from metadata or CI. + Execute repository-reuse, default-off, authority and real-path counterfactuals, not prose declarations. + Fill `result.reviewer` per `review_execution_contract.reviewer_declaration`; open with its `body_marker` line. + Read `problem_context.spec_basis`'s specification before the diff, plus any `repository_experience` advice. + In `problem_context`, state adoption, rejection or irrelevance with current-head evidence; do not inherit a case verdict or treat context delivery as semantic application or demonstrated utility. 3. Apply `completion_gate` literally: save final Markdown in `review_body`, then check evidence and that exact body. Follow capability-owned floors and scope counterfactuals; prose cannot replace missing execution: diff --git a/skills/loopx-self-repair/references/repair-patterns.md b/skills/loopx-self-repair/references/repair-patterns.md index 766f14f56a..0bd66e96a0 100644 --- a/skills/loopx-self-repair/references/repair-patterns.md +++ b/skills/loopx-self-repair/references/repair-patterns.md @@ -16,7 +16,7 @@ teaches a reusable control-plane lesson. | `acceptance_hold_recovery_selection_split` | Newly created advancement work is acceptance-unbound, repeated vision replans never expose its hold, or a replan packet also selects an unrelated due monitor. | Canonical acceptance tasks, scoped source Todos, bounded trigger checkpoints, effective action and original Turn receipt. | Recovery covered stale associations only; generic vision gaps displaced hold identities; candidate inventory leaked into the selected execution target. | Route missing and stale associations through the existing bounded replan lane, retain exact hold checkpoints before generic gaps, and separate replan from candidate selection. Keep owner association and completion validation enforced. A new unbound repair Todo is not runnable recovery; only a qualified successor or concrete blocker settles the exact hold. Validate real File/SQLite CLI paths and receipt reentry without mutating an active Goal. | | `acceptance_validation_failure_flattened` | A bound Todo is `ready` and its ordinary validator passes, yet completion reports only `goal_acceptance_validation_rejected`; the agent searches contract bindings before discovering a workspace or runner failure. | Exact Todo acceptance state, completion's typed criterion failure, recorded delivery workspace, current worktree cleanliness, and the privacy-safe runner receipt. | The completion boundary collapsed a failed criterion receipt into a generic contract error, hiding the workspace or command status. | Project the failed criterion ID, allowlisted validation status, safe exit code and bounded next action without command output or local paths. Repair the execution context and retry under the same Turn/lease; do not rebind owner criteria or infer a Goal-wide hold. | | `review_compatibility_assumption_gap` | A correct fix retains parallel protocol paths, and the review treats historical receipt recovery as proof that every old request decoder is needed. | Published review, structured compatibility rationale, real caller/deployment inventory, stored request versus receipt shape, and smaller-design readback. | Re-review verifies the last bug but does not separate compatibility obligations or test consolidation; free-text claims pass as evidence. | Replace the capability's existing compatibility rationale with a bounded structured assessment. Distinguish transient requests, independent client rollout and persisted replay formats; compare one typed current contract; preserve real legacy consumers. Cover needless retention and unsafe removal with positive twins. Keep optional simplifications advisory and do not make field completeness certify evidence truth. | -| `review_outcome_continuity_gap` | Local feature checks pass while later work starves, recovery only records a blocker, or ordinary users face new repeated intervention. | Accepted product outcome, published review, real later invocations, affected user surfaces, durable progress and authorized recovery. | Review proved one operation or prevention of bypass without checking sustained progress and the complete user journey. | Make long-horizon continuity and user experience explicit judgments in the existing problem context. Reuse bounded real walkthroughs; apply scope counterexamples when gates are involved. Accept deliberate waits only with an independent basis and recovery or safe terminal route. Qualify with exact-source historical counterexamples and repaired positive controls; keep old verdicts out of model inputs and distinguish reported historical checks from newly executed tests. Section length and declared evidence cannot certify truth. For frontend friction, compare the shortest authorized task path before encoding the UI sequence; inspect populated composition and real state readback. Record fixture/build/install boundaries and repair the existing delivery workflow, not a second checklist. | +| `review_outcome_continuity_gap` | Local feature checks pass while later work starves, recovery only records a blocker, or ordinary users face new repeated intervention. | Accepted product outcome, published review, real later invocations, affected user surfaces, durable progress and authorized recovery. | Review proved one operation or prevention of bypass without checking sustained progress and the complete user journey. | Make long-horizon continuity and user experience explicit judgments in the existing problem context. Reuse bounded real walkthroughs; apply scope counterexamples when gates are involved. Accept deliberate waits only with an independent basis and recovery or safe terminal route. Qualify with exact-source historical counterexamples and repaired positive controls; keep old verdicts out of model inputs and distinguish reported historical checks from newly executed tests. Section length and declared evidence cannot certify truth. For frontend friction, compare the shortest authorized task path before encoding the UI sequence; inspect populated composition and real state readback. Record fixture/build/install boundaries and repair the existing delivery workflow, not a second checklist. For architecture/RFC reviews, distinguish a docs-only diff from a design that commits future caller journeys. Reuse the [PR #5944 comparison and qualified experience](../../../loopx/capabilities/pr_review_queue/experiences/README.md); keep differing frames and unresolved outcomes, then test current-head adoption separately from context delivery and utility. | | `goal_activation_consumer_drift` | A canonically stopped Goal still accepts new peer requests, delegations, Chat execution, scheduled work, or another effect. | The registry Goal's `activation` object, any legacy top-level `status`, the active-admission predicate at every effect entrypoint, and state bytes or process calls before refusal. | The activation writer moved to the typed `activation.state` owner, but one or more consumers still read only the legacy top-level `status`. | Reuse `goal_activation_state()` or `goal_is_stopped()` in one domain predicate, retain only required legacy or archival compatibility, and route every effect entrypoint through that predicate. Run red-green tests for the canonical nested state and assert refusal occurs before writes, process launch, or external effects. Keep historical reads available after stop. | | `source_context_legacy_downgrade` | An unstamped record mutates state through a legacy path after the authoritative source disappears, although another record in the same owner context preserves an exact source identity. | Selected record identity, all same-context durable candidates, source path availability, operation entrypoint, and business-file bytes before and after refusal. | The adapter treated one selected record's missing identity as proof that the whole context was legacy and ignored exact historical evidence when the current authority could not be read. | Resolve source-context availability once for selection and mutation. Exact history may prove that legacy fallback is unsafe, but it must never supply or guess the current identity. Fail closed before writes when authority is unavailable; preserve contexts with no exact history through opposite legacy controls across every mutation entrypoint. | | `archive_capture_classification_gap` | Whole-Goal capture rejects a reachable archived Agent record although its active read was valid. | Recorded role/class, existing legacy read classification, transitive dependency closure, bootstrap and writer-outbox readback. | Archive storage preserved the role but omitted the resolved class; capture treated missing class as missing authority. | Keep recorded identity separate from compatibility classification. Only a recorded Agent role can adopt the existing read class; use it consistently for closure and materialization. Preserve the class on new archive moves, keep user authority fail-closed, and validate full source capture in disposable real providers without changing the active Goal. | diff --git a/tests/capabilities/test_pr_review_repository_experience.py b/tests/capabilities/test_pr_review_repository_experience.py new file mode 100644 index 0000000000..197b9e85b8 --- /dev/null +++ b/tests/capabilities/test_pr_review_repository_experience.py @@ -0,0 +1,166 @@ +"""Real repository-source readback and exact-head CLI context delivery. + +Enablement fixtures represent an already qualified configuration. Provider +enablement itself retains its separate production preflight coverage. +""" +from copy import deepcopy +import json +from pathlib import Path + +import pytest + +from loopx.cli import main +from loopx.pr_review import build_pr_review_packet, render_pr_review_markdown +from loopx.capabilities.pr_review_queue import repository_experience as source +from tests.capabilities.test_reward_memory_experiment import ( + _experiment, _v1_config, _write_v1_config, +) + +ROOT = Path(__file__).resolve().parents[2] +ASSET = ROOT / "loopx/capabilities/pr_review_queue/experiences/loopx-project/loopx/pr-5944-v1.json" +HEAD = "a" * 40 + + +def enabled_goal(tmp_path): + registry, _, _ = _experiment(tmp_path) + config = _v1_config() + review = deepcopy(config["surfaces"][0]) + review["surface_id"] = source.REPOSITORY_REVIEW_SURFACE_ID + config["surfaces"].append(review) + for entry in config["corpora"][:2]: + entry["corpus"]["scope"]["surface_ids"].append(source.REPOSITORY_REVIEW_SURFACE_ID) + entry["standing_policy"]["scope"]["surface_ids"].append(source.REPOSITORY_REVIEW_SURFACE_ID) + _write_v1_config(registry, config) + return registry, json.loads(registry.read_text())["goals"][0] + + +def rows(): + return [{"number": 8001, "headRefOid": HEAD, "state": "OPEN", + "title": "RFC: recovery design and phased user outcome", + "url": "https://github.com/loopx-project/loopx/pull/8001", + "author": {"login": "contributor"}, "body": "Review the first recovered Goal.", + "files": [{"path": "docs/architecture/rfcs/recovery.md", "additions": 8, "deletions": 0}], + "changedFiles": 1, "isDraft": False, "reviews": []}] + + +def packet(repository="loopx-project/loopx"): + return build_pr_review_packet(pull_requests=rows(), repository=repository, source="fixture", limit=10) + + +def test_real_file_recall_delivers_advice_not_a_verdict_or_utility(tmp_path): + _, goal = enabled_goal(tmp_path) + value = packet() + source.attach_repository_review_experience(value, goal=goal, agent_id="pilot") + memory = value["pull_requests"][0]["repository_experience"] + decision = memory["decision"] + assert decision["status"] == "context_delivered" + assert decision["context_delivery_verified"] is True + assert decision["artifact_ref"].endswith(f":8001:{HEAD}") + assert decision["semantic_disposition"] is None + assert decision["decision_consumption_complete"] is False + assert decision["utility_verified"] is False + assert decision["grants_new_action_authority"] is False + assert decision["external_writes_performed"] is False + experience = memory["guidance"][0]["experience"] + assert experience == json.loads(ASSET.read_text()) + assert "first useful user outcome" in experience["future_behavior"]["action"] + assert "APPROVE" not in json.dumps(memory) + assert "REQUEST_CHANGES" not in json.dumps(memory) + assert "Repository experience (advisory)" in render_pr_review_markdown(value) + + +@pytest.mark.parametrize("condition", ["no_goal", "no_agent", "disabled", "not_enabled", + "recall_off", "config_drift", "wrong_surface", "unverified", "different_repository", "inactive", "readiness"]) +def test_off_and_unqualified_paths_preserve_the_entire_packet(tmp_path, condition): + registry, goal = enabled_goal(tmp_path) + agent = "pilot" + value = packet("other/repository" if condition == "different_repository" else "loopx-project/loopx") + if condition == "no_goal": + goal = None + elif condition == "no_agent": + agent = None + elif condition == "disabled": + goal["control_plane"]["reward_memory"]["enabled"] = False + elif condition == "not_enabled": + agent = "meta" + elif condition in {"recall_off", "config_drift", "wrong_surface"}: + config = _v1_config() + config["automation"]["automatic_recall"] = condition == "wrong_surface" + _write_v1_config(registry, config) + if condition in {"recall_off", "wrong_surface"}: + goal = json.loads(registry.read_text())["goals"][0] + elif condition == "unverified": + goal["control_plane"]["reward_memory"]["enablement_receipts"] = {} + elif condition == "inactive": + value["pull_requests"][0]["review_action_kind"] = None + elif condition == "readiness": + value["pull_requests"][0]["review_action_kind"] = "qualify_pull_request_merge_readiness" + before = deepcopy(value) + source.attach_repository_review_experience(value, goal=goal, agent_id=agent) + assert value == before + + +def test_malformed_or_changed_source_fails_open_without_partial_context(tmp_path, monkeypatch): + _, goal = enabled_goal(tmp_path) + root = tmp_path / "assets/experiences/loopx-project/loopx" + root.mkdir(parents=True) + target = root / "case.json" + target.write_text(ASSET.read_text()) + (root / "invalid.json").write_text('{"schema_version":"unqualified"}') + monkeypatch.setattr(source, "files", lambda package: tmp_path / "assets") + value = packet() + source.attach_repository_review_experience(value, goal=goal, agent_id="pilot") + memory = value["pull_requests"][0]["repository_experience"] + assert memory["guidance"] == [] + assert memory["decision"]["context_delivery_verified"] is False + assert memory["decision"]["preserve_base_output"] is True + (root / "invalid.json").unlink() + original_read = Path.read_bytes + reads = 0 + def changing_read(path): + nonlocal reads + if path == target: + reads += 1 + if reads == 2: + return b"changed" + return original_read(path) + monkeypatch.setattr(Path, "read_bytes", changing_read) + value = packet() + source.attach_repository_review_experience(value, goal=goal, agent_id="pilot") + assert value["pull_requests"][0]["repository_experience"]["guidance"] == [] + + +def test_cli_json_and_markdown_readback(tmp_path, capsys): + registry, _ = enabled_goal(tmp_path) + fixture = tmp_path / "prs.json" + fixture.write_text(json.dumps({"pull_requests": rows()})) + common = ["--registry", str(registry), "pr-review", "--goal-id", "reward-memory-goal", + "--agent-id", "pilot", "--repo", "loopx-project/loopx", "--fixture", str(fixture)] + assert main([*common, "--format", "json"]) == 0 + value = json.loads(capsys.readouterr().out) + assert value["pull_requests"][0]["repository_experience"]["decision"]["context_delivery_verified"] is True + assert main([*common, "--format", "markdown"]) == 0 + assert "first useful user outcome" in capsys.readouterr().out + + +@pytest.mark.parametrize("path, expected", [ + ("docs/architecture/rfcs/recovery.md", "context_delivered"), + ("src/opaque_xyz.py", "empty"), +]) +def test_cli_uses_normalized_changed_paths_with_an_ordinary_title(tmp_path, capsys, path, expected): + registry, _ = enabled_goal(tmp_path) + raw = rows() + raw[0]["title"] = "Adjust wording" + raw[0]["body"] = "" + raw[0]["files"][0]["path"] = path + fixture = tmp_path / "prs.json" + fixture.write_text(json.dumps({"pull_requests": raw})) + assert main(["--registry", str(registry), "pr-review", "--goal-id", "reward-memory-goal", + "--agent-id", "pilot", "--repo", "loopx-project/loopx", "--fixture", str(fixture), + "--format", "json"]) == 0 + item = json.loads(capsys.readouterr().out)["pull_requests"][0] + assert "files" not in item + assert item["key_files"][0]["path"] == path + memory = item["repository_experience"] + assert memory["decision"]["status"] == expected + assert bool(memory["guidance"]) is (expected == "context_delivered")