Epic: account for every narrowing decision — exclusion ledger, conservation invariant, and skip-requires-evidence

[pengfei-threemoonslab]

The pattern

Across two first-time adoption walks — google/adk-samples#1745 and github/github-mcp-server#3076 — every failure had the same shape. Shipgate computed the correct signal, stored it, and did not connect it to the decision:

Site	It knew	It did
trigger	the diff contains an MCP tool schema (__toolsnaps__/delete_repository.snap)	should_run: false, skip_reason: "no_match"
binding	unbound_tools: 1	pass_eligible: true, gap_count: 0
scope resolution (#394)	all 17 changed files share one directory prefix	routed setup to the repository root
project discovery (#395)	the walk stopped at the 1000-file cap	reported 53 projects as a complete list
reason text (#396)	12/12 reachable, 0 unbound, 0 warnings	"the tool surface could not be fully enumerated"

These are not five unrelated bugs. They are one architectural habit: a stage narrows the analysed surface, and the narrowing is neither recorded as evidence nor reachable by the release decision. Each stage answers "is this in scope?" independently, and a negative answer removes the subject from every downstream stage — including from the evidence-gap accounting that exists to catch exactly this.

Evidence A — a destructive tool addition, silently excluded

github-mcp-server#3076 adds delete_repository (annotations: {destructiveHint: true, idempotentHint: false, readOnlyHint: false}) to GitHub's official MCP server. Reproduced with the 117 checked-in pkg/github/__toolsnaps__/*.snap schemas merged into one type: mcp source, manifest reviewed at base declaring the 116-tool surface complete: true, head at 117.

One variable — whether the new tool appears in the declaration:

	A: declaration lists 116	B: declaration lists 117
binding_coverage	unbound_tools: 1, pass_eligible: **true**, gap_count: **0**	unbound_tools: 0
capability_review	added=0 modified=0 removed=0	—
delete_repository mentions in report.json	1 (only tool_catalog[102].name)	41
blockers naming it	0	2
verdict	blocked — for 14 pre-existing findings on the other tools	blocked — 16

Run B's blockers are exactly right:

SHIP-POLICY-APPROVAL-MISSING              delete_repository lacks a declared approval policy
SHIP-ACTION-DESTRUCTIVE-ROLLBACK-MISSING  delete_repository has destructive capability without required controls

The checks work. The tool is dropped from the analysed surface before they run, and the drop is counted (unbound_tools: 1) but not gated (gap_count: 0).

This is a fail-open in the reward-hacking shape the product exists to prevent, and it has a pincer: shipgate.yaml is a trust root, so updating the declaration returns "This PR edits a release trust root — a coding agent cannot self-approve that change", while not updating it leaves the new destructive capability invisible. For an MCP server, "unbound" is not a meaningful safety state at all — a published tool is callable by any client.

Evidence B — the gate never ran

Same PR, unmodified repository, full clone, worktree at the PR head:

"diff_status": {"completeness": "complete"},
"trigger": {"should_run": false, "skip": true, "skip_reason": "no_match",
            "matched_rules": [], "diff_tokens": [],
            "rationale": "No rules matched; nothing in this PR signals a tool-surface change."}

TRIGGER-MCP-EXPORT-CHANGED matches on filename globs only — **/*mcp*.json and .agents-shipgate/*.json. The file is pkg/github/__toolsnaps__/delete_repository.snap: wrong extension, no mcp in the basename. Its content is unmistakably an MCP tool definition (inputSchema plus MCP annotation hints), but recognition is by naming convention. The same glob would also miss mcp-server/tools.json.

Proposed work

1. Exclusion ledger. Every stage that removes a subject from the analysed surface emits one typed record — {stage, subject, reason, source_ref}. Trigger no-match, discovery cap truncation, unresolved scope, unbound tool, unparsed region, source-type completeness denial are all the same event class, currently handled five different ways (or not at all). One representation, carried on the report, reachable by release_decision.

• Define the record type and attach it to the report schema
• Emit from: trigger, discovery walk, scope resolution, binding, adapter parse, _surface_is_complete
• Surface the ledger in the human-facing reason text and in next_action

2. Conservation invariant. Enforce, in code and as a property test across every adapter and every fixture:

observed == analysed ∪ excluded
excluded ≠ ∅  ⟹  gap_count > 0

binding_coverage already computes both sides and reports unbound_tools: 1 alongside gap_count: 0. The invariant makes that state unrepresentable rather than relying on each call site to remember.

• Property test over samples/ and benchmark/repos/ asserting conservation
• Negative controls — mutate a fixture to drop a tool and assert the ledger catches it
• Wire excluded ≠ ∅ into the release decision so exclusions cannot silently pass

3. skip requires positive evidence. A diff containing files the trigger cannot classify is not evidence of irrelevance. TRIGGER-DOCS-ONLY-NEGATIVE is a legitimate skip; no_match is not. Route unclassified content to unknown and let the scan decide.

This is the #308 monotonicity rule — already accepted for diff readability — applied to diff comprehension. Content-shaped recognition (JSON carrying inputSchema plus MCP annotation hints is a tool surface, whatever it is named) follows naturally once the trigger has to justify a skip.

• no_match on unclassified changed files becomes unknown, not skip
• Record the unclassified files in the exclusion ledger
• Recognise MCP tool schemas by content, not filename

Why this first

It is largely additive and it depends on nothing else being correct. Landing it does not require fixing extraction confidence (#393), scope resolution (#394), the discovery cap (#395) or the reason text (#396) — it makes each of those visible rather than silent. A gate that says "I saw 32 changed files and could not classify any of them" is honest and useful; one that says "nothing in this PR signals a tool-surface change" is not.

Instances and related: #393, #394, #395, #396, #397, #398.

---

Evidence gathered 2026-08-20/21 during two first-time-user adoption walks at CLI 0.16.0b7, contract 24.