docs(service-discoverability): gate the manifest on the ICP MCP App Operator Terms

[aterga]

What changed and why

The upstream guide (docs/guides/frontends/service-discoverability.md @ 217e9c2) gained a caution block that changes what Layer 1 is: publishing the manifest at /.well-known/ic-architecture opts the app into ICP MCP and, on its own, constitutes the operator's acceptance of the ICP MCP App Operator Terms — an agreement between DFINITY Stiftung (the "DFINITY Foundation") and the app's operator.

The skill previously treated that file as pure deploy config. An agent following it would generate the manifest in a presync hook and ship it without a word to the user — entering the app builder into a contract they never saw. This PR makes the skill stop and put the terms to the user first.

Facts were taken from the terms themselves and the ICP MCP Privacy Policy, not only the docs summary, so the skill carries the operative details: who may accept, the exact off switch, the registration payload.

On the terms version. The skill names the version the operator is accepting — currently version 1.1, effective 2026-09-02 — because that is what they are agreeing to and section 3 makes them record it when registering. It is paired with a staleness guard rather than left as a bare pin: the gate section tells the agent to check the top of the terms page first, and that a later version shown there means the page is right and the skill is stale — say so to the user, and get the skill fixed. The registration instruction resolves the same way, so the value that reaches mcp@dfinity.org always comes from the live page. (An earlier revision of this PR pinned v1.0 with no guard and went stale within a day; that is the failure this design prevents.)

skills/service-discoverability/SKILL.md

• New ## Before Layer 1 section, placed ahead of ## Prerequisites so an agent hits it before any manifest work:
  • publishing a valid manifest is the acceptance — no signature, checkbox, or registration step, and nothing else is required;
  • the version being accepted is named to the user, checked against the live page first;
  • the two things to confirm with the user first: (1) they are the operator, or authorized to bind it — section 1 is explicit that a developer who does not operate the app cannot accept for it; (2) they accept what participation permits under section 2 (metadata reads on declared canisters, indexing and public listing, query and state-changing calls signed with users' per-app II identities, file fetches to stay current);
  • the manifest as both switch and scope: only declared canisters are eligible, an absent manifest means metadata reads and calls are refused, removing it is the off switch (registered or not) though it does not stop public-website fetching, and keeping it published is continuing acceptance of updated versions;
  • deactivating is not terminating — removing the manifest ends participation, ending the agreement is a separate notice under section 11, neither undoes calls already executed, and sections 8, 12, 13 and 15 survive termination;
  • registration is optional and is not the opt-in — what it is for (notices under §§10/11/14, a record of who accepted and which version) and the exact payload to email mcp@dfinity.org;
  • standing obligations under §§5–8: interface accuracy, no assistant manipulation, origin/canister security, and independent-controller responsibility for personal data.
• Layer 1 now opens with a Gated callout pointing back at that section.
• Layer 3 carries the §6 constraints on getApiDoc content: never present a state-changing method as read-only or conceal a fee, and never steer the calling assistant. Prompt-injection-shaped API docs are a breach, not a feature.
• Deployment checklist gains terms-accepted and optional-registration items, ahead of the technical ones.
• Four new pitfalls (12–15): publishing without asking the operator; mistaking registration for the opt-in or the opt-out (also covering termination); expecting manifest removal to delist the app entirely; declaring canisters the operator does not control.
• Description mentions ICP MCP and the operator-consent gate, so the skill triggers on "make my app available through ICP MCP".

The manifest schema's own "version": "1.0.0" at lines 82 and 139 is unrelated to the terms version and untouched.

evaluations/service-discoverability.json

Five new output evals (one per new agent failure mode) and two new trigger queries.

Eval results

Each added or changed case was run with baseline. Trigger evals were re-run because the description changed, and two existing cases covering sections this PR touched were re-run as regression checks.

Case	With skill	Baseline
10. Publishing the manifest accepts the ICP MCP App Operator Terms	7/7	1/7
11. Adversarial: registration is neither the opt-in nor the opt-out	4/4	1/4
12. Adversarial: declaring canisters the operator does not control	4/4	0/4
13. Exiting ICP MCP: deactivation, delisting and termination differ	5/5	0/5
14. The named terms version is a snapshot; the live page decides	4/4	0/4
Trigger evals	11/11 should-trigger, 8/8 should-not-trigger	—
1. Generate ic-architecture at deploy time (regression)	5/5	skipped
5. Discoverable getApiDoc (regression)	4/4	skipped

What the baseline actually did, which is the argument for the change:

• Case 10 — the unguided agent denies any such spec or terms exists, never mentions ICP MCP or DFINITY, gives no URL, says nothing about operator authority, and asks the user to supply a source.
• Case 11 — declines to answer whether the app is participating, makes no claim about registration versus publishing, and never addresses the "do I unregister to stop?" question.
• Case 12 — answers "Yes — list them" and recommends putting the ICP ledger and Internet Identity canisters in the manifest, tagged as external dependencies. Serving that manifest is a repeated §5 representation that every declared canister is operated by the operator or under their authority, so the unguided answer walks the operator into a breach. This is the sharpest result in the set.
• Case 13 — the unguided agent treats deleting the manifest as the whole exit.
• Case 14 — no notion that a version named in a summary needs checking against the source.

Three eval-scoping notes, all caught by running the cases rather than reading them. Case 11 was overloaded: a sixth assertion crowded two previously-passing ones out of its "just the key points" budget, so the exits moved into case 13. Case 13's termination expectation initially failed with the skill — correctly, because that content had landed only in a pitfall while an agent answering "how do I stop?" reads the gate section; the fix was moving the content, not relaxing the assertion. Case 14 exists for the same reason: the staleness discipline needs a prompt that invites it, and it deliberately asserts the behavior without hardcoding a version number so the case cannot itself go stale.

Checks

• npm run validate — 30 skills validated, all passed; 21 warnings, unchanged from main.
• npm run build — 34 pages built.

Notes for reviewers

• service-discoverability is not listed in .claude/upstream.md, and dfinity/developer-docs is not one of the three repos sync-upstream.yml watches, so this change was applied by hand from the linked commit. A fourth watcher job is being opened as a follow-up; the version staleness caught in review is exactly what that gap produces.
• CONTRIBUTING asks for eval output collapsed in a details block. The tooling used to write this description strips raw HTML tags, so the results are condensed above instead of collapsed.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Tand5PzYhCagTujSUfDRgF

[github-actions]

Skill Validation Report

Validating skill: /home/runner/work/icskills/icskills/skills/service-discoverability

Structure

• Pass: SKILL.md found

Frontmatter

• Pass: name: "service-discoverability" (valid)
• Pass: description: (1007 chars)
• Pass: license: "Apache-2.0"
• Pass: compatibility: (88 chars)
• Pass: metadata: (2 entries)

Tokens

• Warning: SKILL.md body is 7773 tokens (spec recommends < 5000)

Markdown

• Pass: no unclosed code fences found

Tokens

File	Tokens
SKILL.md body	7,773
Total	7,773

Content Analysis

Metric	Value
Word count	4,891
Code block ratio	0.10
Imperative ratio	0.11
Information density	0.10
Instruction specificity	0.66
Sections	18
List items	79
Code blocks	12

Contamination Analysis

Metric	Value
Contamination level	low
Contamination score	0.20
Primary language category	config
Scope breadth	4

• Warning: Language mismatch: shell, systems (2 categories differ from primary)

Result: 1 warning

Project Checks

✓ Project checks passed for 1 skills (0 warnings)

[Copilot]

🟡 Changes recommended

There is an inconsistency in the eval text (“DFINITY Foundation”) vs the skill text (“DFINITY Stiftung”) that should be aligned to avoid confusion and brittle evaluation expectations.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR updates the service-discoverability skill to explicitly gate publishing /.well-known/ic-architecture on the operator’s informed acceptance of the ICP MCP App Operator Terms, and adds evaluation coverage for common failure modes around that gate.

Changes:

• Adds a new “Before Layer 1” gating section that explains the legal/operational implications of publishing the manifest and requires confirming operator authority and consent first.
• Adds ICP MCP-specific constraints and checklist/pitfalls updates to prevent accidental opt-in, incorrect registration assumptions, and over-declaring canisters.
• Extends evaluations/service-discoverability.json with new output evals and trigger queries covering the new ICP MCP terms/consent behaviors.

File summaries

File	Description
skills/service-discoverability/SKILL.md	Introduces the operator-consent gate and ICP MCP terms guidance around Layer 1 manifest publishing, plus checklist/pitfalls updates.
evaluations/service-discoverability.json	Adds output eval cases and new trigger queries to validate the new gating behaviors and adversarial scenarios.

Review details

• Files reviewed: 2/2 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[marc0olo]

Reviewed by verifying the claims against the live terms rather than the PR summary, since this skill tells agents that publishing a file forms a contract.

The substance holds up well. Spot-checking against the current text: the §1 quote is verbatim, §3's "sufficient on its own" and "nothing further is required" are exact, §4's manifest-is-the-switch (registered or not), §5's repeated representation, §7's delisting path via mcp@dfinity.org, §8's independent-controller framing, §2's "free in both directions". All nine cited section numbers are correct. DFINITY Stiftung ("DFINITY Foundation") matches the terms exactly. The upstream commit 217e9c2 exists and contains the caution block. npm run validate reproduces at 30 passed / 21 warnings, identical to main; eval JSON parses at 12 output evals and 11/8 triggers.

Case 12 is the sharpest result in the set — the unguided answer recommending the ledger and II canisters would put the operator in breach of §5. Good catch to build a case around.

Requested change (blocking)

The terms are pinned as v1.0, effective 2026-09-01. The live page now reads:

Effective date: September 2, 2026 · Version 1.1

The timeline is sympathetic — this PR opened 2026-09-02T22:01Z and v1.1 carries that same effective date, so v1.0 was almost certainly accurate when written.

It matters because §3 requires the registration email to carry "the version of these Terms being accepted, as shown at the top of this page," and the skill instructs the operator to send exactly that. As written, an agent walks the operator into registering acceptance of a superseded version — in the one skill whose purpose is preventing unwitting-contract errors.

Rather than bumping 1.0 → 1.1, I'd suggest dropping the pin entirely. The terms name their own page as the source of truth for the version, so a number copied into the skill is a snapshot competing with that pointer — and it lost within a day. There's no watcher on this source, so a stale pin also reads as verified-current when it isn't.

SKILL.md line 31:

-...[ICP MCP App Operator Terms](https://internetcomputer.org/icp-mcp/app-operator-terms/)
-(v1.0, effective 2026-09-01), an agreement between DFINITY Stiftung...
+...[ICP MCP App Operator Terms](https://internetcomputer.org/icp-mcp/app-operator-terms/),
+an agreement between DFINITY Stiftung...

SKILL.md line 57, so the registration instruction names where the value comes from:

-...and the version of the terms accepted. Keep it current as any of those change.
+...and the version of the terms being accepted — read from the top of the terms page
+at that moment, which is where section 3 says it is shown. Keep it current as any of
+those change.

Plus the same claim in the PR description.

Careful with the fix: lines 82 and 139 contain "version": "1.0.0" for the manifest schema, unrelated to the terms — a find-and-replace on "1.0" would corrupt the examples.

The evals don't hardcode the version, so they need no changes, and case 10's "points to the terms rather than only summarizing" expectation is arguably strengthened.

Suggestion (non-blocking)

§11 is worth one clause, because it answers the question this skill's own disclosure provokes. The skill covers the technical exit (remove the manifest) but not the contractual one — §11 makes them separate acts, and the non-obvious tail is that sections 8, 12, 13 and 15 survive termination and it "does not undo actions already executed on the Internet Computer." An unguided agent will say "delete the file and you're done," which is wrong rather than merely incomplete. Folding into pitfall 13 seems better than a new section:

Removing the manifest deactivates participation; terminating the agreement is a separate notice to mcp@dfinity.org. Neither undoes calls already executed, and the data-protection, liability, and governing-law sections survive termination.

Not asking for §9 (Participation Rules) — those bind what the app is and how the operator behaves, with no decision point an agent touches, and the one agent-shaped sliver ("misrepresents its operator") is already covered by the §6 accuracy content here. Adding it would push the skill toward a legal summary.

Follow-up beyond this PR

service-discoverability has no .claude/upstream.md entry and dfinity/developer-docs isn't watched by sync-upstream.yml, which the PR notes as out of scope — fair. Worth flagging that the version staleness above is exactly what that gap produces, one day later. Happy to open the issue for a fourth watcher job.

[aterga]

Thanks — both applied in dedf86b, and the blocking one was right.

Version pin (blocking) — dropped, not bumped

Confirmed independently before editing:

Effective date: September 2, 2026 - Version 1.1

Your reasoning for dropping rather than bumping is the part I'd have got wrong on my own. I'd have written v1.1 and reintroduced the same defect on a slower fuse. The terms name their own page as where the version is shown, §3 makes the operator copy that value into the registration email, and nothing in this repo watches that page — so any number in the skill is a snapshot competing with a live pointer, and it loses.

SKILL.md:31 now reads through to the terms with no pin. SKILL.md:57 goes slightly further than your diff:

…and the version of the terms being accepted — read from the top of the terms page at that moment, which is where section 3 says it is shown. Never copy a version number from here or from any other summary: the page is the only current source, and it is versioned independently of this skill.

The added sentence is aimed at the failure mode that produced the bug: an agent finding a version number in prose and treating it as current. Left the manifest-schema "version": "1.0.0" at lines 82 and 139 alone — thanks for the flag, that is exactly the shape of thing a careless fix would have eaten.

One thing your report prompted that goes past it. v1.1 is a new revision of the document this skill quotes from in nine places, so a pin fix alone would have been treating the symptom. I re-verified the whole surface against v1.1: all 16 section headings are unchanged, and the §1, §3, §4 and §5 quotes the skill leans on are still verbatim. Only the version line moved — but that was worth establishing rather than assuming, and it is the check that goes missing without a watcher.

§11 (non-blocking) — taken, and it exposed a real gap

Applied, with a wording tweak: §11 lists sections 8, 12, 13 and 15 as surviving, so the clause names warranties alongside data protection, liability and governing law.

More interestingly, the eval I added for it failed with the skill on the first run. The content had gone into pitfall 13, but an agent answering "how do I get out?" reads the gate section and answers from the off-switch bullet — never reaching the pitfalls. So the fix was moving the content to where it is actually read, not relaxing the assertion. The disclosure and its exit now sit together, which is your original rationale for raising §11 at all.

Agreed on not adding §9. Nothing there is a decision point an agent reaches, and the one agent-shaped sliver is already carried by the §6 content.

Evals

Contrary to your read, they did need changes — though not for the version, which they never hardcoded.

• Case 11 was overloaded. Adding a sixth assertion crowded two previously-passing ones out of its "just the key points" answer budget. Split into case 11 (participation status, registration is not the opt-in) and new case 13 (the three different exits). Also rescoped one assertion that demanded the word "optional" from a prompt where the user has already registered — the substantive half, "registration is not the opt-in", is what that prompt invites, and optionality stays covered by case 10.
• Case 13 baseline: 0/5. The unguided agent treats deleting the manifest as the whole exit, which is the "wrong rather than merely incomplete" answer you predicted.

Case	With skill	Baseline
10. Terms gate (regression after the line 31 edit)	6/6	0/6
11. Registration is neither opt-in nor opt-out	4/4	1/4
13. Exiting: deactivation vs delisting vs termination	5/5	0/5

npm run validate still 30 passed / 21 warnings, unchanged from main. PR description updated — it carried the same stale claim.

Follow-up

Yes please, go ahead and open the watcher issue. Your framing is the right one: this was not a one-off slip but the predictable output of an unwatched source, visible one day later.

---

Generated by Claude Code

[marc0olo]

Verified dedf86b: pin dropped, and line 57 naming the failure mode is better than my diff. §11 moved to where an agent actually reads it — the 0/5 baseline on case 13 earns its place. npm run validate reproduces at 30 passed / 21 warnings, unchanged from main.

Approving. Version-as-a-concept is covered in five places; only the number is gone, which is the point.

[marc0olo]

Re-checked on 9fed202 (my earlier approval was on dedf86b). Live page still reads Version 1.1, effective September 2 2026, so the named version is accurate.

The guard is what makes naming it safe: hedged as "current when this skill was written", page wins on disagreement, and the page value is what goes in the registration email — so §3 stays satisfiable. Evals encode that discipline with zero hardcoded version strings, so the cases cannot rot with the number. validate unchanged at 30 passed / 21 warnings.

Still approved.