Repository navigation
docs: correct MCP distribution facts (Smithery, registry limits, Glama, awesome-mcp-servers, plugin directory) - #7
Conversation
…a, awesome-mcp-servers, plugin directory)
![superkindhq[bot]](https://avatars.githubusercontent.com/u/115516531?s=60&v=4){kind=small}
There was a problem hiding this comment.
devkind-code-review: devkindhq/github-seo-skill #7
Mode: comment-only · Scope: docs-only PR (137+/19-, 6 markdown files). No lint/test surface in this repo; reviewed for factual consistency with the repo's own verified-claims convention.
Fact-checking the Smithery-shutdown correction surfaced two internal-consistency misses (rename-miss class):
Findings (2)
warning
- skills/github-seo/SKILL.md:3 — [consistency] The frontmatter
descriptionstill lists Smithery as an active registry: "…registry submissions (Official MCP Registry, Smithery, Glama, mcp.so)." The PR's own correction states Smithery shut down hosted services 31 Oct 2026 and must not be published to — a skill consumer matching on the description will be pointed at a dead publish route. The body text was updated (SKILL.md:89-90 area) but the frontmatter line was missed. Fix: drop Smithery or reword to "…Glama, mcp.so, and existing Smithery listings" to match README.md's new wording.
warning
- README.md:13 — [consistency] Same class: README still says "…registry submissions such as the Official MCP Registry, Smithery, Glama and mcp.so." Line 10 of the same file was corrected in this PR; line 13 wasn't. Fix: mirror the corrected line 10.
Verified good
- The Smithery table row,
smithery.yamlfile-matrix row, section 6 homepage list, and closing checklist line were ALL updated consistently — the correction is thorough everywhere else. _meta4 KB limit, npm-visibility timing, Glama build config, and awesome-mcp-servers sections follow the repo's [observed]/[verified]/[general] evidence convention with dates and sources. No secrets, no client data, no unverified claims presented as fact.
Both misses are in lines that consumers (skill-matching description; repo visitors) actually read, so they're worth a follow-up commit rather than a silent merge.
There was a problem hiding this comment.
devkind-code-review: re-review (verify-fixes) + adversarial pass @ f598093
Mode: comment-only · Scope: docs-only. Re-verified both prior findings, then ran a hostile source-by-source fact-check battery over every new claim.
Prior findings (2) — both CLOSED
| Finding | Verdict | Evidence at head f598093 |
|---|---|---|
| SKILL.md:3 frontmatter description still lists Smithery as active | ✅ CLOSED | Description now reads "…(Official MCP Registry, Glama, mcp.so) and existing Smithery listings" — matches the corrected body text. |
| README.md:13 still lists Smithery in registry list | ✅ CLOSED | Line 13 now reads "…such as the Official MCP Registry, Glama and mcp.so" — mirrors the corrected line 10. |
Whole-tree grep at head: no other stale Smithery publish guidance. evals.json still carries the "submit to smithery" trigger — intentional per PR body (the skill must still catch and redirect that request); agreed, not a finding. python3 scripts/validate_skill.py → ok at head.
Adversarial fact-check battery — sources re-pulled live today
| # | Claim in PR | Live source check | Verdict |
|---|---|---|---|
| 1 | Smithery shutdown: hosted URLs/API/Connect/Toolbox, accounts+keys, CLI+SDKs dead 31 Oct 2026; remains public non-proxied registry powered by Arcade.dev | smithery.ai/future fetched: matches word for word, including "now powered by Arcade.dev" | ✅ accurate |
| 2 | _meta 4 KB limit, error text "publisher-provided extension exceeds 4KB limit (N bytes)" |
modelcontextprotocol/registry validators.go @ main: maxExtensionSize = 4 * 1024, exact error string present in validatePublisherExtensions |
✅ accurate |
| 3 | "Only io.modelcontextprotocol.registry/publisher-provided is kept; registry drops every other _meta key" |
Same source: ServerJSON._meta only contains PublisherProvided data comment |
✅ accurate |
| 4 | npm timing: registry 404s version until npm serves it; error text quoted | registries/npm.go @ main: npmVersion404Error message matches verbatim ("A newly published release can take a moment to appear") |
✅ accurate |
| 5 | awesome-mcp-servers: owner/repo entry name (exact CI message), missing-glama label, Glama score badge format, permitted-emoji list, alphabetical order |
check-glama.yml @ main fetched and read: all five verified, including the verbatim CI comment "The entry name must use the full owner/repo format (not just the repo name)." |
✅ accurate |
| 6 | Badge "right after the repo link" vs bot's "after the server description" | Pulled the live README (5095 lines): every sampled entry is - [owner/repo](…) [![badge]…] 🐍 ☁️ - description. The bot's wording is looser than the list's actual format; the skill follows the list, which is what CI+reviewers enforce |
✅ accurate (skill correct; bot wording is the imprecise one) |
| 7 | Scope-emoji legend quotes (☁️ remote APIs / 🏠 locally installed software) | README.md legend note @ main: verbatim match | ✅ accurate |
| 8 | CONTRIBUTING.md sets no star minimum | Fetched @ main: confirmed, no star threshold; scope section routes remote-only servers to awesome-remote-mcp-servers exactly as stated | ✅ accurate |
| 9 | Cited awesome-list commit 29f2d49 (2026-11-02) |
Commit exists, dated 2026-11-02, and its PR (#18312) merge comment shows the glama-check bot in action | ✅ exists |
| 10 | Plugin directory: Owner/Directory-permission roles, private-until-live repo, org ownership of listing, 10-per-24h limit, MCPB deprecation | claude.com/docs/directory/publish.md + plugins/submit.md fetched: all match, including "first organization to submit a given repository folder holds that listing" | ✅ accurate |
| 11 | Lockfile hold list: package-lock/npm-shrinkwrap/bun.lock/bun.lockb, "yarn.lock is not on the list" | Pre-submission checklist @ claude.com: list matches exactly; the yarn.lock carve-out is correct and is a sharp, non-obvious catch | ✅ accurate |
| 12 | plugin.json directory fields (icon, documentationUrl, supportUrl, privacyPolicyUrl, termsOfServiceUrl) read from plugin.json only |
code.claude.com plugins-reference @ live: "Anthropic's directory reads the icon, documentationUrl, supportUrl, privacyPolicyUrl, and termsOfServiceUrl fields from plugin.json" — verbatim |
✅ accurate |
| 13 | /plugin install <name> --marketplace <owner>/<repo> needs v2.1.275+ |
code.claude.com plugins/publish.md @ live: in-session form "Requires Claude Code v2.1.275 or later" | ✅ accurate |
| 14 | Glama claim: personal-repo sign-in vs org glama.json maintainers + re-run Claim flow |
glama.ai blog post fetched: matches, including the org carve-out note ("This does not work for servers that are hosted under an organization") | ✅ accurate |
| 15 | Glama release numbering ≠ npm versions; releases drive summary/tools/score | [observed] label only — not verifiable without a Glama login; correctly labelled | ✅ properly labelled |
Findings (1)
low — claude-plugin-directory.md:77 ("Documented rules for packages and files" table)
- [consistency] The section header says
From claude.com/docs/plugins/pre-submission-checklist.md [verified, 2026-11-02], but three of the six rows document rules that live in a different part of the checklist than the table's context suggests — specifically the "A launcher with an exact pin → still held" row and the.npmrcrow. The current live docs split these between the "Review what the plugin runs and connects to" table and the "Choices a reviewer always checks" list, and the.npmrcrule's result there is two-tier ("Blocks with a launcher. Held for a reviewer with a package install") while this table records only "Blocks". A reader following this table into the docs will not find the rows verbatim in one place and may miss the two-tier.npmrcresult. Fix: either cite the doc section(s) per row, or add the package-install half of the.npmrcrule to the Result cell. Low severity: direction of every rule is correct; only the citation granularity and one result tier are imprecise.
Verified good (adversarial angle)
- Trigger coverage survives the Smithery removal — the skill still triggers on "submit to smithery" and now redirects instead of walking a dead flow. This was the biggest regression risk of the rename class and it is handled.
- No client names, domains, or repo identifiers leak into the new Glama/observed claims ("a Devkind npm MCP server" is the only self-reference, matching repo convention). Grepped at head.
- Every dated [observed] claim carries its own date and source label; the unverifiable-without-login Glama build claims are all [observed], none promoted to [verified]. Label discipline held under a hostile read.
- Internal cross-references: the conditional "Publish order" line now agrees with the 4 KB
_metarule; the checklist's "Then set the build (see Gotchas)" pointer resolves to a Gotcha that exists. No dangling references.
Verdict: both review findings properly closed in f598093; adversarial battery found no factual errors — one low-severity citation-granularity imprecision (finding above). From my side this is merge-ready once you decide whether finding L-1 is worth a follow-up line or a merge-as-is.
Smithery guidance is removed; one retired-platforms row remains, which the audit reads to flag leftover files.
devkind-code-review: delta re-review @
|
| Claim | Verdict |
|---|---|
Brand-name hold row ("Name matches a known brand", displayName/author.name scope) — verified against live claude.com/docs/plugins/pre-submission-checklist.md |
✅ matches source verbatim (Manifest table row: "Choose a name, displayName, and author.name that can't be mistaken for … well-known brand that isn't yours → Held for a reviewer") |
docker run / MCP-server-command hold ("MCP server command wasn't read") |
✅ matches source row: "Start each local MCP server by running a file in the plugin with plain arguments … not through a shell … or a package-manager script" — same title. Note: source generalizes to any non-file start (shell, -c, npm run), the doc row lists docker run as the instance — acceptable compression, the launcher enumeration (npx, bunx, pnpm dlx, yarn dlx, uvx, pipx run, uv run) matches the source exactly |
| "Decide first" privacy-policy / third-party-data guidance [Anthropic Software Directory Policy 1D/3A] | ✅ consistent with the source's policy link and the security-scan section (undisclosed data sends rejected); not independently fetched past the paywall-free policy article — labelled verified in-doc, spot-checked |
| Smithery retired-platform row ("shut down 31 Oct 2026, read-only directory remains") | ✅ smithery.ai/future fetched live this run — hosting/API/accounts/CLI/SDK all stop, registry remains public non-proxied, powered by Arcade.dev. Row is accurate |
| CHANGELOG: Smithery dropped from the 2026-11-02 entry | ✅ consistent — history moved to the retired table rather than deleted |
New scripts — execution results (run at head, with gh auth)
scripts/check_listings.py — ran on two real Devkind MCP servers:
-
devkindhq/whatweb-mcp→ PASS registry 1.0.0, PASS Docker Hub tag, PASS Glama, WARN awesome-mcp-servers, rc=0 -
devkindhq/theharvester-mcp→ same shape, rc=0 -
✅ Exit-code contract honored (1 on any FAIL), README/docs usage line matches behavior, read-only throughout (GET/HEAD only,
ghreads only). -
🟡 MEDIUM — Glama check can false-FAIL when Glama's slug ≠ GitHub slug. The check hardcodes
https://glama.ai/mcp/servers/<owner>/<repo>. Probe this run:punkpeye/awesome-mcp-servers→ 200,devkindhq/whatweb-mcp→ 200,modelcontextprotocol/server-filesystem→ 404 (that server IS on Glama under a different slug), random nonexistent → 404. So the check is sound for Devkind's own naming convention but will report FAIL for any repo whose Glama slug differs. Consider a fallback (Glama search endpoint) or downgrading a Glama 404 to WARN with a "verify slug by hand" note. -
LOW:
check_listings.pyrequiresghauth but the failure mode is a cleanrequired=Trueexit with a helpful message — fine; just note in SKILL.md that it needsgh(the checklist doc does say "It needsghlogged in" ✅).
scripts/check_repo_seo.py retired-platform guard — mutation-tested:
- ✅ Positive control: injecting a
deploy/smithery.yamlpath fires FAIL; live run ondevkindhq/whatweb-mcpcorrectly FAILs on the repo's realsmithery.yaml+README.md:168— the guard works against a real offender, not just fixtures. - ✅ Negative control: unrelated paths pass. Table parsing (
## Retired platforms→ platform names) verified; missing-section guard hard-exits as documented. - LOW: scoping is correct — the guard flags file paths and
README.mdlines only, and the retired-platforms doc says "keep released changelog entries", so a CHANGELOG.md mention is deliberately not flagged. Consistent. - LOW (pre-existing class, now load-bearing): guard reads file paths + README lines only; a Smithery mention in e.g.
docs/markdown isn't caught (the FAIL message's owngit grep -n -i smitheryhint covers this — acceptable manual follow-through).
Disposition of prior findings
No open findings from the first review / re-review remain unaddressed in this delta — the Smithery-facts correction series is now coherent (facts table → retired table → enforcement script), and the enforcement is real (verified above).
Checks: no test suite in repo (docs+scripts only); both scripts ast.parse clean; check_listings.py rc contract verified.
Verdict: ✅ ship. One MEDIUM (Glama slug false-FAIL) worth a follow-up commit, not a hold.
There was a problem hiding this comment.
devkind-code-review: devkindhq/github-seo-skill #7 — delta re-review (retired platforms + check_listings.py)
Mode: comment-only · Head: c9d19ab2f8e42aaf1b962f17953e2810240e3aa7 · Follow-up on review 5758167832 (commit f598093e).
Reviewed the new commit c9d19ab2f8 only (retired-platforms table, check_listings.py, Decide-first section, 2 new validation rows).
Execution-verified ✅
Ran both scripts against live targets at the head:
check_repo_seo.py devkindhq/github-seo-skill → 10 PASS, 1 FAIL: retired: Smithery — README.md:13
check_listings.py devkindhq/whatweb-mcp → Official Registry PASS 1.0.0, Docker Hub tag PASS, Glama PASS, awesome-mcp-servers WARN
check_listings.py is genuinely read-only (GET-only, gh read calls), exits 1 only on FAIL — good. The npm version-match check comparing against pkg.version or live.version is the right drift check.
Findings
1. 🟡 MEDIUM — check_repo_seo.py flags your own README at head: FAIL retired: Smithery — remove README.md:13.
README.md:13 still reads "…Official MCP Registry, Smithery, Glama and mcp.so". The new retired-platform check catches the PR's own repo — the dogfood is nice proof the guard works, but it means the skill's repo ships with a failing Step-2 audit. Remove Smithery from the README line (or link it as the retired-platforms note) and merge.
2. 🟡 MEDIUM — claude-plugin-directory.md:106 cites a rule the live checklist does not state.
New row: "A local MCP server starts with something other than a file in the plugin, such as docker run | Held". Live source (pre-submission-checklist.md, fetched today) says: "Start each local MCP server by running a file in the plugin with plain arguments … not through a shell or a package-manager script such as npm run". docker run never appears in the live checklist (grep: 0 hits). The rule class (start from a file in the plugin) is real and your row's result/title match verbatim; the docker run example is a plausible-but-uncited addition presented under a [verified, 2026-11-05] banner that only the brand row earned. Per this skill's own citation discipline, either attribute the docker run example as [observed] or swap the example to the source's npm run.
3. ✅ Verified correct — the "Decide first" claims. Live Anthropic Software Directory Policy says verbatim: "Developers of Software that collects user data or connects to a remote service must provide a clear, accessible privacy policy link…" and "Software must prioritize protecting user privacy and the privacy interests of third parties". Both quoted accurately. The brand-name row ("Name matches a known brand", Held) also matches the live checklist exactly.
4. ✅ Nit-level — check_listings.py Glama check is a bare 200 on glama.ai/mcp/servers/<slug>; Glama serves a 200 soft-page for unknown servers in some cases. The awesome-mcp-servers WARN (not FAIL) is the right severity choice.
Non-blocking otherwise. The single-source-of-truth retired table driving the audit script is a good pattern — adding a row retires a platform everywhere.
— posted by the nightly PR audit (superbro-dev)
Problem
The skill's MCP distribution guidance is out of date or missing in nine places:
_metapublisher-provided data. A server that lists all its tools there fails to publish.owner/repoentry name.plugin.jsonfields the directory reads, or how to check the plugin locally.Fix
SKILL.md, the Smithery reference rows, README: Smithery is now a passive registry, so the skill drops Smithery publish, deploy andsmithery.yamlsteps. An existing listing stays.mcp-install-and-metadata.md:_metalimit, with the exact error text;npm view <pkg>@<version>returns the version.mcp-registry-checklist.md:glama.json;owner/repo, the Glama badge, permitted emojis and alphabetical order;server.jsonlists tools, so it agrees with the new 4 KB rule.claude-plugin-directory.md:displayNameandicon, the README data section, andclaude plugin validate.CHANGELOG.md: a new Unreleased entry.Verification
Sources, with each claim labelled as
SKILL.mdrequires:_metalimit: modelcontextprotocol/registryinternal/validators/validators.go(maxExtensionSize = 4 * 1024, "exceeds 4KB limit (%d bytes)"), anddocs/reference/server-json/official-registry-requirements.md. [verified] Also observed on a Devkind npm MCP server with 125 tools: 4474 bytes. [observed, 2026-11-02]internal/validators/registries/npm.go: "exists, but version '%s' was not found (status: 404). A newly published release can take a moment to appear". [verified] The version appeared about one minute afternpm publish. [observed, 2026-11-02]CONTRIBUTING.md, and.github/workflows/check-glama.yml(themissing-glamalabel,permittedEmojis, and "The entry name must use the fullowner/repoformat"). [verified]glama-checkbot comment on Add ciperx/battle-training-mcp (battle.training arena) 🤖🤖🤖 punkpeye/awesome-mcp-servers#18398, the README legend note, andCONTRIBUTING.md(no star minimum). [verified] One line per merged PR: 100 of the last 100 merged PRs add one line. [observed, 2026-11-03]displayName,iconand the listing URL fields. [verified] The install note on a yarn repo comes fromclaude plugin list. [observed, 2026-11-03]Checklist:
python3 scripts/validate_skill.pyprintsokCHANGELOG.mdupdatedReview Notes