Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@
"name": "agent-stack",
"displayName": "Agent Stack",
"source": "./plugins/agent-stack",
"description": "Two skills: agent-orchestrator — tool-calling loops, multi-stage pipelines with checkpoints, provider routing with fallback, four-layer memory, context engineering, plus the wallet side of reselling LLM access; and agent-evals — run/trace/thread evals, judges, and fixtures grown from production.",
"version": "0.16.0",
"description": "Four skills: agent-orchestrator — tool-calling loops, pipelines with checkpoints, provider routing with fallback, memory architecture, plus the wallet side of reselling LLM access; agent-evals — run/trace/thread evals, LLM judges, and fixtures grown from production; agent-interop — MCP servers and clients, A2A agent cards, the MCP Registry, and gateways; agent-harness — system prompts, tool shaping, workflow-vs-agent, and auditing an agent system.",
"version": "0.16.1",
"author": {
"name": "ssheleg",
"url": "https://x.com/sshlg93"
Expand Down
32 changes: 10 additions & 22 deletions .github/workflows/validate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -731,28 +731,16 @@ jobs:
- name: install.sh syntax check
run: bash -n install.sh

- name: npm installer functional test (fresh / rerun-skip / --force / bad arg)
run: |
set -eu
export HOME=/tmp/fakehome
mkdir -p "$HOME"
node bin/agent-stack.js
test -f "$HOME/.claude/skills/agent-orchestrator/SKILL.md"
test -f "$HOME/.claude/skills/agent-orchestrator/references/patterns.md"
test -f "$HOME/.claude/skills/agent-orchestrator/references/llm-proxy-billing.md"
# Every skill the plugin ships must arrive. The installer ENUMERATES the skills
# directory rather than naming skills, so this asserts the enumeration works --
# a regression there would install a subset and report success.
for s in $(ls plugins/agent-stack/skills); do
test -f "$HOME/.claude/skills/$s/SKILL.md" || { echo "ERROR: $s did not install"; exit 1; }
done
test -f "$HOME/.claude/skills/agent-interop/references/mcp.md"
node bin/agent-stack.js | grep -q '^skip:'
node bin/agent-stack.js --force | grep -q '^Installed'
if node bin/agent-stack.js --wat; then
echo "ERROR: unknown arg accepted"; exit 1
fi
echo "OK: installer behaves"
- name: npm installer functional test (fresh / rerun-skip / --force / plugin-present refusal)
# The cases live in test/installer_test.js — one home for the logic, so
# `npm test` locally and this job exercise the same assertions. The one
# this step exists for is PLUGIN-PRESENT: a fake HOME whose
# installed_plugins.json declares agent-stack, asserting the non-zero
# exit, the remedy in the output, and that nothing was written. Every
# member's CI tested only a fresh HOME until 2026-08-29, which is how
# nine installers shipped a shadow-writer nobody had ever run against a
# plugin (telegram-dev, reproduced live that day).
run: node test/installer_test.js

- name: Workflow YAML parses
run: |
Expand Down
35 changes: 35 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,40 @@
# Changelog

## v0.16.1 — the installers refuse the shadow, and the pack stops mis-selling itself

- **Both installers refuse to write plain copies over an installed plugin.** The family
audit of 2026-08-29 reproduced the shadow live: a bare `npx @ssheleg/telegram-dev`
created three plain copies in `~/.claude/skills/` while that plugin was enabled — and
this member had **no plugin check at all**, in either installer, so a bare
`npx @ssheleg/agent-stack` on a machine with the plugin would have shipped **four**
shadows, one per skill, each serving its frozen version forever. `bin/agent-stack.js`
and `install.sh` now implement make-skill v0.25.0's canon (`distribution.md`, "The
installer must refuse the shadow it documents"): detect from the TARGET home's
`installed_plugins.json` (keys are `<name>@<marketplace>`, and the two names differ
often), keep the `marketplaces/` dir read only as the fallback signal, refuse with
exit **3** and a remedy that names the spec read from the JSON
(`claude plugin marketplace update agent-stack` + `claude plugin update
agent-stack@<marketplace>`, plus the family launcher), offer `--force` as the recorded
deliberate override, fail open on a missing or corrupt JSON, and gate only the
`~/.claude` write — no other agent has plugins.
- **`test/installer_test.js` joins `npm test` and CI** — 11 cases against throwaway
HOMEs: fresh / rerun-skip / `--force` / unknown-arg, plugin-present refusal (exit
code, remedy text, nothing written — all three asserted), a differently-named
marketplace in the remedy spec, corrupt JSON failing open, no false refusal on other
plugins or an `agent-stack-extra` prefix-collider, the marketplaces-dir fallback, and
the same matrix for `install.sh`. Watched failing before trusted: **7 of 11 red**
against the pre-fix installers (`git stash` the two, run, pop). The suite follows the
house residue rule — a failing case keeps its HOME, and the run ends by saying what it
left. It replaces the inline fresh-HOME-only step in `validate.yml`, which is the CI
shape that let the plugin-present case go unrun everywhere.
- **A successful install now says how the next version arrives** — the last line names
`npx @ssheleg/agent-stack@latest --force` and the family launcher, in both installers.
- **AST-01: the plugin stops selling itself as two skills.** `plugin.json` and
`marketplace.json` both opened with "Two skills: agent-orchestrator … and agent-evals"
while the pack ships **four** — `agent-interop` and `agent-harness` were invisible in
`claude plugin details` and on the marketplace. Both descriptions now name all four
skills with what each covers.

## v0.16.0 — the whole survey, not just its taxonomy

v0.15.0 took the taxonomy and the named failure modes. This takes the rest: the write
Expand Down
88 changes: 88 additions & 0 deletions bin/agent-stack.js
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,44 @@ const os = require('os');

const ROOT = path.resolve(__dirname, '..');
const REPO = 'ssheleg/agent-stack';
const PLUGIN = 'agent-stack';

// Exit codes are the contract: 0 installed or skipped, 1 corrupted package,
// 2 usage error, 3 refused — the plugin channel owns this agent (--force overrides).
const EXIT_PLUGIN_PRESENT = 3;

/**
* The plugin spec (`<name>@<marketplace>`) installed for `name` in this home,
* or null.
*
* `installed_plugins.json` is the record of what is actually installed. The
* `plugins/marketplaces/<name>` directory under-reports: a marketplace added
* from a local `directory` source has no dir there at all, and plugin names
* differ from marketplace names, so a check keyed on it stays green while the
* shadow lands. Absence and corruption both read as "no plugin": the fresh
* HOME is the common case, and an installer that crashes on a parse error
* refuses the machines that need it most.
*/
function installedPluginSpec(home, name) {
try {
const raw = fs.readFileSync(
path.join(home, '.claude', 'plugins', 'installed_plugins.json'), 'utf8');
const parsed = JSON.parse(raw);
const plugins =
parsed && typeof parsed === 'object' &&
parsed.plugins && typeof parsed.plugins === 'object'
? parsed.plugins
: parsed;
if (!plugins || typeof plugins !== 'object') return null;
for (const spec of Object.keys(plugins)) {
if (spec === name) return `${name}@${name}`;
if (spec.startsWith(name + '@')) return spec;
}
} catch {
// missing or corrupt = no plugin — fail open on absence, never crash
}
return null;
}

function usage() {
console.log(`agent-stack installer
Expand All @@ -25,6 +63,12 @@ Usage:
into ~/.claude (skip existing unless --force)
npx @ssheleg/agent-stack --help

Exit codes:
0 installed or skipped 2 usage error
1 corrupted package 3 refused: the agent-stack PLUGIN is installed in
this home — plain copies would shadow it (pass
--force to write them anyway)

Other install paths:
Claude Code plugin: /plugin marketplace add ${REPO}
/plugin install agent-stack@agent-stack
Expand Down Expand Up @@ -86,6 +130,41 @@ function main(argv) {
}

const home = os.homedir();

// One channel per agent. A plain ~/.claude/skills/<name> beside the installed
// agent-stack plugin is two listings of the same skill per skill this pack
// ships, and the stale copy wins — the exact shadow make-skill's distribution
// canon forbids (§ "The installer must refuse the shadow it documents").
// Refuse rather than create it, and refuse LOUDLY: a presence check keyed on
// the marketplaces/ dir alone that exits 0 is the fail-open class — a
// directory-sourced marketplace has no dir there, plugin names differ from
// marketplace names, and exit 0 reads as success to every script above it.
// Reproduced live 2026-08-29: a bare `npx @ssheleg/telegram-dev` shipped
// three shadows past exactly this hole while the plugin was enabled. Only the
// ~/.claude write is gated — no other agent has plugins.
const spec = installedPluginSpec(home, PLUGIN);
const marketplace = path.join(home, '.claude', 'plugins', 'marketplaces', PLUGIN);
const viaMarketplaceDir = !spec && fs.existsSync(marketplace);
if ((spec || viaMarketplaceDir) && !force) {
const found = spec
? `installed as the Claude Code plugin ${spec}\n` +
' (declared in ~/.claude/plugins/installed_plugins.json)'
: `registered as a Claude Code marketplace\n (${marketplace})`;
console.error(
`refused: agent-stack is already ${found}.\n` +
` Plain copies in ~/.claude/skills/ (${names.join(', ')})\n` +
' would shadow the plugin and serve this frozen version forever.\n' +
' Update the plugin channel instead:\n' +
' claude plugin marketplace update agent-stack\n' +
` claude plugin update ${spec || 'agent-stack@agent-stack'}\n` +
' Family launcher (updates every member, prunes shadow copies):\n' +
' npx --yes sshlg-skills@latest update\n' +
' Pass --force to write the plain copies anyway — a deliberate choice\n' +
' to run two channels, where the stale one wins.'
);
return EXIT_PLUGIN_PRESENT;
}

for (const name of names) {
installOne(
`${name} skill`,
Expand All @@ -95,6 +174,15 @@ function main(argv) {
force
);
}
// The last line says how the next version arrives — "Installed" is not a
// complete sentence. Auto-update is off on purpose: this member composes
// with its family, and per-marketplace autoUpdate moves each member on its
// own clock, into combinations nobody tested together.
console.log(
'\nUpdates: rerun `npx @ssheleg/agent-stack@latest --force`, or refresh the\n' +
'whole family with `npx --yes sshlg-skills@latest update` (every channel,\n' +
'and it prunes plain copies that would shadow a plugin).'
);
return 0;
}

Expand Down
11 changes: 11 additions & 0 deletions docs/evidence/verification.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,17 @@ This file exists because its absence read as zero exposure. `sshlg-skills` board

---

## Shipped state — v0.16.1 (2026-08-29)

Measured on the release-candidate tree before the tag exists. The wave-1.5 rollout:
make-skill v0.25.0's installer-shadow canon ported into both installers, plus AST-01.

| REQ | What ships | How it was confirmed | Confirmed |
|---|---|---|---|
| R-01 | Both installers refuse to write plain copies over an installed plugin, reading `installed_plugins.json` from the target home, exit 3, remedy naming the spec from the JSON, `--force` as the recorded override, fail-open on absent/corrupt JSON | `node test/installer_test.js` → `PASS: installer — 11 case(s)` with `residue: this run left nothing — 11 temp home(s) created, 11 removed`; the plugin-present cases assert exit `3`, the `claude plugin update agent-stack@<marketplace>` line, and that no `~/.claude/skills/<skill>` was created for any of the four | **planted** + **observed** |
| R-02 | The suite fails against the pre-fix installers, so the guard has been watched failing | `git stash push bin/agent-stack.js install.sh` → `node test/installer_test.js` → `FAIL: installer — 7 case(s) red` (every plugin-present, marketplace-dir and update-line case) → `git stash pop` → green again | **planted** |
| R-03 | AST-01: `plugin.json` and `marketplace.json` name all four shipped skills instead of two | `grep -rn "Two skills" --include="*.json" .` → no matches; both descriptions open "Four skills:" and name `agent-orchestrator`, `agent-evals`, `agent-interop`, `agent-harness`; `ls plugins/agent-stack/skills` → those same four | **observed** |

## Run 2026-08-20 — the document that refused a score and computed one (AG-04 … AG-09, in tree at v0.12.0, release pending)

Rows: `docs/evidence/backlog.md` **AG-04 … AG-09**. Manifesto requirements **M-?** are not
Expand Down
43 changes: 43 additions & 0 deletions install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,52 @@ set -euo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SRC_ROOT="$ROOT/plugins/agent-stack/skills"

FORCE=0
if [[ "${1:-}" == "--force" ]]; then
FORCE=1
elif [[ -n "${1:-}" ]]; then
echo "usage: $0 [--force]" >&2
exit 2
fi

if [ ! -d "$SRC_ROOT" ]; then
echo "error: skill sources missing at $SRC_ROOT" >&2
exit 1
fi

# One channel per agent: plain copies beside an installed plugin are two
# listings of the same skill — one per skill this pack ships — and the stale
# one wins. Refuse rather than create that, and refuse loudly: a presence
# check keyed on the marketplaces/ dir alone that exits 0 is the fail-open
# class — a directory-sourced marketplace has no dir there, plugin names
# differ from marketplace names, and an exit 0 reads as success to every
# script above it. installed_plugins.json is the record of what is installed;
# a missing or unparsable one reads as "no plugin".
INSTALLED_JSON="${HOME}/.claude/plugins/installed_plugins.json"
MARKETPLACE="${HOME}/.claude/plugins/marketplaces/agent-stack"
SPEC=""
if [[ -f "$INSTALLED_JSON" ]]; then
SPEC="$(sed -n 's/.*"\(agent-stack@[^"]*\)".*/\1/p' "$INSTALLED_JSON" 2>/dev/null | head -n 1)" || true
fi
if [[ ( -n "$SPEC" || -e "$MARKETPLACE" ) && "$FORCE" -eq 0 ]]; then
{
if [[ -n "$SPEC" ]]; then
echo "refused: agent-stack is already installed as the Claude Code plugin $SPEC"
echo " (declared in ~/.claude/plugins/installed_plugins.json)."
else
echo "refused: agent-stack is already registered as a Claude Code marketplace"
echo " ($MARKETPLACE)."
fi
echo " Plain copies in ~/.claude/skills would shadow the plugin and serve"
echo " this frozen version forever. Update the plugin channel instead:"
echo " claude plugin marketplace update agent-stack"
echo " claude plugin update ${SPEC:-agent-stack@agent-stack}"
echo " Family launcher: npx --yes sshlg-skills@latest update"
echo " Pass --force to write the plain copies anyway."
} >&2
exit 3
fi

# Iterate rather than name one skill: a skill added to the plugin must not
# require an installer change to reach anybody.
for SRC in "$SRC_ROOT"/*/; do
Expand All @@ -22,3 +63,5 @@ for SRC in "$SRC_ROOT"/*/; do
echo "Installed ${NAME} skill -> $DEST"
done
echo "Restart your agent — skills load at session start."
# The last line says how the next version arrives.
echo "Updates: git pull && ./install.sh --force, or npx --yes sshlg-skills@latest update"
4 changes: 2 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
{
"name": "@ssheleg/agent-stack",
"version": "0.16.0",
"version": "0.16.1",
"scripts": {
"test": "python3 test/validate.py && python3 test/plant_guard_test.py"
"test": "python3 test/validate.py && python3 test/plant_guard_test.py && node test/installer_test.js"
},
"description": "Production patterns for AI agent orchestrators — tool-calling loops, multi-stage pipelines with checkpoints, LLM provider routing with fallback, four-layer memory with confidence decay — plus the wallet side of reselling LLM access. This package is the installer CLI.",
"bin": {
Expand Down
4 changes: 2 additions & 2 deletions plugins/agent-stack/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
{
"name": "agent-stack",
"displayName": "Agent Stack",
"description": "Two skills: agent-orchestrator — tool-calling loops, multi-stage pipelines with checkpoints, provider routing with fallback, four-layer memory, context engineering, plus the wallet side of reselling LLM access; and agent-evals — run/trace/thread evals, judges, and fixtures grown from production.",
"version": "0.16.0",
"description": "Four skills: agent-orchestrator — tool-calling loops, pipelines with checkpoints, provider routing with fallback, memory architecture, plus the wallet side of reselling LLM access; agent-evals — run/trace/thread evals, LLM judges, and fixtures grown from production; agent-interop — MCP servers and clients, A2A agent cards, the MCP Registry, and gateways; agent-harness — system prompts, tool shaping, workflow-vs-agent, and auditing an agent system.",
"version": "0.16.1",
"author": {
"name": "ssheleg",
"url": "https://x.com/sshlg93"
Expand Down
Loading
Loading