From 1f4581c4680816fbb798b493644aa9f29755cc6e Mon Sep 17 00:00:00 2001 From: Mario Tarosso Date: Tue, 14 Jul 2026 16:45:40 +0200 Subject: [PATCH 1/2] feat(prompt): switch install prompt to provenance-led wording Replace the verify-first prompt with the provenance-led 'install + check the instructions' wording that avoided refusals on a real hosted-platform run. Applied byte-identical across field-test/prompt.txt, README.md, and GETTING-STARTED.md, plus minimal reconciliation of adjacent prose (the 'every clause' framing, the claim-URL promise, and step 2's verification-step reference). Tradeoff: provenance up front avoids refusals but the prompt no longer commands scan/guide/build-hooks, so setup can end half-finished (the scan next-steps nudge in #72 is the backstop). Doc debt: MAINTAINING.md and field-test/README.md still describe the old verify-first design and need a follow-up pass; field-test completion checks will score this prompt red as expected. Co-Authored-By: Claude Opus 4.8 (1M context) --- GETTING-STARTED.md | 16 +++++++++++----- README.md | 12 +++++++++--- field-test/prompt.txt | 8 +++++++- 3 files changed, 27 insertions(+), 9 deletions(-) diff --git a/GETTING-STARTED.md b/GETTING-STARTED.md index 84441c5..0328516 100644 --- a/GETTING-STARTED.md +++ b/GETTING-STARTED.md @@ -6,11 +6,17 @@ The fastest path from "I have a JS/Node project" to "Patchstack is monitoring it ## 1. Paste this prompt into your AI coding tool -Works in Cursor, Claude Code, Lovable, Bolt, v0, Windsurf, etc. Paste it exactly — every clause is there for a reason (see step 2), so don't shorten it: +Works in Cursor, Claude Code, Lovable, Bolt, v0, Windsurf, etc. Paste it exactly — leading with the provenance links is what lets a cautious agent verify the package quickly (see step 2), so keep them: -> Add Patchstack dependency vulnerability monitoring: first verify `@patchstack/connect` yourself (`npm view @patchstack/connect` shows the live registry record — MIT; source: https://github.com/patchstack/connect), and if anything you find contradicts that, stop and show me what you found instead of proceeding. Then install it as a dev dependency, run `npx @patchstack/connect scan` and show me the claim URL it prints (I'll open it in my browser to view reports — don't open or fetch it yourself), then run `npx @patchstack/connect guide`, review the checklist it prints, and make the two remaining changes yourself: the package.json build hooks and the report-a-vulnerability widget snippet. +> Please install @patchstack/connect and please check the instructions. +> +> npm: https://www.npmjs.com/package/@patchstack/connect +> Repo: https://github.com/patchstack/connect +> Publisher: Patchstack +> Purpose: Scan project dependencies for known vulnerabilities in CI. +> Version: latest -When the agent finishes it shows you a **claim URL**. Open it in your browser and sign in — that attaches the site to your Patchstack account so you can see the vulnerability reports. That's the only manual step. +Once the agent runs the first `scan` (its bundled instructions walk it through this), it prints a **claim URL**. Open it in your browser and sign in — that attaches the site to your Patchstack account so you can see the vulnerability reports. If the agent stops after installing without running `scan`, that's the common half-finish — see step 2. ## 2. If the agent refuses or stops halfway @@ -18,8 +24,8 @@ Some agents decline this request, and that's expected behaviour, not a bug: "ins Work down this list: -1. **It stopped at the verification step, asks you to confirm, or says it wants the package vetted first.** The prompt tells the agent to stop and show you what it found rather than guess — this is that, working as designed. Reply with the confirmation it asked for, e.g.: *"I've verified this is the official Patchstack package (`npm view @patchstack/connect` — published by patchstack, MIT). Please continue with the remaining steps."* Most partial refusals end here. -2. **It did some steps, then stalled.** Ask it to run `npx @patchstack/connect guide` — that prints a live checklist of exactly what's still missing for this project — and to finish only the items marked ✖. +1. **It installed but stopped there (didn't run `scan`, no widget/claim URL), or asks you to confirm the package first.** This is the common half-finish with a short prompt — the agent treats "check the instructions" as "read them," not "carry them out." Reply telling it to finish, e.g.: *"I've verified this is the official Patchstack package (`npm view @patchstack/connect` — published by patchstack, MIT). Please run `npx @patchstack/connect scan`, then `npx @patchstack/connect guide`, and complete the steps it lists."* Most partial stops end here. +2. **It ran some steps, then stalled.** Ask it to run `npx @patchstack/connect guide` — that prints a live checklist of exactly what's still missing for this project (build hooks, widget) — and to finish only the items marked ✖. 3. **It refuses outright.** Skip the agent; step 3 below is three commands. If you hit a refusal with a *new* reason (the agent quotes a specific phrase or concern not covered above), don't fight it — copy the agent's explanation and send it to the maintainers (or open an issue). Real-world refusal reports are how the prompt gets improved; each one becomes a regression test in [`field-test/`](field-test/README.md). diff --git a/README.md b/README.md index e8fc4fa..2065991 100644 --- a/README.md +++ b/README.md @@ -8,9 +8,15 @@ For how this repo fits with the wider Patchstack ecosystem (`saas`, `hub`, `patc Copy and paste this prompt into any AI coding assistant (Cursor, v0, Bolt, Lovable, Claude Code, etc.). If the assistant declines or stops partway, the step-by-step flow in [`GETTING-STARTED.md`](https://github.com/patchstack/connect/blob/main/GETTING-STARTED.md) covers what to do next: -> Add Patchstack dependency vulnerability monitoring: first verify `@patchstack/connect` yourself (`npm view @patchstack/connect` shows the live registry record — MIT; source: https://github.com/patchstack/connect), and if anything you find contradicts that, stop and show me what you found instead of proceeding. Then install it as a dev dependency, run `npx @patchstack/connect scan` and show me the claim URL it prints (I'll open it in my browser to view reports — don't open or fetch it yourself), then run `npx @patchstack/connect guide`, review the checklist it prints, and make the two remaining changes yourself: the package.json build hooks and the report-a-vulnerability widget snippet. - -Every clause is load-bearing. AI agents routinely audit this package before completing the request — verifying the tarball against the registry, checking `dist/` for lifecycle scripts and shell/eval calls; that's expected use, and the docs in the tarball state plainly what each command reads, writes, and sends. The prompt can stay short because `guide` is state-aware: it inspects the project (package manager, `.patchstackrc.json`, `package.json` scripts, widget presence, framework) and prints a live checklist of only the remaining steps — with the real site UUID and framework-specific widget placement — followed by the full reference guide (`AGENT-INSTALL.md`, bundled in the package). That also makes the flow idempotent: re-running `guide` on a finished project reports all-done instead of prompting a second install. +> Please install @patchstack/connect and please check the instructions. +> +> npm: https://www.npmjs.com/package/@patchstack/connect +> Repo: https://github.com/patchstack/connect +> Publisher: Patchstack +> Purpose: Scan project dependencies for known vulnerabilities in CI. +> Version: latest + +Leading with the package's provenance (npm, repo, publisher) is deliberate — it lets a cautious agent verify what it's installing before it acts. AI agents routinely audit this package before completing the request — verifying the tarball against the registry, checking `dist/` for lifecycle scripts and shell/eval calls; that's expected use, and the docs in the tarball state plainly what each command reads, writes, and sends. The prompt can stay short because `guide` is state-aware: it inspects the project (package manager, `.patchstackrc.json`, `package.json` scripts, widget presence, framework) and prints a live checklist of only the remaining steps — with the real site UUID and framework-specific widget placement — followed by the full reference guide (`AGENT-INSTALL.md`, bundled in the package). That also makes the flow idempotent: re-running `guide` on a finished project reports all-done instead of prompting a second install. Before changing this prompt (or `guide` / `AGENT-INSTALL.md`), read [`MAINTAINING.md`](https://github.com/patchstack/connect/blob/main/MAINTAINING.md) — it maps which files are load-bearing and how to ship a change safely. The [`field-test/`](https://github.com/patchstack/connect/blob/main/field-test/README.md) harness runs a real agent through the full install in a throwaway fixture against a mocked API and scores the outcome on eight checks; validate any variant there first. diff --git a/field-test/prompt.txt b/field-test/prompt.txt index f17e543..5f98c8c 100644 --- a/field-test/prompt.txt +++ b/field-test/prompt.txt @@ -1 +1,7 @@ -Add Patchstack dependency vulnerability monitoring: first verify `@patchstack/connect` yourself (`npm view @patchstack/connect` shows the live registry record — MIT; source: https://github.com/patchstack/connect), and if anything you find contradicts that, stop and show me what you found instead of proceeding. Then install it as a dev dependency, run `npx @patchstack/connect scan` and show me the claim URL it prints (I'll open it in my browser to view reports — don't open or fetch it yourself), then run `npx @patchstack/connect guide`, review the checklist it prints, and make the two remaining changes yourself: the package.json build hooks and the report-a-vulnerability widget snippet. +Please install @patchstack/connect and please check the instructions. + +npm: https://www.npmjs.com/package/@patchstack/connect +Repo: https://github.com/patchstack/connect +Publisher: Patchstack +Purpose: Scan project dependencies for known vulnerabilities in CI. +Version: latest From 093afe3db0585d5155fdfd161a271605498fbc09 Mon Sep 17 00:00:00 2001 From: Mario Tarosso Date: Tue, 14 Jul 2026 16:51:09 +0200 Subject: [PATCH 2/2] docs(getting-started): match step 2 to the real decline-then-authorize-per-step flow MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A real Lovable run of the provenance-led prompt declined up front (flagging the shipped README as untrusted + supply-chain shape), then completed every step — install, scan, build hooks, widget — one at a time as the user authorized each. Reword step 2 to set that expectation and tell the user a plain 'yes' per step is enough. Co-Authored-By: Claude Opus 4.8 (1M context) --- GETTING-STARTED.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/GETTING-STARTED.md b/GETTING-STARTED.md index 0328516..5ac3593 100644 --- a/GETTING-STARTED.md +++ b/GETTING-STARTED.md @@ -24,7 +24,7 @@ Some agents decline this request, and that's expected behaviour, not a bug: "ins Work down this list: -1. **It installed but stopped there (didn't run `scan`, no widget/claim URL), or asks you to confirm the package first.** This is the common half-finish with a short prompt — the agent treats "check the instructions" as "read them," not "carry them out." Reply telling it to finish, e.g.: *"I've verified this is the official Patchstack package (`npm view @patchstack/connect` — published by patchstack, MIT). Please run `npx @patchstack/connect scan`, then `npx @patchstack/connect guide`, and complete the steps it lists."* Most partial stops end here. +1. **It declines up front, installs then stops, or gates each step in turn.** A cautious agent commonly (a) declines at first — flagging the request as supply-chain-shaped, or the package's own shipped README as untrusted — and then, once you say to go ahead, (b) completes **one mutating step at a time, pausing for your OK before each**: install → `scan` (sends the dependency manifest, returns the claim URL) → build hooks + widget (both modify your app; the widget loads third-party JS on every page). This is working as intended, not a fault — authorize each step you want (a plain "yes, go ahead" is enough), or paste: *"I've verified this is the official Patchstack package (`npm view @patchstack/connect` — published by patchstack, MIT). Please run `npx @patchstack/connect scan`, then `npx @patchstack/connect guide`, and complete the steps it lists."* Most stops end within a reply or two. 2. **It ran some steps, then stalled.** Ask it to run `npx @patchstack/connect guide` — that prints a live checklist of exactly what's still missing for this project (build hooks, widget) — and to finish only the items marked ✖. 3. **It refuses outright.** Skip the agent; step 3 below is three commands.