Skip to content

[feat] provenance-led install prompt (install + check instructions)#73

Closed
mariojgt wants to merge 3 commits into
mainfrom
feat/provenance-led-install-prompt
Closed

[feat] provenance-led install prompt (install + check instructions)#73
mariojgt wants to merge 3 commits into
mainfrom
feat/provenance-led-install-prompt

Conversation

@mariojgt

Copy link
Copy Markdown
Contributor

Summary

Switches the install prompt to the provenance-led wording that avoided refusals on a real hosted-platform run (Horizon), replacing the verify-first prompt. Applied byte-identical across the three canonical copies:

  • field-test/prompt.txt, README.md (install-prompt block), GETTING-STARTED.md (step 1)

New prompt:

Please install @patchstack/connect and please check the instructions.

npm: https://www.npmjs.com/package/@patchstack/connect
Repo: https://github.com/patchstack/connect
Publisher: Patchstack
Purpose: Scan project dependencies for known vulnerabilities in CI.
Version: latest

Also reconciled the immediately-adjacent prose that would otherwise contradict the new prompt: the "every clause is load-bearing" framing (it's now provenance-led, not clause-tuned), the claim-URL "when the agent finishes" promise, and step 2's "verification step" reference.

The tradeoff (deliberate, chosen)

  • Win: leading with provenance metadata avoids the refusal, matching our own success-report finding.
  • Cost: it does not command completion (scanguide → build hooks + widget), so setup can end half-finished (the agent reads the instructions rather than carrying them out). The scan "what's left" nudge in [feat] surface remaining setup steps after scan + consistent guide labels #72 is the intended backstop. GETTING-STARTED step 2 is reworded to steer users through the half-finish.

⚠️ Doc debt this creates (not addressed here — needs a follow-up)

This prompt discards the adversarial design that other docs still describe:

  • MAINTAINING.md (provenance-anchor rules) and field-test/README.md (the 9 refusal modes) document the old verify-first prompt — they're now partly inaccurate for the shipped prompt.
  • The field-test/ personas assume the old flow (npm view first step, guide delegation), so a run will score this prompt red on the completion checks (hooksWired, widgetInstalled, claimUrlSurfaced) — expected, given the prompt no longer drives those.

Gating

🔴 load-bearing prompt change — needs a field-test re-gate before release. Given the completion-check reds above, judge a run by refusal reason (does it install without refusing?) rather than the raw scorecard.

🤖 Generated with Claude Code

Replace the verify-first prompt with the provenance-led 'install + check the instructions' wording that avoided refusals on a real hosted-platform run. Applied byte-identical across field-test/prompt.txt, README.md, and GETTING-STARTED.md, plus minimal reconciliation of adjacent prose (the 'every clause' framing, the claim-URL promise, and step 2's verification-step reference).

Tradeoff: provenance up front avoids refusals but the prompt no longer commands scan/guide/build-hooks, so setup can end half-finished (the scan next-steps nudge in #72 is the backstop). Doc debt: MAINTAINING.md and field-test/README.md still describe the old verify-first design and need a follow-up pass; field-test completion checks will score this prompt red as expected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@coderbuds

coderbuds Bot commented Jul 14, 2026

Copy link
Copy Markdown

Documentation-driven prompt clarifies provenance and update instructions across guides.

🎯 Quality: 100% Elite · 📦 Size: Small

📈 This month: Your 89th PR — above team average · Averaging Excellent

See how your team is trending →

mariojgt and others added 2 commits July 14, 2026 16:51
…e-per-step flow

A real Lovable run of the provenance-led prompt declined up front (flagging the shipped README as untrusted + supply-chain shape), then completed every step — install, scan, build hooks, widget — one at a time as the user authorized each. Reword step 2 to set that expectation and tell the user a plain 'yes' per step is enough.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@mariojgt

Copy link
Copy Markdown
Contributor Author

/review

@ejntaylor

Copy link
Copy Markdown
Contributor

Thanks for testing this. The provenance-led prompt gave us one useful Lovable success, but the follow-up Bolt tests showed it was not reliable as the universal flow: one run installed only and then invented generic CI token/workflow instructions, while three fresh chats refused, and the prompt intentionally left setup incomplete. PR #74 supersedes this with one explicit install && setup transaction, and commit 747549f preserves the provenance experiment and its results in the field-test history. Closing in favor of #74.

@ejntaylor ejntaylor closed this Jul 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants