Pattern quality: Emit tools.github.min-integrity for PR-review archetypes - #254
Draft
github-actions[bot] wants to merge 1 commit into
Draft
Pattern quality: Emit tools.github.min-integrity for PR-review archetypes#254github-actions[bot] wants to merge 1 commit into
github-actions[bot] wants to merge 1 commit into
Conversation
The pr-review and skill-pr-reviewer archetypes' own tips already recommend setting tools.github.min-integrity so the agent only acts on content from trusted authors, but the generator only surfaced this as a prose bullet in the requirements list rather than baking it into the emitted workflow YAML. A downstream agent following the suggested draft was therefore likely to omit this guardrail entirely, leaving PR-triggered review workflows exposed to untrusted external-contributor content. Add a min_integrity field to these two archetype definitions in patterns/workflow-generation.json and have generateWorkflowFile emit tools.github.min-integrity in the generated frontmatter whenever it is set, so the safety-critical setting ships by default instead of relying on prose-only guidance. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was evaluated
Reviewed all 6 samples in
/tmp/gh-aw/data/generated-patterns-and-prompts.json(status-report, issue-triage, code-improvement, documentation-updater, dependency-monitor, pr-review) againstpatterns/manifest.json,patterns/workflow-generation.json,patterns/archetypes/*.json, and the generator logic insrc/js/workflow.js.Recurring quality issue found
For the
pr-review-readysample (and the closely relatedskill-pr-reviewerarchetype), the archetype's own curated tip already says:This tip is data the wizard already knows is safety-critical for
pull_request-triggered workflows, since PR content can come from untrusted external contributors. However, the generator only surfaced it as a prose "for example" bullet in the requirements list — it was never baked into the emittedtools.github:block of the suggested workflow YAML draft. A downstream agent implementing the draft is likely to treat prose tips as optional guidance and ship a workflow that reviews/reacts to unverified external PR content without the integrity gate, undermining the "Include DO NOT constraints to avoid false positive comments" goal stated elsewhere in the same prompt.What changed
patterns/workflow-generation.json: added"min_integrity": "approved"to thepr-reviewandskill-pr-reviewerarchetype definitions (both arepull_request/ready_for_review-triggered reviewers that read potentially untrusted diff/PR content).src/js/workflow.js:generateWorkflowFilenow emitstools.github.min-integrity: <value>directly in the generated frontmatter whenever an archetype definesmin_integrity, so the guardrail ships by default in the suggested draft instead of only appearing as prose.test/workflow.test.js: added assertions that pr-review workflows includemin-integrity: approvedundertools.github, and that unrelated archetypes (e.g.status-report) do not emitmin-integrityat all.Validation
npm test: 319/319 tests pass (18 test files).npm run build: production build succeeds,dist/patterns/archetypes/pr-review.jsonregenerated correctly.Follow-ups deliberately left out
min_integrityto other pull_request-related archetypes (e.g.pr-iteration-loop,accessibility-expert,user-simulator,community-digest,ci-failure-triage) since those don't primarily process untrusted external diff content the way PR reviewers do — a broader audit of which archetypes need this guardrail is worth a separate pass.patterns/from raw scan data, per task constraints.