DOC-384: Add docs-tester.yml CI workflow for AWS docs PRs - #902
Open
quetzalliwrites wants to merge 1 commit into
Open
DOC-384: Add docs-tester.yml CI workflow for AWS docs PRs#902quetzalliwrites wants to merge 1 commit into
quetzalliwrites wants to merge 1 commit into
Conversation
Runs the agents-tester.md reviewer agent on every pull request that touches src/content/docs/aws/**. The agent independently re-verifies the diff (coverage JSON, live AWS docs, sample links, sibling doc conventions) and runs the real build, then posts a severity-tagged Verification Report as a PR comment, ending with a VERDICT line the workflow parses to decide pass/fail. The agent is granted no Edit/Write tools at all, so "report, don't fix" is enforced by the tool permissions themselves rather than by prompt instruction alone: its final message text becomes the report, and the workflow (not the agent) posts it as a PR comment via `gh pr comment`. A FAIL or ESCALATE verdict fails the check, so this can later be wired up as a required status check per the parent ticket (DOC-368) and PR #813's noted follow-ups. Mirrors docs-ci-agent.yml's conventions (Claude Code CLI, MCP config for Linear + LocalStack, pinned action SHAs) but triggers on pull_request instead of workflow_dispatch, and reviews rather than implements.
quetzalliwrites
requested review from
HarshCasper and
remotesynth
as code owners
August 28, 2026 14:43
Deploying localstack-docs with
|
| Latest commit: |
32b94f3
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://6ccfe55b.localstack-docs.pages.dev |
| Branch Preview URL: | https://doc-384-docs-ci-workflow-doc.localstack-docs.pages.dev |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
docs-tester.yml, the CI counterpart toagents-tester.md(merged in #813). Runs on every pull request that touchessrc/content/docs/aws/**.agents-tester.md(andagents.mdfor context on what the writer was supposed to do), independently re-verifies the changes per that process — coverage JSON, live AWS docs, sample links, sibling doc conventions, tone — and runs the realnpm run build.Edit/Writetools at all. "Report, don't fix" is enforced at the tool-permission level, not just by prompt instruction: the agent's final message is the Verification Report, and the workflow (not the agent) posts it as a PR comment viagh pr comment.VERDICT: PASS|PASS_WITH_MINOR_FINDINGS|FAIL|ESCALATEline, which the workflow parses — aFAILorESCALATEverdict fails the check, so this can later be wired up as a required status check (the remaining item from DOC-368: Add agents-tester.md verification agent for AWS docs #813 alongside theANTHROPIC_API_KEY/OAuth token secret, which I assume is already set up asCLAUDE_CODE_OAUTH_TOKENsincedocs-ci-agent.ymldepends on it).Mirrors
docs-ci-agent.yml's conventions (Claude Code CLI install, MCP config for Linear + LocalStack, pinned action SHAs) but triggers onpull_requestinstead ofworkflow_dispatch, and reviews a diff instead of implementing a ticket.Linear ticket
https://linear.app/localstack/issue/DOC-384/docs-ci-workflow-docs-testeryml-that-runs-on-every-pr-for-aws-docs
Test plan
js-yamlparse)pull_request-triggered workflow before merge)