Skip to content

fix: repair docs consistency audit and clarify Slack reporting - #943

Merged
rachaelrenk merged 2 commits into
mainfrom
rrenk/docs-consistency-reporting
Oct 22, 2026
Merged

rachaelrenk merged 2 commits into
mainfrom
rrenk/docs-consistency-reporting

Conversation

@rachaelrenk

@rachaelrenk rachaelrenk commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Repair the docs consistency audit's redirect handling and make its Slack reports explain the outcome, coverage, and next action.

The October 21 audit stopped before extracting any pages. Its helper normalized /agents → /agents/ and /platform/environments → /platform/environments/ into self-loops. Reproducing the recorded input checked 718 routes and produced 30 false cycle errors before this fix and zero after it.

Changes

  • Ignore slash-only canonicalization edges while retaining detection of literal self-redirects and genuine multi-route cycles.
  • Lead notifications with completed, partial, or blocked outcomes rather than run UUIDs.
  • Explain blockers, known coverage, and whether saved findings were rechecked. Include a next action and descriptive links.
  • Summarize up to three priority changes with public page links, confidence, and review actions. Resolved findings do not request edits; policy findings require human review.
  • Add a blocked-report command for early failures that reads, but does not modify, prior state. Use controlled failure codes instead of raw exceptions.
  • Update the skill instructions, reporting eval prompts, and regression tests.

Slack message preview

Generated by the new formatter using the October 21 blocked-run context. This is a local preview, not a new audit result or a message posted to Slack. Slack's emphasis and link syntax are converted to GitHub Markdown for review.

Docs consistency audit — could not check docs
Checks public documentation for conflicting or stale claims. It does not edit docs.

What happened: The audit's redirect checker detected a loop and stopped comparison.
Stopped at: redirect normalization. 0/390 pages extracted.
What changed: No new results are available from this blocked audit.
2 previously active findings remain in saved history; this run did not verify their current status.
Next action: Inspect redirect normalization in the audit helper and the public redirect chain, correct the cause, then rerun the audit.
Saved history records no completed full audit.

Run details · Saved findings and audit history
The linked PR stores audit state; it contains no documentation fixes.

Companion automation update

The same reporting structure is applied to the schema-audit automation in docs-factory-config PR #26. That separate diff changes reporting prompts only, not the checker or schedules; it is not included in this docs PR's code changes.

Factory configuration docs check preview

Illustrative message using the October 22 run's findings and the revised prompt. This is not a new audit result or a message posted to Slack.

Factory configuration docs check — review needed
Checks factory configuration docs against the published schemas.

What happened: The checker found incomplete mappings between schema fields and doc sections, plus GitHub trigger options that need review. Some findings may be checker gaps rather than missing documentation.

What changed: No docs changed and no PR was opened. Release and runtime verification remains incomplete.

Next action: Repair the checker mappings, rerun the check, then review the remaining documentation gaps and confirm public availability before editing.

Run details

The companion change passed 39 repository tests and 38 offline checker tests. Server parser validation passed; state-dependent apply checks and live Slack delivery were not run.

Validation

  • Audit regression suite: all 60 tests passed.
  • Existing fixture-based regression benchmark: all 24 cases derived from Falconer's documentation audit passed, including three delegation checks. Six additional false-positive controls also passed. This does not verify the current live docs.
  • Internal link checker: 4,482 links checked, zero broken links.
  • Python compilation, eval JSON validation, and git diff --check passed.
  • Generated and inspected deterministic before-and-after previews for blocked, partial, and early-preflight failures.
  • Trunk was unavailable locally. The site build was not run because public pages and site code are unchanged.

Rollout and scope

The audit remains detection-only. No public documentation claims, site redirects, schedules, or factory configuration changed. No live audit was triggered and no Slack message was posted. Live recovery still needs verification after merge.

The existing audit-state PR #876 remains a standing draft for findings and history, not a fixes PR to merge.

Unverified claims

No public product claims were added or changed. Scheduled execution and Slack delivery have not been tested live.

Documentation risk

Risk: low
Rationale: Internal Docs-team audit tooling and skill maintenance only; no public product claims or developer-facing commands are changed.
Docs override: none

Agent context

Investigation and approved implementation plan

Co-Authored-By: Warp agent@warp.dev

Ignore slash-only canonicalization when resolving redirect identities, preserve genuine cycle detection, and report audit outcomes with coverage and actionable next steps. Add safe early-failure artifacts, skill guidance, and regression coverage.

Co-Authored-By: Warp <agent@warp.dev>
@rachaelrenk rachaelrenk added the warpy-factory Opened by the Warp factory agents label Oct 21, 2026
@cla-bot cla-bot Bot added the cla-signed label Oct 21, 2026
@vercel

vercel Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs Ready Ready Preview Oct 21, 2026 7:11pm UTC

Request Review

@warp-for-oss

warp-for-oss Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

@rachaelrenk

I'm starting a first review of this pull request.

You can view the conversation on Warp.

I completed the review and no human review was requested for this pull request.

Comment /warp-agent-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review summary

The independent agent completed its review for this commit.

Findings

  • No findings.

Verdict

Approve

@warp-for-oss warp-for-oss Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overview

This PR updates the docs consistency audit skill and helper to ignore slash-only redirect canonicalization loops, add public-safe blocked-report artifacts, and improve Slack notification language for completed, partial, and blocked runs. The changed tests cover redirect normalization, blocked-report state handling, Slack markup escaping, partial/blocked resolution behavior, and notification output cases.

Concerns

  • No blocking correctness, security, comment-quality, test-quality, or spec-alignment concerns were found in the annotated diff.
  • spec_context.md reports that no approved or repository spec context was found, so there were no implementation-vs-spec commitments to enforce.

Verdict

Found: 0 critical, 0 important, 0 suggestions

Approve

Comment /warp-agent-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

@rachaelrenk rachaelrenk self-assigned this Oct 21, 2026
@warp-agent-staging warp-agent-staging Bot added the factory:docs-factory Label associated to the "docs-factory" factory label Oct 22, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review summary

The independent agent completed its review for this commit.

Findings

  • No findings.

Verdict

Approve

@warp-agent-staging warp-agent-staging Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blocking findings at 44b3325. All 12 checks pass; 60 regression tests and benchmarks passed. Verified complete, partial, and blocked rendering against the requester-provided Slack screenshot. Live scheduled-audit recovery was not exercised.

Responding as Docs Factory (V2): Open session · View in factory

@rachaelrenk
rachaelrenk merged commit 542d4e7 into main Oct 22, 2026
24 of 25 checks passed
@rachaelrenk
rachaelrenk deleted the rrenk/docs-consistency-reporting branch October 22, 2026 21:30

This branch was successfully deployed

1 active deployment
Preview — 44b33250 Deployed Oct 21, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-signed factory:docs-factory Label associated to the "docs-factory" factory warpy-factory Opened by the Warp factory agents

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant