Skip to content

docs: add an advisory PR-time doc-integrity check to docs-ui-tests - #2991

Open
marcel-rbro wants to merge 1 commit into
masterfrom
docs/docs-tests-pr-integrity-check
Open

marcel-rbro wants to merge 1 commit into
masterfrom
docs/docs-tests-pr-integrity-check

Conversation

@marcel-rbro

@marcel-rbro marcel-rbro commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Part of #2671. Addresses the cause behind #2988.

The harness only ran weekly, so #2919 moved a doc on 09-01 and nothing surfaced it until 09-07 — into an unassigned issue that then sat 10 days.

Splits the workflow into two jobs:

Job Runs on Does
doc-integrity (new) PRs touching sources/platform/** or docs-tests/** doc-side checks only — each assertion's source_file/source_quote still resolves
ui-drift (unchanged) weekly + manual full suite against staging, files a drift issue

Informative only: continue-on-error: true, so a failure never blocks the merge. Findings render as a table in the job summary. Promoting it to a gate later means dropping that flag and marking the check required.

No playwright install and no secrets — the doc-side project never opens a browser, so it also works on fork PRs. Verified against the pre-#2988 state with no browser present: all 16 stale references reported in 4.3s.

Note: this PR only touches .github/workflows/, so the job won't self-trigger here. It first runs on the next docs PR after merge.

🤖 Generated with Claude Code

https://claude.ai/code/session_01FSj1mTUbMKgmu2JoGJLSKD

Splits the workflow into two jobs so doc-side drift is caught in review
instead of up to a week later.

doc-integrity runs on pull requests touching sources/platform/** or
docs-tests/**. It runs only the `integrity` project — static checks that
each assertion's source_file and source_quote still resolve. No browser
and no credentials, so it needs no Playwright install and works on fork
PRs, where secrets are unavailable.

The check is informative only: `continue-on-error` keeps a failure from
blocking the merge, and findings are written to the job summary as a
table of assertion IDs and their doc sources, so a reviewer sees them
without opening logs. Promoting it to a gate later means removing
`continue-on-error` and marking it required in branch protection.

ui-drift is unchanged except for an `if` guard keeping it off pull
requests. It still runs weekly and on demand, against staging, and still
files a drift issue on failure.

The motivation is #2951: #2919 moved account/settings.md on 09-01, the
baseline went stale, and nothing surfaced it until the scheduled run on
09-07 — into an unassigned issue that sat another 10 days. Run against
that state locally, this job reports all 16 stale back-references in
4.3 seconds with no browser and no credentials.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FSj1mTUbMKgmu2JoGJLSKD
@marcel-rbro marcel-rbro added documentation Improvements or additions to documentation. t-docs Issues owned by technical writing team. labels Sep 17, 2026
@marcel-rbro marcel-rbro self-assigned this Sep 17, 2026
@apify-service-account apify-service-account linked an issue Sep 17, 2026 that may be closed by this pull request
6 tasks
@apify-service-account apify-service-account added the tested Temporary label used only programatically for some analytics. label Sep 17, 2026
@apify-service-account

apify-service-account commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

✅ Preview for this PR (commit b07ef8cc) is ready at https://pr-2991.preview.docs.apify.com (see action run).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation. t-docs Issues owned by technical writing team. tested Temporary label used only programatically for some analytics.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Docs-as-tests: automated Console UI drift detection

2 participants