feat(qa): CodePress-parity QA gate set for staging PRs (#5319) - #9
Conversation
- docs/qa: gate-set README, canonical QA report format, verification graph - scripts/qa-head-binding.mjs: emit/validate exact base/head + clean-tree binding - scripts/qa-browser-check.mjs: browser discovery (system Chrome, then Playwright chromium), renderFrame smoke, screenshot/log/manifest artifacts; exit 3 = BLOCKED when no browser session exists - scripts/qa-redaction-check.mjs: no secrets/tokens/local paths/media bytes in QA docs and artifacts - wire qa:binding, qa:browser, check:qa-redaction into package scripts/lint - regenerate provenance package.json SHA256 pins for the script changes - fallowrc: exclude scripts/qa-*.mjs from health complexity alongside the existing local-quality-tooling exclusions
Canonical QA ReportVerifier revision bindingbase: 1a570eb (origin/staging) Gate results
Inherited baseline failures
Environment blockers
Visual / browser artifacts
Overall✅ PASS — PENDING JUDGE — all gates pass except disclosed inherited baseline failures reproduced at base 1a570eb; browser evidence produced with system Chrome 151. No merge until trusted judge PASS. |
|
@codepress /judge-verification can you judge this verification? The canonical QA report above (head 9ecf66f, base 1a570eb) is the current-head evidence for this QA-parity gate infrastructure PR. |
|
👋 Hey @lightninglu10! Spawning a verification judge session to help with your request. View session |
|
Dispatched by judge-verification. Work the Path to merge below. Reuse healthy exact-head verification environments; do not redeploy or restart unless stale or unhealthy. Continue only when there is meaningful work to clear the blocker. If the path is already satisfied, the same blocker remains, or another judge pass would add no useful evidence, stop and report that instead of submitting another verification report. Verification Judgment — PR #9 (local)Verdict: ❌ NOT READY TO MERGE PR Head SHA: Summary: The report is bound to the live head and its 20 gate observations are credible; the inherited failures are immaterial to this diff, and the browser row has only a publication caveat. The PR is not safe to merge because the new redaction gate does not enforce its stated privacy boundary: it misses generic absolute paths and short embedded-media data URIs, while browser log sanitization only replaces two known roots. A clean fixture scan cannot establish fail-closed behavior. Revision binding
Diff trigger inventory
Per-item
Overall status check
Path to merge
Becomes Dispatch: yes What stood up
|
Judge FAIL follow-up: revert the .fallowrc.json health-ignore exclusion for scripts/qa-*.mjs and instead decompose qa-head-binding/qa-browser-check/ qa-redaction-check main flows so every function sits under the fallow CRAP/cognitive thresholds (0 introduced findings, verified with fallow audit). Also correct docs/qa/README.md: verify aggregates gates 2 and 4-13 (lint is not part of it), and gate 3 documents that lint requires the editor-surface dist on a fresh tree (pre-existing TS2307 quirk, present at base).
Canonical QA Report (re-run after judge FAIL fix)Supersedes the report at head Verifier revision bindingbase: 1a570eb (origin/staging) Gate results
Inherited baseline failures
Environment blockers
Visual / browser artifacts
Overall✅ PASS — PENDING JUDGE — all gates pass except disclosed inherited baseline No merge until trusted judge PASS. |
- qa-redaction-check: replace the three known-root path patterns with a generic absolute-path pattern (any Unix root or Windows drive path, URL prefixes excluded) and match any image/video/audio data URI regardless of payload length; secrets keep their patterns. Add a per-line qa-redaction:allow marker for intentional doc examples (secrets never allowlisted). - qa-browser-check: sanitize logs with the same generic path pattern instead of replacing only ROOT and os.homedir(). - Add deterministic negative fixtures (scripts/fixtures/qa-redaction/) and a node:test self-test proving the gate fails closed on a tmp path, a generic absolute path, a short media data URI, and a Windows path, and passes on the clean set; wired into check:qa-redaction itself. - Regenerate provenance package.json SHA256 pins for the script changes.
|
@codepress /judge-verification can you judge this verification? Local VerificationRemediation for judge G1 at previous heads: the redaction gate no longer PR Head SHA: Verifier revision bindingbase: 1a570eb (origin/staging) Diff Trigger Inventory
Verification Contract Results
Inherited baseline failures
Environment blockers
Overall: PASS — pending judge; only the disclosed inherited baseline failures (reproduced at base) remain red, and G1 is fixed with negative-fixture proof at the bound head.No merge until trusted judge PASS. |
|
👋 Hey @lightninglu10! Spawning a verification judge session to help with your request. View session |
|
Dispatched by judge-verification. This judgment is READY TO MERGE at the current head. If the PR is still a draft, transition it to ready for review now with the guarded readiness transaction: Verification Judgment — PR #9 (local)Verdict: ✅ READY TO MERGE PR Head SHA: Summary: The current-head report is credible and complete for the QA infrastructure diff. The exact-head binding is clean, the live browser and headless observations exercise their real harness surfaces, the inherited failures are unchanged at base and outside the changed surfaces, and the prior redaction false-negative gap is fixed in code with deterministic negative-fixture proof. Revision binding
Diff trigger inventory
Per-item
Overall status check
What stood up
|
|
Report correction (form-only, gate 18): the browser-QA screenshots/logs cited in the canonical report at head a61c537 are local gitignored artifacts under artifacts/qa/browser-a61c537b9e3a/, not durably published visual evidence. Their durable record is the manifest: frame.png sha256 8206275c7f9794b6f22fb726923957fbde05fbb422c0cded71300306eb872cd9 (15682 bytes), harness.png sha256 2360d2cc7439d7cb3ad958691f0d9ba7510d6c2ed3779538ff45d797aa15538b (7535 bytes), console.log sha256 01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b. Reproduce with npm run qa:browser -- --skip-build at the bound head on a machine with a browser session. This is the documented form caveat accepted for this PR; no behavioral rerun required. |
|
Starting initial review of |
![codepress-reviewer-dev[bot]](https://avatars.githubusercontent.com/in/3963614?s=60&v=4){kind=small}
There was a problem hiding this comment.
The QA-gate shape is sound: a small, local checker with fail-closed fixtures and matching browser-log sanitization is appropriate for these generated artifacts.
Blocking
scripts/qa-redaction-check.mjs:34— The generic Windows branch matches only backslash paths. A valid absolute local path such asC:/Users/builder/freecut/dist/bundle.jsis not detected, and the same expression inscripts/qa-browser-check.mjsconsequently leaves it in the emitted console log. This contradicts the documented “any Windows drive path” privacy boundary. Please support both path separators and add a forward-slash Windows negative fixture to keep the checker and sanitizer fail-closed.
With that correction, the rest of the QA infrastructure looks merge-ready.
Reviewed `1a570ebd`...`a61c537b`. Diff collection and GitHub posting were handled by the automation.
|
Review head |
The generic absolute-path pattern's Windows branch only matched backslash separators, so C:/Users/... passed both the checker and the browser-log sanitizer. Accept either separator in both scripts and add the negative-windows-forward-slash fixture to the fail-closed self-test (now 6/6).
|
@codepress /judge-verification can you judge this verification? Local VerificationReviewer follow-up (codepress-reviewer-dev[bot], CHANGES REQUESTED): the PR Head SHA: Verifier revision bindingbase: 1a570eb (origin/staging) Diff Trigger Inventory
Verification Contract Results
Inherited baseline failures
Environment blockers
Overall: PASS — pending judge; only the disclosed inherited baseline failures (reproduced at base) remain red, and the review finding is fixed with a new negative fixture at the bound head.No merge until trusted judge PASS. |
|
👋 Hey @lightninglu10! Spawning a verification judge session to help with your request. View session |
|
Dispatched by judge-verification. This judgment is READY TO MERGE at the current head. If the PR is still a draft, transition it to ready for review now with the guarded readiness transaction: Verification Judgment — PR #9 (local)Verdict: ✅ READY TO MERGE PR Head SHA: Summary: The current PR head is safe to merge based on the bound report and the audited diff. The report is bound to the live head with verifier-side clean-tree capture, all material QA surfaces have current-head evidence, the three failing static rows are disclosed baseline findings outside the diff, and the prior redaction gap is fixed with deterministic negative coverage. Browser artifacts remain a publication form caveat only. Revision binding
Diff trigger inventory
Per-item
Overall status check
What stood up
|
|
Report correction (form-only, gate 18): the browser-QA screenshots/logs cited in the canonical report at head 439dad9 are local gitignored artifacts under artifacts/qa/browser-439dad966b7c/, not durably published visual evidence. Their durable record is the manifest: frame.png sha256 8206275c7f9794b6f22fb726923957fbde05fbb422c0cded71300306eb872cd9 (15682 bytes), harness.png sha256 2360d2cc7439d7cb3ad958691f0d9ba7510d6c2ed3779538ff45d797aa15538b (7535 bytes), console.log sha256 01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b. Reproduce with npm run qa:browser -- --skip-build at the bound head on a machine with a browser session. This is the documented form caveat accepted for this PR; no behavioral rerun required. |
|
Starting new changes review of |
There was a problem hiding this comment.
The requested Windows-path remediation is complete. The shared absolute-path expression now accepts both backslash and forward-slash drive paths, and the new negative fixture ensures C:/Users/... remains fail-closed through the gate.
The scoped scanner plus matching browser-log sanitization remains an appropriate lightweight boundary for these generated QA artifacts. No new actionable issues found in this follow-up diff.
Reviewed `a61c537b`...`439dad96`. Diff collection and GitHub posting were handled by the automation.
1 participant
QA-parity gates for FreeCut (#5319)
Establishes the CodePress-equivalent QA gate set for FreeCut as QA
infrastructure only — no product/feature code changes. Motivated by PR #8,
whose deterministic gates were green but whose canonical QA was BLOCKED on
"browser discovery returned no available browser session".
Verifier revision binding
base: 1a570eb (origin/staging)
head: 9ecf66f
What this PR adds
docs/qa/README.md— the full 20-gate set, how to run it, the BLOCKEDrule, inherited-failure disclosure rule, and draft-to-ready discipline.
docs/qa/canonical-report.md— canonical QA report format: revision-bindingblock, per-gate PASS/FAIL/BLOCKED with bounded evidence,
PASS — PENDING JUDGEas the strongest verifier self-verdict, no merge until trusted judgePASS.
docs/qa/verification-graph.md— FreeCut-local verification graph ofcross-feature couplings that break silently (host-surface contract vs
package exports vs standalone mode,
window.freecutharness API, provenancepins, fallow allowlist ratchets, edge budgets).
scripts/qa-head-binding.mjs(npm run qa:binding [-- --check]) — emits andvalidates the exact base/head + dirty-tree cleanliness block every canonical
QA report must carry.
scripts/qa-browser-check.mjs(npm run qa:browser) — browser discovery(system Chrome via
channel: 'chrome', then Playwright-bundled chromium),drives the headless render harness, writes
frame.png/harness.png/console.log/manifest.json(with SHA256s) to gitignoredartifacts/qa/browser-<short-head>/. Exit 3 = BLOCKED (no browser session);BLOCKED never counts as browser evidence.
scripts/qa-redaction-check.mjs(npm run check:qa-redaction) — fails if QAdocs/artifacts contain secrets, tokens, raw absolute local paths, or embedded
media bytes.
package.json— wires the three gates into scripts and the lint/format filelists.
provenance/— regeneratedpackageJsonSha256pins required byverify:provenanceafter thepackage.jsonscript changes (gate work, notscope creep;
npm run verify:provenancepasses)..fallowrc.json— excludesscripts/qa-*.mjsfrom health complexity next tothe existing
scripts/check-fallow-*.mjslocal-quality-tooling exclusion.Root cause of PR #8's missing browser session
This environment has system Chrome but no Playwright-bundled chromium
(
~/Library/Caches/ms-playwrighthas nochrome-headless-shell). PR #8's QAran where neither discovery target existed.
qa:browsermakes discoveryexplicit, and the docs define the remediation: run where system Chrome exists
or
npx playwright install chromiumfirst. The gate is never waived — it isBLOCKED until re-run with a browser.
Inherited baseline failures (reproduced at base 1a570eb, not caused by this PR)
npm run lint— FAIL: TS2307Cannot find module '@quantfive/freecut-editor-surface'inpackages/freecut-editor/consumer-smoke.test.tsx; identical at base.npm run test:run— FAIL: 581/4829 tests fail (44 files) with the knownjsdom
localStorageundefined signature via zustand persist; reproduced atbase (e.g.
src/headless/edit.test.ts18/18 fail at base).npm run check:unused-exports/npm run check:unused-class-members—FAIL at base with identical findings (fallow ratchet drift).
npm run format:check— FAIL at base: 168 pre-existing files; none touchedby this PR. Not part of the
verifyaggregate.No allowlist, skip-list, or config was edited to hide any inherited failure.
Gate results at head 9ecf66f
All deterministic gates pass except the inherited failures above. The full
canonical QA report with per-gate evidence is posted as a PR comment.
This PR stays DRAFT until a trusted judge returns PASS. No self-merge.