v3.5.1.0 fix: repair the landing-page guard and the mutation harness that hid its holes - #85
Open
savvides wants to merge 4 commits into
Open
v3.5.1.0 fix: repair the landing-page guard and the mutation harness that hid its holes#85savvides wants to merge 4 commits into
savvides wants to merge 4 commits into
Conversation
…its holes (v3.5.1.0) The mutation suite could not fail. fresh() copies the repo per mutation and omitted extension/, so test-extension.sh failed on every copy before any mutation was applied; since expect_fail only checks for a non-zero exit, all 36 mutations reported GUARDED regardless of whether their guard worked. Adds extension/ to the copy list and a null-mutation control that aborts the suite when an unmutated copy is already red. Fixes three defects on idstack.org, all phone-only: - .section had no safe-area inset, so every content region sat under the notch once viewport-fit=cover landed. The four containers now share one rule. - Three ad-hoc touch floors (36/40/42px), with the footer controls applying theirs only below 480px. One --tap-min token, 44px, at every width. - overflow-x: clip on html/body hid horizontal overflow instead of preventing it. Removed; the grid tracks that actually contain long install commands size with minmax(0, ...). Measured clean at 11 widths from 320px. Adds test/test-rendered-landing.js, which renders the page in headless Chrome and asserts the outcome rather than the CSS text. Six kinds of edit shipped a broken page past the text suite (a selector list, :is(), @container, a print-only media query, a media query nested in a desktop one, and a <style> inside an HTML comment); all six fail the rendered suite. No dependencies, and it skips loudly without a browser. Hardens the text suite where assertions accepted the regression they existed to catch, and stops it failing correct refactors. Every fix carries a mutation: 37 guarded, 0 not guarded, on a verified-green baseline. Landed as one commit: the page edits and their guards cannot be separated without a red intermediate, since the previously committed test asserts the overflow-x: clip this change removes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CLAUDE.md: the test-helper.sh note named two node suites and omitted test/test-rendered-landing.js, which the same release added. ROADMAP.md: drop the "ten today" suite count (stale again — the same defect CHANGELOG records for the mutation count) and point at CONTRIBUTING.md; add the v3.5.1.0 entry under "Just shipped". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CLAUDE.md: integration-test is 51 assertions, not 48. CLAUDE.md, CONTRIBUTING.md: "sourced by every bash suite" was still wrong after this release's rewrite — test/test-extension.sh is a bash suite and does not source test-helper.sh, because it delegates to node and asserts nothing of its own. smoke-test enforces the sourcing on nine suites. ROADMAP.md: drop the doc-maintenance aside from the CI bullet; the roadmap records what shipped, not how the file is kept accurate. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CI was red on all three legs with "WebSocket is not defined". test/test-rendered-landing.js drives Chrome over the global WebSocket, which node exposes unflagged only from 22.4, and the setup-node step this release added pinned 20. The failure was invisible locally, where node 24 gives a green 362/362 — so the release that repaired the landing-page guard shipped a guard that could not run in the one place it matters. Also: - smoke-test.sh gated three node checks but printed two SKIP lines, so a node-less run reported a short total with nothing naming the third omission. That is the exact failure the SKIP lines exist to prevent. - The mutation job comment said "mutations 23-31"; it is 23-32, and 32 needs Chrome, not node. - CHANGELOG said the rendered suite "needs no dependencies". True of npm, misleading about the runtime; it now states the node 22.4 floor. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three phone-only defects on idstack.org, and a repair of the test harness that was supposed to catch them.
The mutation suite could not fail.
fresh()copies the repo into a throwaway directory per mutation and omittedextension/, sotest-extension.shfailed on every copy before any mutation was applied. Becauseexpect_failonly checks for a non-zero exit, all 36 mutations reported GUARDED regardless of whether their guard worked. The suite whose job is proving the other tests work had been unable to fail since the extension landed. Fixed, plus a null-mutation control that aborts when an unmutated copy is already red.idstack.org on a phone
.sectionhad no safe-area inset, so every content region sat under the notch onceviewport-fit=coverlanded. The four containers now share one rule.--tap-mintoken, 44px, everywhere.overflow-x: cliponhtml/bodyhid horizontal overflow instead of preventing it, turning a real overflow into unreachable content. Removed; the grid tracks that contain long install commands size withminmax(0, …).New: the page is tested by rendering it.
test/test-rendered-landing.jsloads the page in headless Chrome and asserts the outcome at 11 widths. The existing suite checks CSS as text, which can only forbid spellings someone thought of — six kinds of edit shipped a broken page past it. All six fail the rendered suite:.copy-btn, .btn-badge { min-height: 30px }:is(.copy-btn)wrapper@containerhoist of the floor@media only print and (max-width: 480px)@media (min-width:900px){@media(max-width:480px){…}}<style>inside an HTML commentGuard hardening. Assertions accepted the regressions they existed to catch:
(clip|hidden)allowed theoverflow-x: hiddenthat breaks the sticky nav;grid-template-columns: 1frmatched the1fr 1frit forbids; a--tap-minexistence check passed while a later rule shipped 30px targets. Guards also no longer fail correct changes (adding a fifth container, reformattingminmax(0, 1fr)).Test Coverage
63% (33/52 paths), 19 gaps, no regressions. Above the 60% minimum, below the 80% target — accepted, with the two largest gaps closed structurally by the rendered suite rather than by more text assertions.
Coverage was measured by mutating each parser helper branch-by-branch and re-running: a branch counts as covered only if breaking it breaks something.
Coverage diagram
Pre-Landing Review
Six passes: 4 cleanup agents, 5 specialists, a coverage audit, a red team, and a Claude adversarial pass. 28 findings, all actioned. Every critical was in the test file, never in the page — the CSS came back clean from all three agents that measured it (container geometry byte-identical at 8 widths, zero overflow at 12 widths unclipped, type ratios preserved within 1%).
Specialists: performance NO FINDINGS · simplification 1 advisory · design 1 · maintainability 13 · testing 13.
Two findings were rejected on measurement, both claiming a removed
min-width: 0would overflow the page: reproduced as identical geometry, zero overflow.overflow-x: auto/overflow: hiddenon the same element already zeroes the automatic minimum size.Design Review
Design specialist: 1 finding (a wrong figure in DESIGN.md, fixed). Verified clean by measurement — container geometry unchanged before/after at 8 widths, no heading/body inversion from the collapsed clamps, focus ring keeps 8px headroom inside
.install-pill, and the taller controls read as normal proportions.Eval Results
No prompt-related files changed — evals skipped.
Adversarial Review
Claude adversarial pass: 3 blocking findings, all fixed and mutation-guarded. Codex did not run — the CLI is installed but its vendored binary is missing (
spawn .../codex-darwin-arm64/vendor/.../codex ENOENT), so this PR has no cross-model coverage.npm install -g @openai/codexrestores it.Verification
CI was red on the first three pushes:
WebSocket is not defined. The rendered suite drives Chrome over the globalWebSocket, which node exposes unflagged only from 22.4, and thesetup-nodestep pinned 20. Invisible locally on node 24. Fixed in4a5aafd.Follow-ups (not in this PR)
env(safe-area-inset-*)has zero executed coverage. The notch-safe gutter that motivated this refactor is asserted by declaration text alone. CDP cannot set safe-area insets; this needs a device or simulator.static.ymldoes not gate ontest.yml. Pages deploys on push tomainregardless of a red suite, which undercuts this entire regression gate. Pre-existing.check-doc-accuracy.py:152pins the literal"353 assertions", two revisions dead, so the guard that should catch documented-count drift is unreachable. It silently permitted 353→355→361→362.check-doc-accuracy.pycarries two byte-identical blocks for CLAUDE.md and CONTRIBUTING.md; this PR had to make the same(?:sh|py|js)edit twice.unconditional()andallMediaBodies()are exact complements computed by two separate@mediawalks; one 12-linesplit()returns both. Verified byte-identical, ~17 lines removable.docs/superpowers/plans/2026-08-20-…mdstill prescribes the three ad-hoc touch floors and themin-width: 0this PR replaced. An agent told to implement it would reintroduce three mutation-guarded defects.