Conversation
Contributor
|
Contributor
Visual regression reportCypress suite: ✅ Passing Visual diff:
Accessibility (axe): ✅ No violations. 📊 View full report — click a screenshot's ⚠ badge to see each violation boxed on the image, with the offending element named and contrast failures shown as color swatches. Diff images (3)badge-canvas.png — 1573 pixels differtooltip-dark.png — 881 pixels differtooltip-light.png — 956 pixels differBaselines come from the |
matyasf
reviewed
Sep 7, 2026
| run: npx wait-on http://localhost:3000 --timeout 120000 | ||
| working-directory: regression-test | ||
|
|
||
| # `trashAssetsBeforeRuns` wipes cypress/screenshots at the start of every |
Collaborator
There was a problem hiding this comment.
trashAssetsBeforeRuns -- what is this referencing?
balzss
marked this pull request as draft
September 7, 2026 11:21
…l regression diffs The suite failed on screenshots that were pixel-identical to their baseline but translated by about a pixel. When a layout box rounds one device pixel differently the whole painted subtree moves, so a direct comparison lights up every edge in the image: on a real baseline, a 1px translation produces 2825 differing pixels. That is not a visual regression, and the suite should not report one. visual-diff now retries the comparison at small integer offsets before calling a screenshot changed, behind --max-shift (default 1, 0 restores exact matching). Scoring happens over the region the two images share, which also covers the related case of a screenshot one pixel taller with identical content. A pixel-count tolerance cannot do this job: the shift above and a genuine 40x40 recolour differ by less than a factor of two, so any threshold loose enough to absorb the first would hide the second. viewportHeight goes 800 to 2000 to cut the stitching that produces those shifts. capture: 'fullPage' scrolls the viewport down the document and stitches the slices, and the stitch is the least reproducible part of the capture. Measured against the baselines branch, this takes the suite from 58 stitch operations to 37, and from 17 of 32 pages captured in one pass to 28. The viewport change alters layout, so the first run shows a cascade of changed rows; merging refreshes the baselines. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
balzss
force-pushed
the
INSTUI-5180-visual-regression-false-positives
branch
from
September 7, 2026 14:52
2bbb988 to
29285f7
Compare
balzss
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
visual-diffretries the comparison at small integer offsets before calling a screenshot changed, behind--max-shift(default 1,0restores exact matching). Scoring over the shared region also covers a screenshot one pixel taller with identical content.viewportHeight800 → 2000, cutting thefullPagestitching that produces those shifts: 58 stitch operations → 37, and 17/32 → 28/32 pages captured in a single pass (measured against thevisual-baselinesbranch).A pixel-count tolerance can't do this job. On a real baseline a 1px translation produces 2825 differing pixels and a genuine 40×40 recolour produces 1479 — less than a factor of two apart, so any threshold loose enough to absorb the first would hide the second.
Verified against four real CI baselines — identical, shifted 1px, 1px taller, and a genuine recolour:
--max-shift 0(current)Test Plan
changedrows on this PR — the viewport change alters layout, so it can't demonstrate its own success. Merging refreshes the baselines.changedrows in the report to confirm they're the viewport reflow and not something unexpected.Fixes INSTUI-5180
🤖 Generated with Claude Code