Skip to content

ci: make the visual comparison mean something again - #38

Open
sinups wants to merge 1 commit into
mainfrom
ci/visual-that-means-something
Open

sinups wants to merge 1 commit into
mainfrom
ci/visual-that-means-something

Conversation

@sinups

@sinups sinups commented Oct 22, 2026

Copy link
Copy Markdown
Owner

Why

The visual job has been red on every pull request, including one that changed only the version and the changelog, and always on the same stories with the same pixel counts:

story #36 (elicitation only) release PR of Oct 9 (no UI at all)
ChatLauncher, shadow DOM 4.94384765625% 4.94384765625%
ShellOutput, ToolGroup, MessageActions, BackgroundTasksPanel, MessageListAdditions red red

An identical pixel count across unrelated pull requests is not antialiasing noise — it is two stable rendering states. A gate that fires on everything stops being read, and this one held 0.8.0 for four days.

What changes

Three causes, three fixes:

  • The freeze did not reach shadow roots. addStyleTag stops at the shadow boundary, so a widget mounted into one kept animating and the baseline and the comparison caught different frames. The freezing stylesheet is now adopted into every shadow root on the page.
  • A self-mounting widget was raced. ChatLauncher's shadow story mounts in an effect; it now reports readiness with data-shadow-ready and the screenshot waits for it.
  • Motion driven from JavaScript ignores stylesheets. Loaders, voice levels and pulse rings run on their own clocks. The page now asks for prefers-reduced-motion: reduce — the switch the kit already honours in useAnimationTime, SpiralLoader, the streaming fade and the launcher rings.

And the job stops running where pixels cannot move: with a paths filter, a release pull request or a change to the docs data no longer pays 25 minutes to compare two identical Storybooks.

How this is verified

This pull request touches .storybook/**, so its own visual job runs the full comparison. Green here means the three fixes hold; the stories listed above are the test.

yarn test (1571 tests) passes locally.

The job was red on every pull request, including one that changed only the
version and the changelog, and always on the same stories with the same pixel
counts — ChatLauncher at 4.94%, then ShellOutput, MicButton, TaskList and the
rest. A gate that fires on everything stops being read, and it held the last
release for four days.

Three causes, three fixes:

- A style tag stops at a shadow boundary, so the animation of a widget mounted
  into a shadow root kept running and the two sides of the comparison caught it
  in different frames. The freeze is now adopted into every shadow root.
- A widget that mounts itself in an effect reports when it is there
  (`data-shadow-ready`), and the screenshot waits for that instead of racing it.
- Loaders, voice levels and pulse rings move from JavaScript, which no
  stylesheet can stop. The page now asks for reduced motion, the switch the kit
  already honours everywhere.

The job also stops running where pixels cannot move: a release pull request, a
change to the docs data or to another workflow no longer pays 25 minutes for a
comparison of two identical Storybooks.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant