Skip to content

Serve the email templates' fonts and logo locally in matrix tests - #6883

Merged
backspace merged 3 commits into
mainfrom
buck-25-hermetic-email-assets
Oct 22, 2026
Merged

backspace merged 3 commits into
mainfrom
buck-25-hermetic-email-assets

Conversation

@backspace

@backspace backspace commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Background and Goal

profile.spec.ts › it can change email in settings failed 4 times in 69 main runs in a week and passed on retry. Every first attempt failed the same way: email-change-verification-email.png differed by about 26,600 pixels (ratio 0.03). In the actual screenshot, the mail client's message pane is blank.

The Synapse email templates (support/synapse/templates/_base.html) load three Google Fonts stylesheets, a fonts.cdnfonts.com stylesheet and the Boxel logo from the internet. The stylesheets are in the email's <head>, so the browser paints nothing in the email's iframe until they load. The test checks only that the text "Verify Email" is in the iframe's DOM, and that is true before anything is painted. In the trace of one failure, the Poppins:wght@700 stylesheet had no response when the screenshot was taken.

The shared fixture now routes those URLs, for every page in the browser context, to the bytes the browser received from them. These are stored in packages/matrix/support/email-assets/, with the Poppins license. A request to those hosts that is not stored gets a quick 404 and a console warning, so a template change cannot bring the network dependency back without notice. The stored CSS is in .prettierignore so it stays byte-for-byte.

Verification

Three throwaway draft PRs hold the Poppins 700 stylesheet for 8 seconds and run the three email screenshot tests 5 times on each of the 3 matrix shards, with one worker and no retries:

Fixes CS-14183

🤖 Generated with Claude Code

https://claude.ai/code/session_01WdM45xnz3vvTmA7HN8nGmK

The Synapse email and validation-page templates load three Google Fonts
stylesheets, a fonts.cdnfonts.com stylesheet and the Boxel logo from the
internet. The stylesheets block rendering. When one is slow, the mail
client shows the email as a blank pane and the screenshot assertion
fails. In one failure the Poppins 700 stylesheet had no response when
the screenshot was taken.

The shared test fixture now routes those URLs, for every page in the
context, to the bytes the browser received from them. A request to
those hosts that is not stored gets a quick 404 and a warning, so a
template change cannot bring the network dependency back without
notice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WdM45xnz3vvTmA7HN8nGmK
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-21T10:17:06.439314Z 6ada687 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@FadhlanR FadhlanR left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Claude Code 🤖] This review checked whether routing the template's fonts and logo locally removes the blank-pane mechanism behind the email screenshot flake, whether anything outside the matrix tests changes, and the fix against the repo's flaky-test procedure. It did not re-run the matrix suite.

No blocking issues. Every email and validation-page screenshot runs in a page opened from the fixture's context (validateEmail uses appPage.context().newPage(), and the validation page comes from context.waitForEvent('page')), so the route reaches the iframe that was painting blank. The Synapse templates, and the workflow that syncs them to deployed environments, are untouched.

  1. Diagnostics in case another cause still blanks the pane. The new 404 warning catches a template that brings the network dependency back. It says nothing if the pane is blank for some other reason. Untraced first attempts are the ones that fail, so nothing would explain such a failure. In validateEmail, before onEmailPage, could you log the message iframe's document.readyState and document.fonts.status (or await document.fonts.ready there)? Then a recurrence shows whether the page or its fonts had finished loading. Non-blocking, this PR or a follow-up.
  2. The route covers every page in every matrix test, not only the email. See the inline comment on the warning in helpers/email-assets.ts. Non-blocking.

Comment thread packages/matrix/helpers/email-assets.ts
backspace and others added 2 commits October 22, 2026 02:17
The route is on the whole browser context, so a host page that loads
from the same hosts, such as a card theme's web font, reached it too
and got a 404 and a warning that called it an email asset. An unstored
request now gets the 404 and the warning only when it comes from the
mail client, the email in its iframe, or a page Synapse serves. Other
pages' requests go to the network as before. Stored URLs are still
served locally for every page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WdM45xnz3vvTmA7HN8nGmK
If the email pane is blank for a reason other than the external
stylesheets, the output now shows whether the email's document,
stylesheets, images and fonts had loaded. A first attempt that fails
has no trace, so this line is the only record of that state.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WdM45xnz3vvTmA7HN8nGmK
@backspace

Copy link
Copy Markdown
Contributor Author

[Claude Code 🤖] Thanks for the review. Both points are addressed in this PR.

  1. Diagnostics (10dbf18): validateEmail and validateEmailForResetPassword now log the email iframe's state right before onEmailPage: document.readyState, document.fonts.status, and each stylesheet and image with whether it has loaded. The line goes to the attempt's stdout in the HTML report, so a first attempt that fails without a trace still shows whether the email had finished loading. It only logs. Awaiting document.fonts.ready would not have covered the stylesheets, and waiting for them is the alternative fix that DO NOT MERGE: email screenshot flake check (wait) #6882 checked.
  2. Route scope (48e2bc1): answered on the inline thread.

@backspace
backspace merged commit 4450b4e into main Oct 22, 2026
44 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants