Repository navigation
CI: App tests on macOS Intel slow down as the run goes on #540
Description
On the macOS Intel runner the App tests take about 50 minutes on main (50 m 36 s in the run for #529), and later tests get slower and slower: in #516's run the same RenderTests cases went from about 22 s to 54 s each towards the end (Close_dialog_renders took 53–54 s per language). With #516's Portuguese render tests the job ran past 90 minutes, so #516 raised the portable job's limit to 120 minutes for now.
A test taking longer the later it runs points to something piling up between tests (windows, view models subscribed to services, bitmaps, fonts). Worth finding out:
- which App tests are slow on macOS Intel, and whether the same test is fast when run alone;
- whether memory grows over the run (a
dotnet-countersor periodic GC/memory log in CI); - whether render fixtures close their windows and dispose what they create.
The goal is to bring the job back well under an hour so the limit can go down again.
Activity
CI: give the portable jobs 180 minutes
SeoJCalive commentedon Oct 27, 2026
@KnifeLemon I'd like to give this a try. #628's macOS Intel job also ran out the 180 minutes in the App tests with no failures, while the same job on main took 1 h 53 m.
I don't have an Intel Mac, so this is what I'd do:
macoslabel and compare its Intel job time with main.I can't promise I'll find it, but I'll report what I find either way.
SeoJCalive commentedon Oct 27, 2026
@KnifeLemon I found what slows the run down, at least on Linux (a full local App run here: 125.8 min).
TestServices.EnsureInitializedswitches the language in almost every test. On its own a switch takes about 70 ms, but during the run it took 4.3 s (median) and up to 70 s late in the run.LocalizationService.SetLanguagesets the 1177 keys one by one on a dictionary merged intoApplication.Resources, and each set notifies every control of every window still alive. So one switch costs about 1177 × (live controls), and windows a test closed but that aren't collected yet still count. DuringTranslated_choices_fit_in_the_preset_dialogthe alive windows went from 13 to 35, and the switch went from 7 s to 70 s with them.MergedDictionariesin one step, so a switch sends one notification. With 8 main windows open a switch went from 11.3 s to 0.25 s, and the whole App run from 125.8 min to 12.2 min. Every one of the 1418 tests ended the same way in both runs, including the ones that check the text follows a language change. The app gets the same speedup when the user changes the language.No test is removed or changed. I'll open a PR with the
macoslabel so the Intel time shows in CI.SeoJCalive commentedon Oct 27, 2026
@KnifeLemon The PR is up: #636. While writing it I rechecked the numbers in my earlier comment, and a few were off:
RenderTests.Main_window_pages_render(page: "watch", language: "ar").Translated_choices_fit_in_the_preset_dialogandRenderTests.Main_window_pages_renderran in turns (not the first one alone), and windows alive after GC went from 7–9 to 36 (not 13 → 35).TopLevelsubscribes weakly and doesn't unsubscribe on close), but I didn't measure how much they add.SetItems, which sends one, and its full run took 12.0 min.TestServices.EnsureInitializedcalledSetLanguage974 times for 1418 tests, and 346 of those asked for the same language as the previousEnsureInitializedcall. Direct calls from tests weren't logged, so they aren't counted.EnsureInitializedin the stand-alone check (76 ms even with English kept).SetLanguagealone is 1 ms with no window open and 151 ms with one.The PR body has the corrected numbers.
Merge pull request #636 from KnifeLemon/test/540-app-test-slowdown
fix(i18n): publish language strings in one resource change (KnifeLemo…
SeoJCalive commentedon Oct 27, 2026
@KnifeLemon #636으로 목표(macOS Intel 1시간 미만)를 이뤘으니 이 이슈는 닫을게요. #636에서 따로 보기로 한 두 가지는 새 이슈로 옮겼어요.
SetupTexts.Publish: #B두 이슈 모두 제가 이어서 할게요.