[tests] Analysis of potential flaky e2e tests (AI generated)

[susnux]

Playwright flakiness audit — findings & todo

Confirmed flaky on CI

• handlePasswordConfirmation helper ( test(playwright): answer password confirmation via a locator handler #64404 ) — support/utils/password-confirmation.ts (~12 call sites; flaked in e2e/appstore/admin-settings-apps.spec.ts via the two handlePasswordConfirmation calls in "Limit app usage to group").
  Still unchanged; now the highest-value fix in this list. Two defects: (1) line 19 probes for the dialog with a hardcoded 500 ms window and silently returns if it isn't visible yet — on a slow shard the confirmation is skipped and the guarded action never fires; (2) line 32 waits for hidden with no bound or retry.
  Fix: race the dialog appearance against the expected success response instead of a 500 ms probe; wrap fill + confirm + hidden in an expect(...).toPass() retry loop.

• Copy-to-same-folder tests time out — e2e/files/files-copy-move.spec.ts:84,95,111 and e2e/files/live-photos.spec.ts:39,50.
  Partly addressed, still open. The global CI timeout was raised 30 s → 45 s, but "can copy a file multiple times to the same folder" has since been observed timing out at 45 s in CopyMoveDialogPage.confirm's waitForResponse (passed on retry). The helper is correct — the client's same-folder copy path (apps/files/src/actions/moveOrCopyAction.ts) does PROPFIND (conflict check) → COPY → PROPFIND (stat) serially, once per copy.
  Fix: test.slow() on the copy-heavy tests specifically. Another global timeout bump only hides it.

High

• Systemtags specs destroy each other's tags across projects — support/utils/systemtags.ts:47 (clearTags() deletes all server-global tags) is wired into afterAll of four default-project specs (files-view, files-sidebar, files-inline-action, files-bulk-action), while e2e/systemtags/admin-settings.spec.ts:14-19 mass-deletes every tag in beforeEach from the concurrently-running admin-settings project, using the fixed names foo/bar.
  Confirmed structurally: the suite listing shows systemtags/admin-settings*.spec.ts in admin-settings and the four files-* systemtags specs in default; those projects run concurrently locally. This matches the known files-bulk-action.spec.ts "Can assign tag to selection" failure at --workers=2. deleteTag (non-forced) additionally throws if a tag vanishes underneath it.
  Fix: track and delete only the tags each test created; random tag names in the admin spec.

• admin-settings project is not serialized against the default project — playwright.config.ts. Per-project workers: 1 only serializes admin-settings tests among themselves; locally they interleave with default-project tests while mutating global server state. Worst offenders:

  • e2e/files_external/admin-settings-home-folder-root-mount.spec.ts:22-43 — mounts a read-only storage over every user's home and flips overwrites_home_folders
  • e2e/files_external/admin-settings-external-storage.spec.ts:20 — deleteAllGlobalStorages() in beforeEach
  • e2e/appstore/admin-settings-apps.spec.ts:12-23 — enables updatenotification and disables testing globally in beforeEach with no restore; this leak is what makes the hardcoded-count findings below brittle

  Fix: needs a mechanism other than project dependencies — that was considered and rejected because with --shard it can re-run the dependency project per shard. A separate serialized invocation, or moving the global-state mutations behind a lock, are the remaining options.

• Global system-config mutation from the parallel default project — same class as the above, but from the project that runs fully parallel:

  • e2e/settings/personal-info.spec.ts:12-18 / :20-25 — mutates has_internet_connection, force_language, force_locale. Worse than first reported: afterAll does not restore force_language/force_locale, it sets them to en/en_US, so the mutation leaks permanently into every later spec on a reused server.
  • e2e/login/login.spec.ts:32-38 — disables auth.bruteforce.protection.enabled globally for the duration of the file.

  Fix: config:system:delete in personal-info's afterAll instead of setting values; move both files to a serialized project or scope the config changes.

• Shared admin account preferences raced by parallel specs — e2e/users/users-columns.spec.ts:11-30 persists column visibility against the single admin account while users.spec.ts, users-disable, users-search, users-manager, users-modify, users-groups (all default project, parallel locally) assert on the same list's columns. Also users-groups.spec.ts:176-205 persists the group sort-order preference.
  Fix: serialize the admin user-list specs or isolate per-test accounts.

• Fixture login/user-creation band-aid: single retry after fixed 800 ms — support/fixtures/random-user.ts:23, random-user-session.ts:20, admin-session.ts:18. Unchanged. The transient is real (login returned HTTP 500 twice in a row in a local run on 2026-07-17, failing a test; createRandomUser non-zero exits and MKCOL 503s are documented too). One fixed-delay retry is not robust.
  Fix: retry with backoff (e.g. 3 attempts) or poll the login endpoint until healthy.

Medium

• ContactsMenuPage.search never awaits the debounced request — support/sections/ContactsMenuPage.ts:67-69. Docstring promises to wait for the /contactsmenu/contacts response but only fill()s a 500 ms-debounced input; callers assert on results immediately.
  Fix: register waitForResponse before fill() (same pattern as open()).

• isFullyInViewport fails on the very case it is asked to detect — support/utils/viewport.ts:61-76. The helper opens with await expect(locator).toHaveCount(1), which throws when a row is virtualized fully out of the DOM — i.e. the strongest form of "not in viewport". e2e/files/scrolling.spec.ts:45,84 poll this helper expecting false, so a fully-virtualized row fails the test instead of satisfying it. fitFilesListToRows additionally hardcodes row-height constants mirrored from the component.
  Fix: treat count 0 as "not in viewport"; derive row height from a rendered row instead of constants.

• FilesListPage.selectAll/deselectAll force-click with no state assertion — support/sections/FilesListPage.ts:356-367. A bare click({ force: true }) landing mid-render can no-op; the caller only fails later.
  Fix: assert the resulting checkbox/selection state after toggling. (Note: this applies to raw click({force:true}) only — check()/uncheck({force:true}) elsewhere in the suite already assert their own end state.)

• SettingsUsersPage.saveEditDialog — support/sections/SettingsUsersPage.ts:85-92. force: true click (can fire before the handler is wired) combined with the 500 ms password-confirmation probe.
  Fix: drop force, and fix handlePasswordConfirmation (above).

• AppstorePage.appRow substring match + .first() — support/sections/AppstorePage.ts:61-63. hasText is a substring match, so "Files" also matches "Files sharing"; .first() silently picks by DOM order. Feeds enableButton/disableButton/appLink.
  Fix: match the app-name cell exactly.

• files_trashbin/files.spec.ts sharee deletes before share propagation is confirmed — e2e/files_trashbin/files.spec.ts:67-79. Bob rms via his DAV path with no waitForShare poll; rm throws on 404. The helper exists and is used by eight other specs.
  Fix: reuse the waitForShare poll before acting as the sharee.

• Hardcoded global-state counts — e2e/core/header-access-levels.spec.ts:19,48 (toHaveCount(6)/(9) account-menu entries), e2e/theming/user-settings-app-order.spec.ts:17 and e2e/theming/admin-settings-default-app.spec.ts:50 (toHaveCount(2) nav apps). Brittle to any globally enabled app — see the unrestored app enable/disable in the admin-settings finding above.
  Fix: assert on the specific expected entries, not totals.

• a11y-color-contrast.spec.ts reads global theming state from the parallel project — e2e/theming/a11y-color-contrast.spec.ts:49-116. Primary color is global admin state mutated by admin-settings-colors.spec.ts; axe runs once after goto with no wait for theming CSS application.
  Fix: wait for the theming stylesheet/CSS variables before running axe; also covered by the project-serialization fix above.

• files-renaming.spec.ts one-shot selection read + layout-measured virtualization test — e2e/files/files-renaming.spec.ts:43-45 reads selectionStart/End once with no retry; :132-174 computes exact viewport heights and scrolls to trigger re-render.
  Fix: poll the selection read; loosen the layout assumptions.

• files_external/admin-settings-home-folder-root-mount.spec.ts one-shot occ config reads + admitted HACK — lines 26/36/41 expect(await getOverwritesHomeFolders()).toBe(...) without expect.poll; lines 34-35 double open() as a propagation workaround.
  Fix: wrap the config reads in expect.poll; replace the double-open with an explicit wait on the observable effect.

• theming.pickColor positional swatch selection — support/utils/theming.ts:67-79. Partly addressed — there is now an expect.poll confirming the trigger's colour actually changed, which catches a dropped selection. Residual: the swatch is still chosen by positional .nth(index) with no stable identity, and the trigger is still force-clicked.
  Fix: select by colour value/label rather than index.

• header-contacts-menu.spec.ts enumeration on a shared server — e2e/core/header-contacts-menu.spec.ts. Partly addressed — the file now sets test.describe.configure({ mode: 'serial' }), which covers the global enumeration-config toggle within the file. Residual: the tests still assert that random users appear in the default, limited contacts listing, and accumulated users on a reused server can push them out.
  Fix: search for the exact user instead of relying on the default listing.

• personal-info.spec.ts uses the repo's only networkidle waits — e2e/settings/personal-info.spec.ts:39 and :371. networkidle is timing-fragile with background polling.
  Fix: replace with assertions on concrete post-reload UI state. (The config-mutation half of this finding was promoted to High, above.)

Low

These are real, but note that the first two cause silent false passes rather than
flakes — they will never appear in a flaky-test report, so they are worth fixing on
correctness grounds even though they cost no CI time.

• drag-n-drop.spec.ts one-shot putFired boolean — e2e/files/drag-n-drop.spec.ts:112-124. Read the instant the conflict dialog appears; a slightly-late PUT is missed, so the assertion passes while the guarantee is violated. The multi-file drop test (:29-42) also has no waitForResponse for the PUTs, unlike the single-file tests. Whole file uses synthetic dispatchEvent DnD — inherently fragile.
  Fix: wait for a settle window / expected request count before asserting; add PUT waits to the multi-file test.

• files-external-failed.spec.ts synchronous URL comparison — e2e/files_external/files-external-failed.spec.ts:42-44,73-75. expect(page.url()).toBe(url) immediately after a click passes if a navigation is merely slow.
  Fix: assert with expect(page).not.toHaveURL(...) / an explicit negative wait. (The isVisible()-then-reload-once pattern at :33-36,65-67 is benign — the following toBeVisible retries — so only the URL assertions need changing.)

• SetupPage.installRecommendedApps swallows a genuine timeout — support/sections/SetupPage.ts:122-141. .catch(() => {}) hides "prompt never consumed"; loop trusts caller-supplied appCount.
  Fix: fail loudly on the timeout; derive the prompt count from the page.

• Generic unnamed getByRole('dialog') locators — support/sections/CopyMoveDialogPage.ts:23-24, BackgroundFilePickerDialogPage.ts:16-17, AppstorePage.ts:146-148. Match any open dialog; fragile if a second dialog/toast overlaps.
  Fix: name-scope the dialogs.

• dav/availability.spec.ts fixed replacement-user — e2e/dav/availability.spec.ts:48-95. Pre-delete + finally mostly handle it, but a fixed global username remains a latent collision.
  Fix: random user name.

• users-modify.spec.ts loose group-name regex — e2e/users/users-modify.spec.ts:124-143. The 6-char UUID fragment is matched via an unanchored RegExp, so it could hit another group.
  Fix: exact-match the option.

• login-redirect.spec.ts URL fragment in redirect regex — e2e/login/login-redirect.spec.ts:42-44. Fragments aren't sent to the server; brittle if the app normalizes the value.

Withdrawn — not actual problems

• login.spec.ts global brute-force toggle lets failures accumulate and 429-throttle other suites — incorrect mechanism. Throttler::registerAttempt() returns early when auth.bruteforce.protection.enabled is false (lib/private/Security/Bruteforce/Throttler.php:54-55), so no attempts are recorded while the file has protection disabled, and there is nothing to throttle once afterAll re-enables it. The residual concern — a default-project spec mutating global system config at all — is tracked under the High finding above.

• FilesNavigationPage.setShowHiddenFiles force-toggle without state assertion — incorrect. Playwright's check()/uncheck() assert the resulting checked state themselves and retry; force only skips the actionability checks, not the post-condition. The same applies to the check/uncheck calls in users-columns.spec.ts. Only bare click({ force: true }) lacks verification, which is tracked separately under FilesListPage.selectAll.

• hotkeys.spec.ts races keyboard focus — fixed in 7ceba63 (2026-09-14): every keypress is now gated on await expect(page).toHaveURL(/\/apps\/files\/files\/\d+/). The "Delete asserts row count with no DELETE response wait" sub-claim was never valid — toHaveCount is a retrying assertion and self-heals.

Done

• files-settings.spec.ts "Can set it to personal files" re-navigates before the default_view config PUT persists — fixed by registering waitForResponse for /apps/files/api/v1/config/default_view before clicking the radio (same idiom as FilesListPage.enableGridView). test(files): wait for PUT before continue with tests #62284
• Setup wizard DB installs run under the default 30 s timeout — e2e/core/setup.spec.ts:136 (MySQL) and :145 (MariaDB). test: mark installation tests as slow #62282
• users.spec.ts fixed username with cleanup outside finally — newuser-basic leaked on assertion failure and collided on the reused local server; now a random id + finally. test(users): prevent flaky tests with hardcoded values #62283
• files-delete "can delete multiple files" — the 5 waitForResponse listeners had identical predicates (all resolving to the same first DELETE) and a one-shot raw-status assertion that parallel DAV DELETEs could trip with a transient 423. test(files): wait for actual requests when deleting files #62286
• CI test timeout raised to 45 s and several unbounded FilesListPage waits bounded — 576d78c. This reduced, but did not eliminate, the copy-to-same-folder timeouts tracked above.

Suggested order of work

If only three things get fixed, these give the most per change:

1. handlePasswordConfirmation — one file, ~12 call sites, the only finding with a repeat CI failure attributed to it.
2. Systemtags tag cleanup — scope clearTags() to the tags each test created; one util plus five afterAlls, and it kills the known --workers=2 failure.
3. personal-info's afterAll — config:system:delete instead of setting force_language/force_locale; a two-line change that stops a permanent global leak.