Android: Chrome Custom Tab web content missing from `dump ui` since the persistent DeviceKitServer (#334), and sticky across a polling session

[rhoque-league]

Environment

• mobilecli: regressed at 1.0.1, still present through 1.0.10 (1.0.0 unaffected)
• devicekit-android agent: 1.2.6
• Device: Android emulator, google_apis system image, API 36 (Android 16)
• Chrome: stable, Custom Tabs

Summary

Since #334 ("dump ui via persistent devicekit server on android") made Android dump ui reuse a single persistent DeviceKitServer UiAutomation for the whole session, the web form fields inside a Chrome Custom Tab (e.g. the <input>s of a hosted OAuth login) frequently never appear in dump ui. The Custom Tab's application window is dumped, but Chrome's renderer (web) accessibility nodes inside it are not populated.

The miss is sticky: because every dump in a session shares one UiAutomation, a client that polls for an element (e.g. a 30s locator wait in a UI-automation framework) never recovers — every poll reuses the same stale connection.

Root cause (from the code)

Chromium builds a Custom Tab's renderer accessibility tree when a fresh accessibility client connects.

• Pre-feat: dump ui via persistent devicekit server on android #334, dump ui spawned a fresh one-shot ViewTreeDump instrumentation per call — a new UiAutomation (fresh a11y client) each dump — so Chrome (re)built the web tree, and a repeated/polling dump would eventually catch it.
• feat: dump ui via persistent devicekit server on android #334 replaced that with one persistent DeviceKitServer UiAutomation, established once (typically before the Custom Tab's web content loads) and reused for every subsequent dump. With no new a11y client, Chrome never rebuilds the web tree — and because all dumps share that one connection, polling can't recover it.

#334's "output schema verified identical" check exercised native UI only, not a WebView/Custom Tab, so this regression wasn't caught.

Reproduction

1. Android emulator (google_apis, API 36), current Chrome.
2. mobilecli dump ui on some screen A — this starts the persistent DeviceKitServer (its UiAutomation connects now).
3. From an app, open a Chrome Custom Tab hosting a web form (any hosted OAuth login with <input> fields).
4. mobilecli dump ui (repeatedly) → the web <input> nodes are absent; only the native Custom Tab chrome + application window appear.
5. adb shell am force-stop com.mobilenext.devicekit to drop the persistent server, then mobilecli dump ui again → the server relaunches with a fresh UiAutomation and the web fields can appear.

Concretely: a locator that polls dump ui for one of the form inputs times out for the entire wait window under the persistent server, but resolves when a fresh instrumentation is used.

Not a fix

--full (#398) — that only un-filters TYPE_INPUT_METHOD (soft-keyboard) windows. The Custom Tab is a normal TYPE_APPLICATION window that is already dumped; the missing part is the renderer nodes inside it, which --full does not affect.

Workaround

Launch Chrome with --force-renderer-accessibility (write it to /data/local/tmp/chrome-command-line; requires a rootable/userdebug image + adb root), which forces Chrome to build the renderer tree from the DOM regardless of a11y-client lifecycle. Needs root; not viable on non-rootable / Play-Store images.

Suggested fixes

• An opt-in flag/env to use the pre-feat: dump ui via persistent devicekit server on android #334 one-shot ViewTreeDump path (fresh UiAutomation per dump).
• Or refresh / re-establish the UiAutomation a11y client when a WebView / Custom-Tab window is foreground, so Chrome rebuilds the renderer tree.
• Or detect an application window hosting a Chrome/WebView renderer with an empty subtree and force a re-attach before returning the dump.

Notes

• Intermittent for a single dump (the initial connection occasionally lands while the tree is up), but reliably breaks a polling client — the common case for UI automation.
• Observable verified on API 36; the same mechanism explains reports appearing after the mobilewright 0.0.53 → 0.0.58 bump, which crossed mobilecli 1.0.0 → 1.0.1.

[gmegidish]

@rhoque-league wow, what a bug! :) thank you for this. will work on a fix. Maybe we should add this to "--full"? what do you think? I'll check the performance of allocating a new UiAutomation, might add 1 second to dumps (which is a lot when you use with mobilewright).

[rhoque-league]

@gmegidish Thanks for jumping on this!

The hesitation with --full: it puts the "do I need this?" decision on the caller. mobilewright polls dumps generically, so realistically it'd just pass it on every dump → everyone eats the ~1s.

The failure's visible right in the dump — a browser window with an empty web subtree — so mobilecli feels like the natural place to handle it: normal dump, and only reconnect UiAutomation when you spot that signature. One fix for every caller, cost only when it's needed.

Clarification: The --force-renderer-accessibility workaround only holds on API 34 — a warm Chrome on 36 doesn't rebuild even with it. Worth confirming the reconnect fixes 36.

[gmegidish]

@rhoque-league thanks for the update, it really helps.

So here's what I suggest. I hate the --full myself. I added --full recently, because the android keyboard was pushing all stable references. It shouldn't be abused here.

Let me 1. benchmark to see how much of an issue this is (maybe it's not that big of a slowdown). 2. Check this option, we might be able to tell by the viewtree if it's chrome or not, and then force a refresh. Flutter is not an issue, it's going through Dart VM now, so I dont know who has lazy accessibility tree nowadays.

I'll keep you updated soon. Thanks for the report! I believe you found a working workaround, that unblocks you for now. Right?

[rhoque-league]

@gmegidish Yes; the workaround unblocks us for now. The primary thing your fix would unlock for us is dropping the API-34 pin (the flag doesn't hold on 36). Happy to test on 36 whenever you've got something.

[gmegidish]

UiAutomation.clearCache() :D
I'll reproduce later today and verify that it's the case.

[gmegidish]

Couldn't reproduce this on an API 36 emulator (google_apis_playstore, Chrome 152, mobilecli main).
Tried: plain am start Custom Tab, warm Chrome, 3 min idle before opening, an app launching
CustomTabsIntent with warmup() + mayLaunchUrl() pre-render, and a 302 redirect into the page.

In every case the persistent server's dump had the full renderer tree within ~5s of load. A few
questions to close the gap:

1. Chrome version on the failing device (Settings → About Chrome), and does it also fail after Chrome updates?
2. Exact emulator image: google_apis vs google_apis_playstore, API 36 revision, arch (arm64 / x86_64), and the avdmanager/SDK image path?
3. Real device too, or emulator only?
4. How does your app open the tab: plain CustomTabsIntent, or with a CustomTabsSession (warmup / mayLaunchUrl)? Any partial/side-sheet or ephemeral Custom Tab?
5. Can you attach one failing mobilecli dump ui --format raw plus adb shell dumpsys window | grep -i customtab, so we can see the window with the empty subtree?
6. Anything non-default in chrome://version (flags, field trials)?

Any information to help me reproduce this would work.

[rhoque-league]

@gmegidish Thanks for digging in on this — and good questions, they sent me down a rabbit hole with a bunch of findings that reframe the whole thing.
TL;DR: it's not the Chrome version, and it's less about the page than about when the first dump happens.

It's not the Chrome version

I grabbed the exact Chrome 152.0.7977.82 APK, side-loaded it onto my failing emulator (install-multiple over the bundled 133.0.6943.137), and A/B'd them back-to-back on the same image. Identical — 152 fails exactly like 133. So a Chrome update doesn't fix it, and the version gap between our setups isn't the cause.

Minimal repro
open a Chrome Custom Tab with a web form and immediately loop dump ui every ~300ms → stays empty. sleep 5 && dump ui once → full tree.

The real trigger: when the first dump lands

• If a client polls dump ui in a tight loop starting the instant the Custom Tab opens — before Chrome has built the renderer a11y tree — the persistent server's UiAutomation locks onto the empty state and never recovers. I polled ~20s and the web fields never appeared on both 133 and 152.
• If instead you do a single dump after the page settles (~2–5s), you get the full tree every single time. I also confirmed a client-side "settle before polling" avoids it reliably (24/24 across 2s/5s/8s waits).

That's almost certainly why you couldn't repro — a single post-load dump works; it's the poll-from-t=0 that locks it. And a real locator-wait (Playwright-style) polls from the moment the tab opens, which is exactly what our sign-in does. It fits the "sticky across polling" framing, just pinned to the early poll specifically: pre-PR #334 each poll was a fresh UiAutomation so a polling loop self-healed; the one reused client can't.

Your questions

1. Chrome: 133.0.6943.137 (bundled) and 152.0.7977.82 (sideloaded) — both fail identically.
2. Image: system-images/android-36/google_apis/arm64-v8a, build BE2A.250530.026.F3, userdebug, arm64 — and also reproduced on google_apis_playstore/arm64-v8a (non-rooted).
3. Emulator only (arm64 Mac).
4. How the tab opens: the app uses Auth0.Android's CustomTabsIntent, but the lock reproduces purely from the persistent server + early polling, independent of the launch details.
5. The failing dump is just the Custom Tab window with its native chrome and an empty renderer subtree — happy to attach a raw one, but the repro above is the crux.
6. Used --disable-fre --no-first-run --no-default-browser-check, but no difference without these.

Hope this helps!

[rhoque-league]

Quick correction to my earlier comment — I'd floated gating the first dump on a web-load signal (DevTools Page.loadEventFired / readyState === "complete", or mobilecli webview wait --state load). That doesn't actually work, flagging it so nobody burns time on it.

Those signals all track the web page's load. But the dump reads the Android accessibility tree, and Chrome mirrors the Custom Tab's form into that tree ~1–2s after the page finishes loading. Measured: the web-side signals fire well before the fields appear on the Android side (readyState=complete ~350ms; the web AXTree is built even earlier), while the Android mirror lands a second or two later. So a load-gated settle still fires the first dump before the mirror exists — and the reused UiAutomation still locks onto the empty state. The Android mirror just isn't visible over CDP; only the UiAutomation side sees it.

Two things that do work, both validated end-to-end (full Auth0 sign-in, back-to-back runs, on API 34 / Chrome 113 and API 36 / Chrome 133 and 152 — it's Chrome-version-independent):

1. A fixed settle (~5s) before the first dump. Crude but reliable — the first dump lands after the mirror is built, so there's no empty state to latch onto.
2. A reactive reconnect. When the field is still absent after a short poll, force a fresh UiAutomation and re-dump — the fresh connection mirrors the form correctly. It has to be a complete restart, though: killing the on-device server alone isn't enough, because mobilecli re-attaches to the lingering adb forward and reuses the same locked connection. Dropping the forward too makes the next call relaunch from scratch, and that recovers every time (5/5).

That second one points at what might be the cleanest upstream fix. Today serverRequest only rebuilds the DeviceServer on errAgentUnreachable — i.e. when the server is gone. In the lock case the server is alive and answering; it just returns a browser window whose renderer subtree is empty. If dumpUiNodes (or its caller) treated "browser window, empty renderer subtree" as a trigger to forgetDeviceServer() + full restart (remove forward + relaunch) + re-dump, it'd restore the pre-PR #334 self-heal, scoped to only when it's actually needed — no per-dump cost in the common case.