Automated GUI tests launch Electron windows and repeatedly steal desktop focus #832

Activity

1. yxlyx commented on Sep 10, 2026

   

   yxlyx

   on Sep 10, 2026

   CollaboratorAuthor

   More actions

   @justrach Please take a look: automated GUI tests repeatedly steal desktop focus, preventing other work while they run.
2. cursor

   added a commit that references this issue on Sep 10, 2026

   fix: map BrowserView host windows so pin Escape reaches the page

   ...

   b279fcb
3. justrach

   closed this as completedin #859on Sep 11, 2026
4. yermakoffivan

   added a commit that references this issue on Sep 11, 2026

   Fix desktop clipboard, updates, snapshot sessions, and GUI test focus

   ...

   b393c8c
5. yxlyx

   reopened this on Oct 21, 2026
6. yxlyx commented on Oct 21, 2026

   

   yxlyx

   on Oct 21, 2026

   CollaboratorAuthor

   More actions

   Follow-up: native shell fallback still takes over the user's desktop

   Versions observed: graff 0.302.48, matching latest release v0.302.48 at investigation time; Harness 0.2.130. Surface: Harness desktop hosting graff over ACP. OS/architecture: macOS / arm64.

   Please reopen this broader background-GUI-testing failure. This is not evidence that the old Electron window policy stopped working: the new trajectory uses a generated Swift helper against a native macOS app, outside that policy. It demonstrates that the agent-level fallback path still defeats the intended background-testing experience.

   Observed trajectory

   1. The user requested verification of a native link-context-menu regression: both internal and external Open actions appeared to do nothing. The agent ran targeted pointer/menu tests but correctly acknowledged that these did not prove the exact live-app sequence.
   2. The computer-use bridge returned the explicit error Computer Use was not approved to use Harness. This was an app-approval failure, not an observed McpClosed transport failure. The agent subsequently acknowledged that it should not have assumed Harness exposed a permission control.
   3. The user explicitly requested native/recorder-style computer interaction. After checking recorder availability, the agent generated a Swift window probe and a Swift input helper, compiled the helper, and used per-window screencapture images to guide it. That request explains the native fallback; it does not make its input background-isolated.
   4. The helper explicitly called app.unhide() and app.activate(options: []) before ordinary interactions, slept, and required app.isActive. Its reveal action also activated the app. Mouse movement, scroll, and button events were posted to .cghidEventTap, the shared desktop input stream. Only the Escape-key branch used process-targeted posting, and even that branch ran after activation.
   5. The agent repeatedly used that helper to reveal the app, scroll the transcript, right-click, click, and change tabs. It reacquired the target process/window after an app restart. The workflow therefore operated on the user's installed, live app and mutable workspace rather than an isolated test instance.
   6. Eventually the agent reported that the pointer and active tab kept changing during its clicks and asked the user to leave the app and mouse untouched for two minutes. A foreground-control choice was then accepted. That later consent is relevant, but does not change the architecture: reliable verification had become dependent on the user surrendering their desktop, after native interaction had already begun.

   The initial native-automation trace segment contained 50 tool calls, including 28 shell calls, 6 edits, and 2 file writes, over approximately 15.5 minutes. This count excludes the later continuation and is not a claim that all of that time was spent injecting input. The continuation documents the eventual request for exclusive control. No raw transcripts, screenshots, session/run identifiers, local paths, process/window IDs, or user content are attached.

   Why background testing failed here

   This was not a shell-job scheduling problem. The generated helper itself made foreground activation a prerequisite and used global mouse events. Running that command asynchronously would still compete for the same pointer, focus, and live application state. Window-specific capture isolates the image source; it does not isolate input. Checking that the target app is active avoids some misdirected clicks, but is not a background-input guarantee and can race with user focus changes.

   The strongest confirmed cause is the helper's explicit activation/global-input design. The record does not establish that macOS or this native app intrinsically requires foreground control for every relevant check. Conversely, merely replacing global event posting with PID-targeted posting is not enough evidence that all GPUI mouse, hover, scrolling, and menu behavior works correctly while inactive; that needs validation. An isolated graphical test session is another possible route for checks that genuinely need foreground semantics.

   Release-source evidence and scope gap

   • v0.302.48:docs/adr/0101-gui-tests-preserve-desktop-focus.md:14-27 defines the existing Electron process policy: default hidden/non-focusable windows, failures on unexpected activation, and explicit foreground opt-in for OS-input checks. This protects those test entry points, not arbitrary shell-generated native helpers.
   • v0.302.48:docs/adr/0101-gui-tests-preserve-desktop-focus.md:43-50 explicitly distinguishes hidden-renderer coverage from native presentation/real-input coverage. Passing a synthetic menu test must not be presented as proving the reported native click sequence.
   • v0.302.48:docs/adr/0209-desktop-app-has-no-computer-use.md:17-28 removes the old desktop computer-use menu/endpoint and distinguishes the separate computer-use plugin bridge. Recovery instructions should not send the user looking for the removed permission UI.
   • The Swift helper described above was session-generated, not release source. Its observed activation and event-posting calls explain this occurrence; they do not identify a checked-in production helper defect.

   Requested fix

   1. Make background-safety a property of the verification workflow, not just the Electron window constructor or shell process. Carry the same expectation into native/recorder fallbacks and agent guidance.
   2. Prefer existing deterministic app-level input/routing tests and nonactivating window capture where they actually cover the claim. For live native sequences, provide a validated app-targeted driver/test instance or an isolated graphical session instead of defaulting to the user's live desktop.
   3. Keep permissions, focus isolation, and coverage separate. OS Accessibility trust and computer-use approval do not establish input isolation; screenshot success does not prove a click was delivered; a passing handler test does not prove the user's native sequence.
   4. If foreground control really is unavoidable, explain the precise unsupported check and obtain a bounded opt-in before any activation/global input. Stop when focus changes; do not repeatedly reactivate over the user. Preserve the real app's selected tab, split-pane state, and drafts. Consent is a last-resort fallback, not the background-testing solution.
   5. Report computer-use app-approval failures accurately, without suggesting removed controls or treating them as evidence that foreground native input is necessary.

   Acceptance criteria

   • Run the native link/menu verification while the user works in another app: their foreground app, mouse position, keyboard focus, and unrelated selected chat remain unchanged in the default mode.
   • User mouse movement or app switching does not invalidate supposedly isolated tests or cause the driver to raise the target app again.
   • Exercise the native fallback after the computer-use bridge reports an app-approval failure: no silent transition to foreground/global input and no invented permission setup.
   • Background-capable checks produce their own assertions and coverage evidence. Checks requiring real foreground semantics run in an isolated graphical session, or remain explicitly unverified unless the user opts into a bounded foreground diagnostic.
   • Observe actual OS focus and cursor state, not only mocked activation calls. Include cleanup and interruption behavior; do not close this solely because an Electron-only helper still passes.

   Related: #237 documents the earlier simulator/global-input distinction and process-targeted alternative; #916 concerns selected-session preservation during browser work. Neither is proof that arbitrary native app interaction is now background-safe. #1423 tracks a different bridge failure (McpClosed), which was not the error observed here.
7. yxlyx

   assigned

   justrach

   on Oct 22, 2026
8. justrach commented on Oct 22, 2026

   

   justrach

   on Oct 22, 2026

   Owner

   More actions

   Fixed by #1715 (merged, 7a3ace4). Agent tool commands on macOS now launch apps in the background (an open -g wrapper first on the tool PATH); GRAFF_GUI_FOREGROUND=1 on a command opts back in.
9. justrach

   closed this as completedon Oct 22, 2026
10. yermakoffivan

    added a commit that references this issue on Oct 22, 2026

    Launch apps from tool commands in the background (justrach#832)

    ...

    275860c
11. justrach

    reopened this on Oct 26, 2026
12. justrach commented on Oct 26, 2026

    

    justrach

    on Oct 26, 2026

    Owner

    More actions

    graff: v0.302.53 source; reproduced in an unreleased integration candidate (latest: v0.302.53)
    Harness: not used by this unit regression
    Surface: terminal unit test
    Platform: macOS arm64

    The background-launch shim can still be skipped for long paths after #1715. The existing regression reports unshimmed /usr/bin/open.

    Root cause: v0.302.53:src/gui_background.zig:76-77 formats a script containing the target path twice into a fixed 512-byte buffer and silently returns when formatting fails. The comparison buffer at line 82 has the same fixed limit.

    Sizing both buffers for the supported path length makes the same previously failing regression pass. The fix is being included in the next release candidate.
13. justrach commented on Oct 26, 2026

    

    justrach

    on Oct 26, 2026

    Owner

    More actions

    Fixed in v0.302.55: shim generation and comparison buffers now accommodate supported path lengths. The previously failing background-launch regression passes, and the release artifacts have been verified.
14. justrach

    closed this as completedon Oct 26, 2026
15. yxlyx

    added 2 commits that reference this issue on Nov 13, 2026

    Screenshot for #832

    2474c4a

    Screenshot for #832

    1641d6f
16. yxlyx commented on Nov 13, 2026

    

    yxlyx

    on Nov 13, 2026

    CollaboratorAuthor

    More actions

    Harness: 0.2.190 (latest: 0.2.190)
    graff: 0.302.74.65 (latest: 0.302.74.65), managed engine; the originating trace also records this version
    App / surface: Harness desktop, graff over ACP
    Platform: macOS 26.4.1 arm64

    Current recurrence: directly executed native fixtures bypass the background-launch shim

    Please reopen this canonical report. The latest resolution, v0.302.55, fixes the open shim's path buffers. This occurrence does not use open: the agent builds and directly executes a generated GPUI example, whose application explicitly calls cx.activate(true). A failing screenshot-capture callback then aborts that foreground process, producing a visible macOS unexpected-quit dialog during ordinary development verification. No foreground-testing opt-in was present in the observed generation/build commands.

    This is a workflow/agent-generated-helper failure, not evidence that the released Harness desktop app itself crashed, nor evidence that the open -g wrapper stopped working. Actual OS focus transitions were not separately instrumented. The confirmed desktop effects are a foreground-role native process and its unexpected-quit UI; the generated source independently establishes explicit activation.

    Evidence and trajectory

    The source checkout was public Harness v0.2.139 (cfdaa360f71017741f485ec8f8077fe8776fae41), even though the installed app and latest release were 0.2.190. That stale-baseline problem is distinct from this activation/capture failure.

    1. The agent read the existing sidebar and completion-fade fixtures, then generated a temporary sidebar capture example from the sidebar fixture. That fixture uses a temporary data directory and synthetic chats; reproducing this failure needs no account, real conversation, live engine request, or personal application state.

    2. The generated example retained cx.activate(true), opened a native window, populated synthetic tabs, waited, and synchronously called window.draw(cx) inside a typed WindowHandle<Shell>::update callback.

    3. The first execution returned 134. Its retained build log states:

       thread 'main' panicked at vendor/zui/crates/gpui/src/app/entity_map.rs:142:36:
       cannot update harness_ui::shell::Shell while it is already being updated
       fatal runtime error: failed to initiate panic, error 3, aborting
       Abort trap: 6
       

       The thread identifier and shell/process details above have been omitted; the error text is otherwise preserved.

    4. The agent immediately reran the same binary with RUST_BACKTRACE=1. It aborted again. Both independent macOS reports say EXC_CRASH (SIGABRT), termination SIGNAL / 6, abort() called, main thread. The process lifetimes were approximately 36.7 s and 2.1 s. These were two executions, not two views of one report.

    5. The backtrace agrees with the source-level chain below: typed root update → probe callback → Window::draw → root layout/render → double_lease_panic<Shell>.

    6. The agent then changed only the capture handle to gpui::AnyWindowHandle. The subsequent before-image capture and later after-image capture both returned 0, and both image artifacts exist. Therefore the observed helper failure was recovered within that trajectory; it is incorrect to say that every execution failed or that the agent simply finished by crashing.

    7. Two retained reports from an earlier native browser-fixture episode also show main-thread SIGABRT / 6 and double_lease_panic<Shell>. Their triggering chain is synthetic mouse dispatch → titlebar mouse listener → nested Shell update, inside a typed-window update, rather than the newer direct-draw chain. This establishes recurrence of root-entity reentrancy in generated verification helpers, not an identical trigger or a newly reproduced production browser bug. The earlier helper's exact release and source revision were not independently established, so those reports are corroboration only.

    Concrete cause of the current abort

    All references in this paragraph are to v0.2.139, the actual compiled source, not current Harness desktop code:

    • vendor/zui/crates/gpui/src/window.rs:6473-6487: WindowHandle<V>::update wraps the callback in view.update, keeping the root entity leased for the callback's full duration. Ignoring the first callback argument does not release it.
    • vendor/zui/crates/gpui/src/app.rs:2663-2675 and app/entity_map.rs:134-153: the entity is removed from the map until the update callback returns.
    • vendor/zui/crates/gpui/src/window.rs:2753-2800,2952-2988 and view.rs:154-167: drawing renders the root and calls view.update again, requiring the same lease.
    • vendor/zui/crates/gpui/src/app/entity_map.rs:207-212: the resulting literal panic is the one preserved above.
    • crates/ui/examples/footer-fixture.rs:101-103,120-125 and composer-completion-fade-fixture.rs:67-89: checked-in fixtures already demonstrate separating typed mutation from capture through AnyWindowHandle.

    Observed unsafe capture:

    // window: WindowHandle<Shell>; inside the fixture's async task
    window.update(cx, |_, window, cx| {
        window.draw(cx).clear();
        window.render_to_image().unwrap().save(output).unwrap();
    }).unwrap();

    Observed correction, after prior typed mutations have returned:

    let capture_window: gpui::AnyWindowHandle = window.into();
    capture_window.update(cx, |_, window, cx| {
        window.draw(cx).clear();
        window.render_to_image().unwrap().save(output).unwrap();
    }).unwrap();

    The handle correction fixes this double lease. It does not by itself fix explicit activation or isolate a native fixture from the user's desktop.

    Reproduction without access to the original machine

    For the observed historical helper mechanism, use a disposable checkout of justrach/harness at v0.2.139, on macOS with a graphical session. Copy crates/ui/examples/sidebar-fixture.rs to crates/ui/examples/native-capture-probe.rs, change let _window = cx.open_window to let window = cx.open_window, and append this immediately after cx.activate(true) inside the application callback:

    cx.spawn(async move |cx| {
        cx.background_executor().timer(std::time::Duration::from_millis(800)).await;
        window.update(cx, |shell, window, cx| {
            for _ in 0..20 { shell.fixture_chat_tab(None, window, cx); }
            shell.fixture_chat_tab(Some(0), window, cx);
        }).unwrap();
        cx.background_executor().timer(std::time::Duration::from_millis(800)).await;
        window.update(cx, |_, window, cx| {
            window.draw(cx).clear();
            window.render_to_image().unwrap().save("probe.png").unwrap();
        }).unwrap();
        cx.update(|cx| cx.quit());
    }).detach();

    Run from that disposable checkout:

    CARGO_PROFILE_DEV_DEBUG=0 CARGO_INCREMENTAL=0 \
      scripts/heavy-build.sh cargo run -p harness-ui \
      --example native-capture-probe \
      --features 'appshots-fixture project-palette-fixture'

    Compare the typed capture callback with the erased-handle correction above. This is a transcription of the observed generator and correction with a generic example/output name; it has not been rerun as a new standalone reproduction in this investigation. The original executions already provide the two aborts and two successful corrected controls. Running the unsafe variant may itself cause a crash dialog; use an isolated graphical session, not an unattended maintainer desktop.

    Why the latest mitigation misses it

    v0.302.74.65:src/gui_background.zig:3-16,27-38,55-68 installs an open wrapper on tool PATH. The file explicitly excludes apps that activate themselves after launch. A direct cargo run / executable launch followed by GPUI cx.activate(true) is exactly that excluded path. No adjustment to open's argument/path buffers can mediate this call.

    Requested fix / acceptance

    • Extend the existing background-safe verification contract to generated native examples and shell-launched GUI helpers, not only open/Electron entry points. Do not silently add activation, global input, or use the user's live data.
    • Prefer a reusable, source-version-appropriate fixture/capture helper rather than repeatedly synthesizing low-level callbacks. Finish typed root mutations before rendering or dispatching root callbacks; test both draw and synthetic-input reentrancy.
    • Add a regression/eval scenario that asks for synthetic native verification, checks the generated launch/activation path, and rejects implicit foreground activation without bounded opt-in. Keep coverage assertions separate from desktop-isolation assertions.
    • Preserve and surface abnormal termination as failed verification. A recovered probe should not be described as an application crash, and successful later screenshots should not erase the earlier side effect.
    • Verify the background-safety fix using real process/focus behavior in an isolated graphical session; a mocked open -g test alone does not cover self-activation.

    No raw crash reports, transcript text, local paths, user/host details, model/provider names, session/run identifiers, or private screenshots are published here. The images are rendered diagnostic summaries, not captures of the user's app.

    Images

    

    
17. yxlyx

    reopened this on Nov 13, 2026
18. justrach commented on Nov 14, 2026

    

    justrach

    on Nov 14, 2026

    Owner

    More actions

    Background-safe verification guidance and strict foreground opt-in are included in #1900 (not yet merged); CI is green. Live desktop focus, cursor and cleanup isolation remains unverified; this is not an OS sandbox.
19. justrach

    closed this as completedon Nov 14, 2026