Show AI control, temporarily block physical input, and allow user takeover

[zengfanfan]

Description

• While AI control is active and its indicator is confirmed visible, intercept physical keyboard and mouse events before they reach applications; AI-injected input continues. This reduces human and AI input being mixed into the same action. Ordinary keypresses are intercepted and do not by themselves trigger takeover.
• Show a soft, breathing blue screen-edge and cursor glow plus a top notice with the takeover shortcut while AI controls the desktop, so a person can tell why physical input is not reaching applications. Hide the indicator and release physical input during screenshot capture, then show a brief orange-red capture acknowledgement by default.
• Let the person take over by moving the physical mouse substantially or pressing Ctrl+Alt+Shift+Backspace. Physical input then passes through, new AI tool calls other than ControlStatus are blocked, and AI control can resume only after 10 seconds without physical activity and after held physical keys and mouse buttons are released.
• Expose read-only ControlStatus during takeover and add tests for input gating, state transitions, visual feedback, capture, and failure paths.

Motivation

• Human keystrokes and clicks can interleave with AI-generated input while both operate the same desktop, causing unintended text, clicks, or button state. Temporarily intercepting physical input during confirmed AI control reduces that confusion.
• A person needs to know when the AI is controlling the desktop, especially when their input is temporarily intercepted. The visible indicator makes ownership clear.
• A person must be able to regain control promptly; mouse movement or the takeover shortcut hands input back without waiting for the current automation sequence to finish.

Testing

• The previous upstream run on b15a549 had 854 passing tests and two unsolicited-notification tests timed out. 904d6af explicitly selects a legacy MCP session for those notification tests. The new upstream Windows / Python 3.14 run on d501a7d passed: 865 passed, 0 failed.
• Locally, Python 3.14.6 with locked FastMCP 4.0.10 passed 855 Git-tracked tests, including both formerly failing tests and the restored visual tests. Two Windows scheduled-task test files were excluded to avoid changing the user's startup setup and because one requires permissions unavailable in this environment. The 71 focused control/overlay tests and Ruff 0.16.9 lint/format checks also passed.
• Interactive smoke test on Windows 10 (build 19045), one display, before the visual restoration: blue indicator visible; two in-memory screenshots excluded the indicator and produced the orange-red acknowledgement; both hotkey and physical mouse takeover passed; indicator disappeared, normal input returned, and the test process exited cleanly. The previously accepted glow/notice implementation has since been restored with automated visual tests, but the combined branch's appearance has not been rechecked on a physical display.
• Multi-display behavior is covered by automated tests, but was not verified on physical multi-monitor hardware.

Limitations

Input interception and takeover are best-effort, not a security boundary or a guarantee that human and AI input can never mix. Physical input is not suppressed before the indicator is visible, while capture hides it, or after a detected hook/indicator failure. Cleanup of AI-held keys and mouse buttons after a detected failure is asynchronous, so a brief overlap with physical input remains possible. Windows can also silently remove a low-level input hook; periodic replacement reduces but does not eliminate that undetected window. Stepwise desktop input stops at a checkpoint, but an already-running external command or launched process is not cancelled and completed side effects are not undone. Modern sessionless MCP clients cannot receive unsolicited state logs and should query ControlStatus.

Screenshots

None attached: the earlier smoke test checked screenshots in memory to avoid retaining private desktop content. Its indicator and capture flash were confirmed on the test display; the restored glow and notice are covered by automated visual tests but have not had a new physical-display check on this combined branch.

Related Issues

None.

[Jeomon]

Can you also show a screenshot of it when the takeover happens

[zengfanfan]

Here are the indicators on a blank background for privacy. Blue shows the AI-control state; amber shows the brief takeover-pending state. The amber state was rendered for this capture without enabling input interception, so this demonstrates the UI rather than a live physical takeover. Once takeover completes, the indicator disappears.

ai control	takeover pending

[Jeomon]

Thanks again