Skip to content

Add stateful pointer input - #340

Open
zengfanfan wants to merge 3 commits into
CursorTouch:mainfrom
zengfanfan:codex/stateful-pointer-input
Open

zengfanfan wants to merge 3 commits into
CursorTouch:mainfrom
zengfanfan:codex/stateful-pointer-input

Conversation

@zengfanfan

@zengfanfan zengfanfan commented Jul 18, 2026 •

Copy link
Copy Markdown
Contributor

Description

Add one consolidated Pointer tool for mouse-button gestures that span multiple MCP calls:

Pointer(action="down", loc=[100, 100], button="left")
Pointer(action="move", loc=[140, 120], duration=0.2)
Pointer(action="move", loc=[180, 160])
Pointer(action="up")

The tool supports down, move, up, and cancel. It tracks at most one held button and automatically attempts release after 30 seconds by default (maximum 120 seconds). cancel releases only the currently tracked button, if any. The existing Click and Move interfaces remain unchanged.

This branch is based on current main, including #445. Pointer holds are recorded in its input ledger, so a physical user takeover can release the held button. The 15-second AI-control lease does not discard a valid Pointer hold; other desktop tools cannot borrow that held input. Each gesture has a unique receipt so a delayed timer or stale call cannot release a later gesture. Timed moves recheck control ownership between steps, and timeout/release failures retain auditable ownership or fail open for physical input. Server shutdown attempts release before stopping desktop control.

Motivation

Move(drag=True) handles an atomic drag in one call. Free-form drawing and non-linear selections may require several independently chosen segments while a button remains held; those workflows need explicit cross-call pointer state.

Testing

  • ruff check and ruff format --check passed on all 12 changed Python files.
  • Independent focused Pointer/control review run: 51 passed; the cross-call integration test file separately passed 14 tests.
  • Local suite with two environment-dependent test files excluded: 923 passed. The excluded files are tests/test_analytics.py (cannot create a directory inside this environment's read-only worktree) and tests/test_cli_legacy_flags.py (requires Windows scheduled-task registration privileges). This is not an unfiltered full-suite result.
  • No real held-mouse-button smoke test was run; the focused tests use mocked UIA input and deterministic timing.
  • Local interpreter: Python 3.13; pyproject.toml requires Python >=3.14. The GitHub CI run for this PR head passed on Python 3.14: 937 passed, 5 warnings.

Screenshots

Not applicable; no UI is added.

Related Issues

None.

@qodo-code-review

Copy link
Copy Markdown

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

@zengfanfan
zengfanfan force-pushed the codex/stateful-pointer-input branch from 22e1e6b to d477b25 Compare October 2, 2026 17:00
@zengfanfan
zengfanfan force-pushed the codex/stateful-pointer-input branch from d477b25 to 281a867 Compare October 3, 2026 09:45

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant