PcGöz

An MCP server that lets Claude Code see and use your Windows desktop.

▶ Watch the 4-minute demo with narration: nine scenes, from feature demos and token savings to real tasks in WhatsApp, the browser and a PDF app.

It reads the UI Automation tree of a window, takes screenshots with numbered boxes drawn over the elements, and clicks and types with real mouse and keyboard input. The numbers match: ref_7 in the tree is box 7 in the screenshot, so Claude can look at the picture and act on the element without guessing pixel coordinates. After each action it reports what changed on screen.

Türkçe · Unofficial, not affiliated with Anthropic.

Why PcGöz

• Numbered refs, not pixel guessing. Every element gets a ref_N, and the screenshot shows the same numbers in boxes. Claude points at an element by its number, so a click can't drift onto the wrong button.
• Text first, far fewer tokens. Claude gets the UI tree, the plain text or just the element it asked for instead of a screenshot. Measured on a sign-in form: about 40 tokens of text against about 1,600 for a full-screen image.
• A change report after every action. Each click or keystroke comes back with what changed on screen, so Claude rarely needs another screenshot to check. Fewer images, fewer tokens.
• Works without the mouse. pc_invoke presses, toggles, selects and expands through UI Automation: the cursor stays where it is, and it works while the window is covered (some apps bring themselves to the front when a button is invoked).
• Built for your real machine. Your own input is held back while an action runs, Ctrl+Alt+Shift+Esc freezes everything, password fields and UAC prompts are refused, and a click on an element that moved or closed is refused too.
• Apps without an accessibility tree. Canvas apps, Qt and games are read with the OCR built into Windows, locally.
• Knows when the screen is stale. A covered or minimized browser or Electron app stops repainting. PcGöz notices and says so instead of handing Claude old content.
• Private by design. No network connections, screenshots are never written to disk, typed text is never logged.

What you can ask for

• "Build and run my WinForms app and check that the main window opens with all its buttons."
• "Read the error dialog that's open on my screen and tell me what went wrong."
• "Open Settings > Display and tell me the recommended scaling."
• "Open Notepad, type this text and check that every character came through."

Works with Claude Code, either the CLI or the Code tab in the Claude desktop app.

Install

You need Windows 10 2004 or later (or Windows 11), Python 3.12+ and uv.

git clone https://github.com/emperorbug46/PcGoz
cd PcGoz
uv sync
claude mcp add pcgoz --scope user -- "$PWD\.venv\Scripts\python.exe" -m pcgoz

Restart Claude Code, then check the setup:

.\.venv\Scripts\python.exe -m pcgoz --doctor

The server is registered with the venv's python.exe rather than uv run. Claude Code starts MCP servers on demand, and if uv run stalls on a dependency check the tools silently never show up.

Optional: the masaustu-kullanimi skill tells Claude when to use PcGöz instead of shell commands and how to handle focus and navigation quirks.

New-Item -ItemType Junction -Path "$HOME\.claude\skills\masaustu-kullanimi" -Target "$PWD\skills\masaustu-kullanimi"

pc_ocr uses the OCR built into Windows, which needs the Optical character recognition feature for at least one installed language (Settings > Time & language > Language & region).

Safety

PcGöz drives your real mouse and keyboard and doesn't ask before each click. These are the guards instead:

• Emergency stop: Ctrl+Alt+Shift+Esc freezes every running PcGöz instance. Press it again to resume. If another app already holds that shortcut, Pause, F12 or Backspace with the same modifiers is used. If none of them can be registered, every action result carries a warning.
• Input lock: while an action runs, your own mouse and keyboard input is held back so you don't collide with it. Key releases still go through, so nothing gets stuck. A small notice in the top-left corner shows when the lock is on. It lets go after 30 seconds at most (60 for a batch). PCGOZ_BLOCK_INPUT=0 turns it off.
• Passwords: typing or pasting into password fields and Windows credential prompts is refused. Only navigation and delete keys (Tab, Enter, arrows, Backspace, ...) are allowed there.
• Secure desktop: no input is sent while a UAC prompt, the lock screen or Ctrl+Alt+Del is showing. The tool returns an error instead.
• Stale targets: if an element has closed, moved or been covered since it was read, the click is refused rather than landing on whatever is there now.
• One session at a time: several Claude sessions wait their turn for the mouse and keyboard instead of fighting over them.
• Dry run: with PCGOZ_DRY_RUN=1 the read-only tools work normally and the input tools only log what they would have done.

If you want to check this yourself before installing, ask Claude Code to review the source. It's also a good way to see exactly what each tool can and can't do.

Privacy

• PcGöz makes no network connections. Screenshots and OCR are processed locally with Windows APIs.
• Screenshots are never written to disk. They do reach Anthropic as part of your conversation, like any image you paste into Claude.
• Actions are logged to ~/.pcgoz/actions.jsonl, rotated at 5 MB. Set PCGOZ_LOG_DIR to move it. The log contains the keys sent with pc_do, but not the text typed: only its length and a short SHA-256 prefix, enough to spot the same text typed twice. Clipboard contents are never logged, only their length. Read-only tools log just the tool name, duration and result.

python -m pcgoz --stats summarizes the log: calls, error rate and latency per tool.

Tools

Tool	What it does	Sends input
pc_windows	Lists open windows, popup menus and monitors	no
pc_read	UI Automation tree of a window, with ref_N numbers	no
pc_find	Searches a window by name, value, role or automation ID, ignoring case and accents	no
pc_text	Plain text of a window or element	no
pc_screenshot	Screen, window or region. annotate=True draws the ref boxes. If the window is hidden behind another, it says which one	no
pc_ocr	Windows OCR, for apps that expose no accessibility info (Qt, canvas, games)	no
pc_wait_for	Waits until an element appears or disappears (up to 600 s; other tools keep working meanwhile)	no
pc_invoke	Invoke, toggle, select, expand or set a value through UI Automation, without moving the mouse	yes
pc_do	Click, right/double/middle click, hover, type, key, scroll, drag	yes
pc_focus	Brings a window to the front, also when Windows' focus lock or a covered title bar is in the way	yes
pc_window	Close, minimize, maximize, restore, move	yes
pc_clipboard	Reads or sets clipboard text	only set
pc_batch	Runs several calls in one go and stops at the first error	if any step does

Tool descriptions and messages are in Turkish (plain ASCII). Claude reads them without trouble and still answers you in your own language.

What's new in 0.6.2

• Stale content is flagged. A Chromium or Electron window (browsers, VS Code, Discord, ...) that is fully covered, minimized or on another virtual desktop stops repainting and stops updating its accessibility tree, so screenshots and text can be several seconds old. PcGöz now works this out the same way Chromium does and adds a [!] line to the result. It also warns when the display is off, the session is locked or a secure desktop is showing.
• pc_focus wakes such windows. A covered Chromium window that was already in the foreground used to stay stale for seconds; it now repaints right away.
• pc_wait_for no longer blocks other tools while it waits, and can wait up to 600 s.
• Sturdier focusing. If the title bar is covered, pc_focus minimizes and restores the window instead of giving up.
• Smaller fixes: one retry on transient UI Automation errors, screenshots say when the target is covered, waits inside a batch are cut to the input-lock time, and typed text is no longer written to the log.

Limitations

• Elevated windows: if an app runs as administrator and Claude Code doesn't, Windows blocks input to it (UIPI). Run Claude Code as administrator to automate something like Task Manager.
• Apps without accessibility info (Qt, canvas, games, remote desktop) don't show up in the tree. Use pc_screenshot and pc_ocr for them.
• Hung apps are probed with a timeout first, so a frozen window doesn't freeze the server.
• Covered Chromium/Electron windows return old content until they're brought to the front; such results carry a [!] line. Only Chromium-based windows are checked, not Firefox or WebView2 apps.

Troubleshooting

• Tools are missing after moving the folder: run --doctor. It checks whether ~/.claude.json still points at the right python.exe.
• Errors from the accessibility layer (comtypes): python -m pcgoz --repair regenerates the cached wrappers, then runs --doctor.
• Fallback mode: if the server can't import something, it still starts, offers the same tools and explains what broke. Without this, Claude would only show a bare CONNECTION_CLOSED.
• A result has a [!] line: the content may be old. Bring the window to the front with pc_focus; if the display is off, move the mouse (pc_do action="hover") to wake it.

Development

The tests are plain scripts, run with e.g. uv run python tests/birim.py. birim.py and mcp_istemci.py send no input and run in CI on Windows. The others (smoke.py, kapsam.py, tus_probu.py, ...) use the real mouse and keyboard, so run them when you're not using the machine. ortulu_chromium.py opens a temporary Brave window to check the stale-content warning, and ekran_kapali.py turns the display off.

Design notes and measurements, in Turkish: docs/GELISTIRICI-NOTLARI.md

License

MIT