This repository was archived by the owner on Nov 9, 2026. It is now read-only.
Repository navigation
Live screen viewer (iphonectl viewer) and WebView debugging over CDP - #1
Merged
Derrors merged 7 commits intoNov 7, 2026
Merged
Conversation
The supervisor now keeps two more services alive next to the WDA forward: - 127.0.0.1:9100: WebDriverAgent's MJPEG screen stream - 127.0.0.1:9222: pymobiledevice3's Chrome DevTools Protocol bridge for Safari tabs and inspectable WebViews Forwards are tracked per UDID and port, so restarting one no longer kills the others. Both ports bind to 127.0.0.1 only and can be turned off with MJPEG_PORT=0 / CDP_PORT=0. doctor reports whether the stream is up and how many pages are inspectable. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
WebDriverAgent runs a single session and opening a new one ends the current one. Reuse the session reported by /status, so the MCP server and other clients (such as iphonectl viewer) can work side by side. A stale session is still replaced once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Tk window on the host shows the MJPEG stream and turns input into touches: click/hold/drag/wheel for tap, long press, swipe and scroll, keyboard for typing, plus Home and Close app buttons for the system gestures iOS does not accept from outside. The window is freely resizable. Stream parsing, window-to-point mapping, gesture classification and session handling are unit tested. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
English README only. Includes the Web Inspector setting on the phone and the Capacitor setting needed when building without Xcode (e.g. with xtool). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TheSalaty
force-pushed
the
feat/live-viewer-and-webview-debugging
branch
from
November 6, 2026 23:10
0298bf1 to
11ce38d
Compare
Sessions: once a cached session went stale, the retry always opened a new one, which ended the other client's session, so after a WebDriverAgent restart the MCP server and the viewer kept ending each other's sessions. Both now join the active session on retry too, and open one only if there is none or the active one is the one that just went stale. Viewer: - Talk to WDA and the stream directly: urllib sends even 127.0.0.1 through HTTP_PROXY on the host. - Keep the action worker, frame reader and frame polling alive after an unexpected error. Before, input or the picture stopped without any notice. - A long status message no longer widens the window, which moved the screen under the pointer and made a click land on the wrong spot. - Explain a missing display instead of showing a traceback. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…vices The CDP bridge gives any process on the host access to the phone's Safari tabs, including logged-in sites, and has no authentication or origin check, so it is now opt-in (CDP_PORT=0) and the README warns about it. doctor checks that the MJPEG port answers with a multipart stream: the container uses host networking and 9100 is a common port, so a plain HTTP 200 from another service used to count as a working live view. README: how to run the viewer on a desktop when the phone is on a headless host. Also fixes the width of a line in the architecture diagram. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
README.zh-CN.md gets the features, the "实时画面" and "调试 WebView" sections, the viewer command and the system gesture limitation from the English README. The detailed docs now cover the MJPEG and CDP ports (security), the shared WebDriverAgent session and per-port forwards (architecture), common viewer and CDP problems (troubleshooting) and src/iphone_viewer/ (development). .env.example comments are in Chinese like the rest of the file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two additions that make the toolkit nicer for people watching or testing alongside the agent, plus a small fix that lets them coexist with the MCP server.
1. Live viewer:
iphonectl viewerA desktop window (Python + Tk + Pillow, runs on the host) that shows the phone screen live from WebDriverAgent's MJPEG server and forwards input:
The window is freely resizable. Gets ~24 fps over USB at 50 % resolution (configurable:
--fps,--scale,--quality). iOS ignores system gestures such as the swipe up from the bottom edge when they are injected (tested over both WDA and CoreDevice HID), so the window offers buttons for them instead.2. WebView debugging over CDP
The supervisor now also runs
pymobiledevice3 webinspector cdpon127.0.0.1:9222. Safari tabs and inspectable WebViews (Capacitor, Cordova, anyWKWebViewwithisInspectable) become available over the Chrome DevTools Protocol. For hybrid apps this is much faster than the element tree: evaluating JavaScript in the page took 3–20 ms here, vs ~1.5 s for/source. Documented: the Web Inspector setting on the phone, and the Capacitor setting needed when building without Xcode (Capacitor's debug detection relies on aCAPACITOR_DEBUGInfo.plist key that only Xcode fills in).3. Supervisor and doctor
ensure_forward), so WDA (8100) and MJPEG (9100) restart independently. Before,pkill -f 'usbmux forward'would have taken down every forward..envoptionsMJPEG_PORT(default 9100) andCDP_PORT(default 9222);0turns either off. Both bind to127.0.0.1only, like WDA.doctorgets a section reporting the stream and the number of inspectable pages (warnings only, both are optional).4. MCP: join the active WDA session
WebDriverAgent has a single session, and
POST /sessionends the current one.wda.session()now reuses the session from/statusand only creates one if there is none, or once to replace a stale one. Without this, the viewer and the MCP server would keep ending each other's sessions.Design notes
src/iphone_viewer/is split into pure, tested modules (mjpegstream parser,geometryfor window-to-point mapping and gesture classification,wdaclient) and the Tk UI inapp.py. It is excluded from the container'siphone-mcppackage (include = ["iphone_mcp*"]).iphonectl viewerruns it withPYTHONPATH=src, so there is nothing to install besides Tk and Pillow.README.zh-CN.mdand the Chinese docs untouched, so a Chinese translation of the new README sections is still open if you want one.Testing
ruff check src tests scripts,pytest(53 passed, 18 new), andshellcheck -x container/*.sh bin/iphonectl scripts/*.shall pass.container/scripts andiphone_mcpmounted in.doctorreports the new checks.iphonectl viewerstreams and controls the phone.imagejob covers that.🤖 Generated with Claude Code