VS Code + Colab frontends, with kernel-side scene replay (#281)

[sspickle]

What this is

The kernel half of VPython support for VS Code notebooks and Google Colab (#281), plus the scene-replay architecture that makes both reliable. With this branch, VPython works in three hosts:

Host	Transport	Module
Classic Jupyter	Comm down / websocket up (unchanged)	with_notebook.py
VS Code	whole protocol over the kernel's tornado websocket	with_wsfrontend.py
Google Colab	whole protocol over Jupyter comms	with_colab.py

Status: working end-to-end in both new hosts. Scenes render live — objects, animation via rate(), zoom/orbit interaction, textures; text() with fonts in VS Code — verified on macOS with VS Code 1.133 and in real Colab (including full-brightness scene replay across output-frame re-renders).

VS Code: websocket-only mode

Under VS Code (auto-detected via VSCODE_PID/VSCODE_CWD, overridable with VPYTHON_FRONTEND=ws|jupyter), vpython skips the classic Comm + nbextension machinery and speaks the entire wire protocol over the tornado websocket it already runs, announcing the port with a custom-MIME display output:

application/vnd.vpython.v1+json   {"api": 1, "port": N, "wsuri": "/ws"}

The companion renderer extension — https://github.com/vpython/vscode-vpython — picks that up, loads GlowScript + glowcomm_host.js in the notebook output webview, connects, and drives the standard trigger ping-pong.

Why websocket-only: VS Code's Jupyter extension deprecated third-party access to raw kernel messaging (the ipywidgets comm mechanism), so the classic Comm downlink is unreachable from a renderer extension — but a webview can open a WebSocket to the kernel's tornado server directly (both directions verified in a spike). Moving the downlink onto the websocket the kernel already runs makes the whole protocol renderer-reachable with no Jupyter plumbing at all.

Colab: comm-only mode

Colab is the mirror image: output frames are sandboxed cross-origin iframes that cannot reach the kernel's tornado port (the port-proxy authenticates with Google cookies that don't flow there), but Colab shims Jupyter comms (google.colab.kernel.comms). So with_colab.py runs the whole protocol over a comm instead.

Colab's platform quirks shaped the design (each was found empirically):

• Browser-initiated handshake: the kernel registers a passive comm target at import; the JS calls comms.open when its frame has actually loaded. (Kernel-initiated opens race Colab's async iframe lifecycle; kernel-side retry loops make Colab drop display outputs.) A per-session nonce freezes out zombie frames from saved outputs.
• No import-time display: Colab always drops display output emitted during import, so the scene box is shown by an explicit wc.show() call.
• CDN bootstrap: large inline <script>/script src= in display HTML are silently neutralized; a tiny inline injector loads glowcomm_colab.js (and GlowScript) from jsDelivr instead.
• Mid-cell browser replies are impossible (replies only arrive between cells), so compound/text/extrusion/scene.pause/waitfor/pick raise a clear NotImplementedError rather than deadlocking; rate() self-clocks its flushes.

Demo notebook: vpython-colab-demo.ipynb.

Scene replay — the reliability architecture

Both new hosts have ephemeral frontends: Colab re-renders output frames on scroll, VS Code can evict webviews, pages reload. The kernel is the only durable holder of the scene, so it now journals it (_scene_journal.py — every constructor cmd and every attribute ever touched, in emission order) and every frontend attach replays: a wire reset cmd (added to glowcomm_host.js), then every cmd in its original order, then the current value of every dirty attribute. Both senders (CommSender, WsSender) share the same replay path (_frontend_replay.py).

Emission order matters — canvas construction emits its constructor, then a lights='empty_list' wipe of glow's built-in defaults, then the two standard distant_lights; replaying out of order deletes the lights and dims the scene (found and fixed during live Colab testing).

Changes

• vpython/with_wsfrontend.py, vpython/_wssender.py (new): websocket-only bootstrap + kernel→browser sender — buffers packages until a renderer connects, then write-through, marshaling writes onto the tornado thread; replays on reconnect.
• vpython/with_colab.py, vpython/_commsender.py (new): comm-only bootstrap (passive target, nonce, show(), self-heal) + comm sender with the same buffer/flush/replay contract.
• vpython/_scene_journal.py, vpython/_frontend_replay.py (new): pure scene journal + replay builder.
• vpython/vpython_libraries/glowcomm_host.js, glowcomm_colab.js (new): host-agnostic frontend runtime (from Pyodide/wasm support: pure-Python wheel, worker transport, GlowScript host factory #291's factoring, plus the reset cmd) and the Colab bootstrap.
• vpython/_notebook_helpers.py: three-way frontend detection (_use_ws_frontend(), _is_colab()).
• vpython/vpython.py: frontend selection, GlowWidget(sender_override=...), journal hooks in appendcmd/addattr/delete, classic per-canvas HTML suppressed when a frontend owns the container.
• vpython/rate_control.py: direct-trigger flush for hosts whose browser triggers can't reach a blocked kernel.
• Tests: test_scene_journal.py, test_colabfrontend.py, test_wsfrontend.py — suite: 38 passed.

Not in scope yet

Remote/WSL kernels (needs VS Code port forwarding), one scene container per notebook, widgets/pause/waitfor in ws mode (same deferrals as glowcomm_host), a kernel pump to lift Colab's mid-cell-reply limits.

Draft while the extension hardens — review welcome. Companion PRs: #292 (merged — the loud import timeout these modes gracefully degrade to), #291 (packaging; glowcomm_host.js originates there).

🤖 Generated with Claude Code

https://claude.ai/code/session_01G7y9rTA1r8r8EhEnPSQenR

[sspickle]

Folded the Colab work into this PR rather than stacking a second one: the scene-replay architecture (journal + replay-on-attach) spans both new frontends — it changes _wssender.py/with_wsfrontend.py too — so the websocket-only half wasn't reviewable in isolation anyway. The PR now covers both hosts + replay; body rewritten accordingly.

Colab is verified working end-to-end in a real notebook (browser-initiated comm handshake, session-nonce zombie-frame protection, scene replay at full brightness across output-frame re-renders). Demo: https://colab.research.google.com/github/vpython/vscode-vpython/blob/main/colab/vpython-colab-demo.ipynb

[sspickle]

GitHub Codespaces now works too — desktop AND browser VS Code, zero config. The companion extension (vpython/vscode-vpython v0.1.7) gained an extension-host side that relays the wire protocol between the output webview and the kernel's tornado websocket over VS Code renderer messaging — the host dials 127.0.0.1 on the kernel's own machine, so no port forwarding, tunnel auth, or port-visibility changes are needed. Same mechanism covers Remote-SSH/WSL. Kernel-side additions on this branch: VPYTHON_WS_PORT (pin the websocket port) and the existing VPYTHON_CONNECT_TIMEOUT. Verified end-to-end in a real codespace from a browser-only session: open notebook → Run All → live scene (textures, animation, zoom/orbit). With classic Jupyter, VS Code, Colab, and Codespaces all working, this branch now delivers VPython on every major notebook host, including fully browser-only zero-install environments (Chromebooks).