Repository navigation
Magic pen: Option-drag over a board quotes that region in the chat - #198
Conversation
…n the chat Held over the canvas, Option makes the pointer a pen. A drag across a board marks a region, and that part of the board goes to the chat as a picture, captioned with the board, where on it, and the elements there. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Deploying super-prototyping with
|
| Latest commit: |
db27fa2
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://57325c3e.super-prototyping.pages.dev |
| Branch Preview URL: | https://magic-pen.super-prototyping.pages.dev |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… not the last view Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # canvas/src/ChatPanel.tsx # canvas/src/canvasAttach.tsx # canvas/src/canvasChrome.tsx # canvas/src/canvasClicks.ts
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| const pageAt = (e: React.PointerEvent) => | ||
| editor.screenToPage({ x: e.clientX, y: e.clientY }); |
There was a problem hiding this comment.
🔴 Board quotes miss the pointer
When the canvas sits beside the chat panel, pageAt converts window coordinates as editor-local screen coordinates. The resulting hit misses the board or crops a region displaced from the drag.
Learn more
The pen receives browser clientX and clientY, which are relative to the window. Tldraw screen coordinates are relative to the editor viewport; CanvasAttachButtons adds the viewport's screen origin to its local pointer coordinates before calling document.elementFromPoint. Here the reverse conversion is missing, so the hit test and drag points do not follow the pointer when the editor has an offset. The offset changes when the chat panel or canvas strip is present.
Example: If the canvas begins at window x=360 and a board starts at its local x=20, a click at window x=380 is converted as though it occurred at local x=380 rather than x=20. The pen can dismiss itself even though the click is on the board.
Recommended fix: Subtract editor.getViewportScreenBounds().x and .y from the event's client coordinates before calling editor.screenToPage, or use the editor's appropriate window-to-page conversion. Verify both board hit testing and crop positions with the chat panel open and closed.
Was this helpful? React with 👍 or 👎 to provide feedback.
| const part = await createImageBitmap( | ||
| whole, | ||
| Math.round(crop.x * w), | ||
| Math.round(crop.y * h), | ||
| Math.round(crop.w * w), | ||
| Math.round(crop.h * h), |
There was a problem hiding this comment.
🟡 Small quotes fail on downscaled boards
On a board larger than the 4000-pixel shoot limit, a four-pixel region can round to zero image pixels. createImageBitmap rejects that crop, leaving the quote failed in chat.
Learn more
The pen accepts a region of at least four board pixels at finish. boardShot reduces boards larger than 4000 pixels per side before asking the server to render them. Multiplying the fractional crop width by this smaller PNG width can yield less than half a pixel, which rounds to zero here; image bitmap crop dimensions must be positive.
Example: An 8000-pixel-wide board is rendered at 4000 pixels. A selected four-pixel-wide region becomes two image pixels, but on a 40000-pixel-wide board it becomes 0.4 and rounds to zero, so the quote fails.
Recommended fix: Clamp crop width and height to at least one source pixel, and clamp crop origin and extents to the image bounds. Apply the minimum to the computed image-pixel dimensions, not the board-pixel threshold.
Was this helpful? React with 👍 or 👎 to provide feedback.
| onPointerUp={() => { | ||
| if (!drag) return; | ||
| setDrag(undefined); | ||
| setArmed(false); | ||
| void finish(drag); | ||
| }} |
There was a problem hiding this comment.
🟡 Cancelled drag leaves pen active
If the browser cancels a pen drag, drag stays set because only onPointerUp clears it. The overlay keeps intercepting canvas gestures until another interaction resets the pen.
Learn more
A browser emits pointercancel without a subsequent pointerup when a gesture is interrupted. The pen has captured that pointer, and the state holding its board and endpoints stays set after cancellation. Since the overlay is rendered while either armed or drag is set, it keeps covering the canvas.
Example: A touch drag begins over a board, then the browser takes over the gesture and cancels the pointer. The region remains drawn and canvas gestures still land on the pen layer.
Recommended fix: Add onPointerCancel that clears drag and disarms the pen without calling finish. Test interrupted gestures as well as normal pointer release.
Was this helpful? React with 👍 or 👎 to provide feedback.
| /** | ||
| * The magic pen. A tap of Option makes the pointer over the canvas a pen, and a drag across a | ||
| * board marks a region of it, which goes to the chat as a quote: that part of the board as a | ||
| * picture, captioned with the board, where on it, and the elements there. One region, and the | ||
| * pointer is itself again; Escape or another tap puts the pen down without one. | ||
| * | ||
| * A tap is Option pressed and let go with nothing between, so Option-drag still copies a board | ||
| * and Option with a key is still that key. The key is heard in both windows, since the focus may | ||
| * be in the chat. A layer over the whole canvas while the pen is up, so tldraw never sees the drag. | ||
| */ |
There was a problem hiding this comment.
🔍 Add release notes for new controls
The pen and Colour menu are user-visible additions, but ## Unreleased has no entry for them. Contributing rules require one in this PR.
Was this helpful? React with 👍 or 👎 to provide feedback.
| * own at the board's size, since the board's frame on the canvas is sandboxed and cannot be read. | ||
| * Scripts stay off there as they do on the canvas, so the layout is the one the person drew over. | ||
| */ | ||
| async function describeRegion(html: string, w: number, h: number, r: Region) { |
There was a problem hiding this comment.
🔍 One-use region helper adds indirection
describeRegion has one caller, finish. Review rules ask that one-use helpers be inlined; assess whether this needs a separate function.
Was this helpful? React with 👍 or 👎 to provide feedback.
| * and Option with a key is still that key. The key is heard in both windows, since the focus may | ||
| * be in the chat. A layer over the whole canvas while the pen is up, so tldraw never sees the drag. | ||
| */ | ||
| export function MagicPen() { |
There was a problem hiding this comment.
🔍 New pen paths lack module tests
The pen and crop paths have no adjacent tests. Contributing rules request a test next to each touched viewer module.
Was this helpful? React with 👍 or 👎 to provide feedback.
1 participant
Stacked on #197 (jump-to).
Tap Option (press and release, nothing else between) and the pointer becomes a pen. Drag across a board to mark a region; that part of the board goes to the chat as a quote, and the pointer is itself again. Escape, a second tap, or a click off a board puts the pen down without one. Option-drag still copies a board and Option with another key is still that shortcut, since only a lone tap counts.
/__sp/shoot). The crop is sent as fractions of the board, because the drawing is at the display's pixel ratio.paint-tasks-flow/04f-note-templates.html at 927,665 130x43: span "取消", span "保存". It names the board, the region in board pixels, and the outermost elements wholly inside it (or else the element that holds it). These are found by laying the board's HTML out offscreen at its own size, since the canvas's frame is sandboxed. Capped at the server's 200 characters.While the pen is up, a layer over the canvas takes the drag, so tldraw never sees it. The drag is held to the board it started on.
Only on a local canvas (
local()), as the other chat controls are.Tested on an isolated dev server in a browser: the cursor, the dashed region, the tile in the chat, and the crop content (2× crop of the 取消/保存 buttons).
tsc -b, oxlint andbun run test(189 passed) are clean.Also here, at the user's request: a second double-click on the board a double-click zoomed to now zooms out to the whole page (
zoomToFit), instead of putting back the view from before the first.And a Colour submenu in the canvas's right-click menu, beside Background: tldraw's palette, each with its dot as the current theme draws it, for any selected shape that takes a colour and is not locked (an arrow, a line, a note). It goes through
setStyleForSelectedShapes, which leaves a locked board in the same selection alone, and is saved tocanvas.jsonlike any other edit.🤖 Generated with Claude Code