Skip to content

Fix frame stepping, halve extraction time, add a frame viewer and thumbnails - #26

Merged
reloru merged 1 commit into
mainfrom
claude-code/optimistic-newton-semt99
Nov 16, 2026
Merged

reloru merged 1 commit into
mainfrom
claude-code/optimistic-newton-semt99

Conversation

@reloru

@reloru reloru commented Nov 16, 2026

Copy link
Copy Markdown
Owner

Bugs fixed

Frame stepping skipped and repeated frames. stepByFrames landed on the exact boundary between two frames, where rounding decides which one is shown. Measured in Chromium on a 29.97 fps clip with a burned-in frame counter, 40 presses of "next frame" showed 0,1,3,3,4,6,6,7,9,9,…. Steps now land on frame centres; the same test shows 1,2,3,…,40 forward and 39,38,…,25 back. This also applies to the ±6-frame buttons.

Batch extraction wasted 250 ms per frame. After each seek the extractor registered a requestVideoFrameCallback and waited for it. Chromium presents the frame before it dispatches seeked, so the callback never fired and every frame ran out the 250 ms fallback deadline. The callback is now registered before the seek, and a callback for a frame more than 0.5 s from the target is treated as belonging to an earlier seek. Measured on a 1080p clip, 300 frames in every-frame mode: 113.9 s before, 52.1 s after (380 → 174 ms per frame). All 300 captured frames hash distinct, so none is a stale repeat.

Added

  • Frame viewer. The arrows in a tile's corner open the frame full-screen at full size: page with the buttons, a swipe or the arrow keys, pinch to zoom, select, save (share sheet, else download), delete, or send the video back to the moment the frame came from. Tapping the rest of a tile still selects it.
  • Thumbnails. Each capture also produces a 512px JPEG for its tile (about 11 KB against 396 KB for the 1080p frame in the test). The full image is only decoded in the viewer.
  • Tiles are kept between renders. The gallery used to be rebuilt on every store change, including every selection tap. Tiles are now built once and updated or re-ordered in place.
  • Scrubbing keeps one seek in flight. Positions reported while a seek is decoding replace each other and the latest is taken when the decoder is free; releasing the thumb always seeks to the final position.
  • Batch runs survive a locked or backgrounded phone. A run holds a screen wake lock where the browser has one, waits while the page is hidden instead of timing out each frame, retries once a frame that failed while hidden, and pauses playback before it starts.
  • "Clear all" takes a second tap when more than one frame would be lost. The offer lapses after 4 s or on any gallery change.

Housekeeping

  • deploy.yml now runs lint, typecheck and tests before wrangler deploy.
  • CLAUDE.md added (checks, deploy, conventions); README features and deployment sections updated.

Verification

  • npm run lint, npm run typecheck, npm run build clean; npm run coverage: 421 tests pass (369 before), 98.5% lines / 90.1% branches.
  • Driven end to end in headless Chromium at iPhone 14 Pro Max size with a generated 20 s 29.97 fps clip: stepping, a 300-frame run, tile thumbnails at 512×288, tiles preserved across a selection, viewer paging / select / jump-to-moment, and the clear confirmation.

Not verified: anything on iOS Safari itself. In particular the size of the extraction speed-up depends on whether Safari also presents the frame before seeked; if it does not, the new code behaves as the old code did. The wake lock needs iOS 16.4+ and is refused in Low Power Mode, in which case the run proceeds without it.


Generated by Claude Code

…mbnails

Frame stepping landed on the boundary between two frames, so at rates like
29.97 a step repeated one frame and skipped the next. Steps now land on frame
centres.

Batch extraction waited for a presented-frame callback it registered after
the seek had finished; where the browser paints before dispatching `seeked`
that callback never fired and every frame cost the full 250 ms deadline. The
callback is now registered before the seek.

Also:
- frame viewer: full-size image, paging and swipe, select, save, delete, and
  jump back to the frame's moment in the video
- gallery tiles show a small thumbnail and are kept between renders instead of
  being rebuilt on every change
- scrubbing keeps one seek in flight and takes up the latest position
- batch runs hold a screen wake lock, wait while the page is in the
  background, and pause playback first
- "Clear all" needs a second tap when more than one frame is at stake
- the deploy workflow runs lint, typecheck and tests before deploying
- CLAUDE.md added; README updated

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DimiAfb92oKDyKCkHLqxTa
@reloru
reloru merged commit 1e1c57d into main Nov 16, 2026
3 checks passed
@reloru
reloru deleted the claude-code/optimistic-newton-semt99 branch November 16, 2026 09:16
@reloru reloru added the enhancement New feature or request label Nov 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants