feat(preview): generate video previews on an ffmpeg-over-ip server by szaimen · Pull Request #65717 · nextcloud/server

szaimen · 2026-10-20T09:49:45Z

Summary

Add the OC\Preview\MovieRemote provider which runs ffprobe and ffmpeg on an ffmpeg-over-ip server (https://github.com/steelbrain/ffmpeg-over-ip), similar to the Imaginary provider for images. It is configured with preview_ffmpeg_url and preview_ffmpeg_key; neither ffmpeg nor the ffmpeg-over-ip client is needed on the Nextcloud server.

OC\Preview\FFmpegOverIpClient implements the client side of the protocol in PHP. The server tunnels all file access of the remote process back over the connection, so the video is streamed from any storage, including S3 and external storages. Only the video itself can be opened, read-only. Streams that cannot seek are copied to a temporary stream first.

Documentation at docs(admin): document external ffmpeg service for video previews documentation#15859

Checklist

Code is properly formatted
Sign-off message is added to all commits
Tests (unit, integration, api and/or acceptance) are included
Screenshots before/after for front-end changes
Documentation (manuals or wiki) has been updated or is not required docs(admin): document external ffmpeg service for video previews documentation#15859
Backports requested where applicable (ex: critical bugfixes)
Labels added where applicable (ex: bug/enhancement, 3. to review, feature component)
Milestone added for target branch/version (ex: 32.x for stable32)

AI (if applicable)

The content of this PR was partly or fully generated using AI

Ship the ffmpeg preview service from nextcloud/server#65717 inside the imaginary container and configure the OC\Preview\MovieRemote provider automatically when imaginary is enabled. The service script gets synced from server via the imaginary-update workflow. Uploaded videos are buffered in a dedicated volume instead of a tmpfs to not use up RAM. ffmpeg is removed from the Nextcloud container as video previews are now generated by the imaginary container. Assisted-by: ClaudeCode:claude-opus-5-5 Signed-off-by: Simon L. <szaimen@e.mail.de>

Relates to nextcloud/server#65717 and nextcloud/all-in-one#8865 Assisted-by: Claude Code:claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Simon L. <szaimen@e.mail.de>

Relates to nextcloud/server#65717 and nextcloud/all-in-one#8865 Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Simon L. <szaimen@e.mail.de> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Simon L. <szaimen@e.mail.de>

susnux

I do not think docker containers for deployment belong into the server repository.
And especially not into the build folder.

This would belong to either some dedicated ffmpeg preview app or at least the container should be handled somewhere else like docker-ci or even a new repository.

SystemKeeper · 2026-10-26T06:45:26Z

I am also a bit twisted on having this in the server repository tbh. But what I am wondering, isn’t there something prebuilt like this that we could use?

susnux · 2026-11-02T14:50:58Z

isn’t there something prebuilt like this that we could use?

maybe https://github.com/steelbrain/ffmpeg-over-ip

szaimen

WIll try to make the implementation compatible with https://github.com/steelbrain/ffmpeg-over-ip going forward :)

Add the OC\Preview\MovieRemote provider which runs ffprobe and ffmpeg on an ffmpeg-over-ip server (https://github.com/steelbrain/ffmpeg-over-ip), similar to the Imaginary provider for images. It is configured with preview_ffmpeg_url and preview_ffmpeg_key; neither ffmpeg nor the ffmpeg-over-ip client is needed on the Nextcloud server. OC\Preview\FFmpegOverIpClient implements the client side of the protocol in PHP. The server tunnels all file access of the remote process back over the connection, so the video is streamed from any storage, including S3 and external storages. Only the video itself can be opened, read-only. Streams that cannot seek are copied to a temporary stream first. Assisted-by: ClaudeCode:claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Simon L. <szaimen@e.mail.de>

Add the nextcloud-aio-ffmpeg container which runs the ffmpeg-over-ip server (https://github.com/steelbrain/ffmpeg-over-ip) and configure the OC\Preview\MovieRemote provider from nextcloud/server#65717 automatically when it is enabled. The container is enabled by default and can be disabled in the AIO interface. The ffmpeg-over-ip release gets pinned by its checksums and is kept up-to-date within the major version by the ffmpeg-over-ip-update workflow. ffmpeg stays in the Nextcloud container so that video previews keep working when the container is disabled. Assisted-by: ClaudeCode:claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Simon L. <szaimen@e.mail.de>

Relates to nextcloud/server#65717 and nextcloud/all-in-one#8865 Assisted-by: Claude Code:claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Simon L. <szaimen@e.mail.de>

szaimen · 2026-11-03T15:43:18Z

isn’t there something prebuilt like this that we could use?

maybe https://github.com/steelbrain/ffmpeg-over-ip

done :)

susnux

Better now :)

- Only download the first 10 MB of videos on storages that cannot seek, using ProviderV2::getLocalFile() like the Movie provider, and send the real size of that file - Allow tls:// addresses with peer verification and reject http(s) urls with a clear error - Apply one configurable deadline (preview_ffmpeg_timeout) to all runs of a preview instead of 120 seconds per run - Fall back to non-HDR previews if ffprobe fails - Reject seeks that would overflow the file offset - Report the provider as unavailable without preview_ffmpeg_address - Rename preview_ffmpeg_url to preview_ffmpeg_address, as it is no url - Share the timestamps and ffmpeg arguments with the Movie provider Assisted-by: ClaudeCode:claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Simon L. <szaimen@e.mail.de>

szaimen · 2026-11-04T13:58:01Z

Better now :)

Addressed all your comments. Please check again :)

Assisted-by: ClaudeCode:claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Simon L. <szaimen@e.mail.de>

susnux

Some findings from a deeper review pass, roughly ordered by severity. Most of them were reproduced against a fake ffmpeg-over-ip server, the real SeekableHttpStream/Icewind wrappers and real ffmpeg 8.1.3.

_{AI-assisted review (Claude Code), findings checked by reproduction.}

szaimen · 2026-11-05T08:31:50Z

Some findings from a deeper review pass, roughly ordered by severity. Most of them were reproduced against a fake ffmpeg-over-ip server, the real SeekableHttpStream/Icewind wrappers and real ffmpeg 8.1.3.

Should be adressed now. Please check again :)

…client - Open the video on the storage, so it is not locked while the server works - Take the size from the opened stream instead of the filecache - Limit the data the server may read of videos on non-local storages to 20 MB, like the Movie provider - Track the stream position after failed seeks and report short reads as I/O errors instead of end of file - Send a cancel message when a run is aborted - Enforce the deadline while writing with non-blocking writes - Catch all errors and give up after the first failed connection - Report the reason of TLS errors, verify IPv6 literals and document the certificate bundle that is used - Register MovieRemote before Movie and log if no address is configured - Use the default timeout for values below 1 Assisted-by: ClaudeCode:claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Simon L. <szaimen@e.mail.de>

Follow nextcloud/server#65717: rename preview_ffmpeg_url to preview_ffmpeg_address, document tls:// and unix: addresses, the preview_ffmpeg_timeout option and the read limits on non-local storages, and that MovieRemote is tried before Movie when both are enabled. Assisted-by: Claude Code:claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Simon L. <szaimen@e.mail.de>

szaimen · 2026-11-05T09:55:18Z

@CarlSchwan would you mind having another look at this? :)

szaimen added this to the Nextcloud 36 milestone Oct 20, 2026

szaimen added enhancement 2. developing Work in progress labels Oct 20, 2026

nextcloud-command added the AI assisted label Oct 20, 2026

szaimen mentioned this pull request Oct 20, 2026

docs(admin): document external ffmpeg service for video previews nextcloud/documentation#15859

Draft

4 tasks done

szaimen added 3. to review Waiting for reviews and removed 2. developing Work in progress labels Oct 20, 2026

szaimen marked this pull request as ready for review October 20, 2026 10:23

szaimen requested a review from a team as a code owner October 20, 2026 10:23

szaimen requested review from Altahrim, CarlSchwan, SystemKeeper, come-nc, icewind1991, provokateurin, sorbaugh and susnux and removed request for a team October 20, 2026 10:23

szaimen mentioned this pull request Oct 20, 2026

feat(FFmpeg): add an optional ffmpeg container for video previews nextcloud/all-in-one#8865

Draft

6 of 9 tasks

CarlSchwan reviewed Oct 20, 2026

View reviewed changes

Comment thread build/ffmpeg-preview-service/server.py Resolved

Comment thread build/ffmpeg-preview-service/Dockerfile Resolved

susnux requested changes Oct 23, 2026

View reviewed changes

szaimen added 2. developing Work in progress and removed 3. to review Waiting for reviews labels Nov 2, 2026

szaimen commented Nov 2, 2026

View reviewed changes

szaimen marked this pull request as draft November 2, 2026 14:56

szaimen force-pushed the feat/ffmpeg-preview-service branch from 2ba7e52 to 6ca6cbe Compare November 3, 2026 15:14

szaimen changed the title ~~feat(preview): generate video previews with an external ffmpeg service~~ feat(preview): generate video previews on an ffmpeg-over-ip server Nov 3, 2026

szaimen added 3. to review Waiting for reviews and removed 2. developing Work in progress labels Nov 3, 2026

szaimen marked this pull request as ready for review November 3, 2026 15:43

susnux requested changes Nov 3, 2026

View reviewed changes

szaimen requested a review from susnux November 4, 2026 13:59

fix(preview): fix psalm and coding style issues in FFmpegOverIpClient

1b4593c

Assisted-by: ClaudeCode:claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Simon L. <szaimen@e.mail.de>

susnux reviewed Nov 4, 2026

View reviewed changes

szaimen requested a review from susnux November 5, 2026 08:31

susnux approved these changes Nov 5, 2026

View reviewed changes

szaimen force-pushed the feat/ffmpeg-preview-service branch from 8a9086e to d18936a Compare November 5, 2026 09:19

This comment was marked as resolved.

Sign in to view

Conversation

szaimen commented Oct 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Checklist

AI (if applicable)

Uh oh!

Uh oh!

Uh oh!

susnux left a comment

Choose a reason for hiding this comment

Uh oh!

SystemKeeper commented Oct 26, 2026

Uh oh!

susnux commented Nov 2, 2026

Uh oh!

szaimen left a comment

Choose a reason for hiding this comment

Uh oh!

szaimen commented Nov 3, 2026

Uh oh!

susnux left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

szaimen commented Nov 4, 2026

Uh oh!

susnux left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

szaimen commented Nov 5, 2026

Uh oh!

This comment was marked as resolved.

szaimen commented Nov 5, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

szaimen commented Oct 20, 2026 •

edited

Loading