Add dynamic V4L2 test pattern controls to emulated_camera_mplane by changyeon-jo · Pull Request #2845 · google/android-cuttlefish

changyeon-jo · 2026-07-10T23:26:59Z

Implement support for dynamic video test patterns and their associated
V4L2 control handlers. Supported patterns include:

Pattern 0 (Legacy Pulse): Uniform color-cycling.
Pattern 1 (SMPTE + Bouncing Box): Full color SMPTE bars with an animated
inverse-color bouncing box overlay, adapted for the 3-sink layout and
640x480 resolution.
Pattern 2 (Animated Julia Set): CPU-intensive fractal pattern simulation
for high-load host load scenarios.

Supports V4L2_CID_TEST_PATTERN and V4L2_CID_IMAGE_PROC_CLASS via the
VirtioMediaIoctlHandler implementation.

Bug: b/490699808

ser-io

Please move commit: "Implement real-time FFmpeg host video streaming for emulated_camera_mplane" into its own PR.

Also, create a brand new virtio-media device for video streaming. Don't extend emulated_camera_mplane for this.

changyeon-jo · 2026-07-14T19:14:29Z

@ser-io

Re:

Please move commit: "Implement real-time FFmpeg host video streaming for emulated_camera_mplane" into its own PR.

I can separate FFmpeg commit from PR.

Re:

Also, create a brand new virtio-media device for video streaming. Don't extend emulated_camera_mplane for this.

May I ask why you prefer having this in separate device? IMHO, it's also "emulating" a camera data stream from a media source and therefore does not sound out of this device's scope.

If you intend to make this as the one strictly generating test patterns, shouldn't we need to rename this like "test_pattern_generator_mplane" or something similar? :)

ser-io · 2026-07-14T21:44:28Z

@ser-io

Re:

Please move commit: "Implement real-time FFmpeg host video streaming for emulated_camera_mplane" into its own PR.

I can separate FFmpeg commit from PR.

Re:

Also, create a brand new virtio-media device for video streaming. Don't extend emulated_camera_mplane for this.

May I ask why you prefer having this in separate device? IMHO, it's also "emulating" a camera data stream from a media source and therefore does not sound out of this device's scope.

If you intend to make this as the one strictly generating test patterns, shouldn't we need to rename this like "test_pattern_generator_mplane" or something similar? :)

Cuttlefish attaches virtio-media devices using vhost-user protocol, which allows the flexibility to have device emulation per host process. Having said that, the case for streaming emulated frames and using FFmpeg for host video streaming are quite different in terms of how the frames are populated which present a case for different binaries. Given we already have the vhost-user protocol in place, it's cleaner to have two different binaries with their own specific flags.

The current name emulated_camera_mplane is based on existing EmulatedCamera which this device aims to replace.

Format pre-existing files inside `emulated_camera_mplane` to comply with the standard Rust formatting style. This eliminates formatting noise in subsequent functional commits. Bug: b/490699808 Bug: b/502639876

changyeon-jo · 2026-07-15T05:14:52Z

Thanks for explanation, @ser-io. Since this replaces the camera framework team's EmulateCamera, it sounds like you're aiming to emulate the Android camera device's attributes and characteristics, rather than just the data stream. Do you plan to support FULL or LEVEL_3? :)

Also, will this device's characteristics be configurable similar to hardware/google/camera/devices/EmulatedCamera/hwl/configs/emu_camera_back.json?

ser-io

Thanks for changing the commits to self-contained smaller commits. Highly appreciated.

ser-io · 2026-07-15T14:33:15Z

    }
 }

+const V4L2_CID_IMAGE_PROC_CLASS: u32 = 0x009f0000;


I can see these are standard const https://docs.kernel.org/userspace-api/media/v4l/ext-ctrls-image-process.html. Are not they already defined in the in the Rust v4l2r::bindings port?

Yes, they are currently missing in v4l2r crate at the moment. We should add them to v4l2r crate eventually.

I can build using bindings::V4L2_CID_TEST_PATTERN and bindings::V4L2_CID_IMAGE_PROC_CLASS, I have linux headers version: 6.18. cat /usr/include/linux/version.h. which version do you have?

ser-io · 2026-07-15T15:19:09Z

Thanks for explanation, @ser-io. Since this replaces the camera framework team's EmulateCamera, it sounds like you're aiming to emulate the Android camera device's attributes and characteristics, rather than just the data stream. Do you plan to support FULL or LEVEL_3? :)

Also, will this device's characteristics be configurable similar to hardware/google/camera/devices/EmulatedCamera/hwl/configs/emu_camera_back.json?

Correct! It should be FULL however this is still in the very early stages, priority is to migrate existing workflow (mostly from auto targets) that can now use the data streams and the existing external camera hal.

Declare the trait-based `FramePattern` interface inside `src/pattern/mod.rs`. This trait defines the interface for dynamically writing multi-planar YUV video frames into queued virtio-media buffers. This first commit adds the trait definitions and registers the pattern module in the build system as a standalone declaration. Bug: b/490699808 Bug: b/502639876

Implement the color-cycling `Pulse` pattern inside `src/pattern/pulse.rs`. Refactor the internal `write_pattern` helper inside `device.rs` to call the `CameraPattern::Pulse` abstraction instead of hardcoding YUV loop math. Bug: b/490699808 Bug: b/502639876

Introduce the V4L2 controls to allow runtime switching of emulated camera test patterns. Wires up: - V4L2_CID_IMAGE_PROC_CLASS - V4L2_CID_TEST_PATTERN Exposes these controls via standard query and control ioctl handlers, including querymenu and control change event subscriptions. Only Pattern 0 (Pulse) is selectable in this commit. We can use the standard `v4l2-ctl` command-line tool inside the guest OS to list, query, and change the test patterns at runtime: # List all available camera controls (shows "Test Pattern" menu) v4l2-ctl -d /dev/video1 --list-ctrls # Get the currently selected pattern index v4l2-ctl -d /dev/video1 -C test_pattern # Switch the selected pattern to Pulse (index 0) v4l2-ctl -d /dev/video1 -c test_pattern=0 Bug: b/490699808 Bug: b/502639876

Implement the color-bar SMPTE Bars test pattern with an animated inverse-color bouncing box overlay inside `src/pattern/smpte.rs`. Register it in the `pattern` module and expand `device.rs` V4L2 control ranges and query menus to support selecting index 1. Bug: b/490699808 Bug: b/502639876

Implement the CPU-intensive animated Julia Set fractal pattern inside `src/pattern/julia_set.rs`. Register it in the `pattern` module and expand `device.rs` V4L2 control ranges and query menus to support selecting index 2. Bug: b/490699808 Bug: b/502639876

ser-io · 2026-07-16T16:58:14Z

When introducing new files this repo we need to add the LICENSE at the top.

// Copyright 2026, The Android Open Source Project ...

Add the license to all the new files introduced in this PR.

ser-io · 2026-07-16T17:00:30Z

 use std::os::fd::AsFd;
 use std::os::fd::BorrowedFd;

+use std::io::Write;


nit: why this was moved here? it should be with other "std::io" declarations.

ser-io · 2026-07-16T17:09:56Z

+    flags: u32,
+}
+
+const CONTROLS: [ControlDef; 3] = [


Let's remove "CID_LENS_FACING" from this list, CID_LENS_FACING struct is managed by lens_facing_query_ext_ctrl method, the new ControlDef instance is not aligned with that method.

ser-io · 2026-07-16T17:12:46Z

+const V4L2_CID_IMAGE_PROC_CLASS: u32 = 0x009f0000;
+const V4L2_CID_TEST_PATTERN: u32 = 0x009f0903;
+
+struct ControlDef {


why not use bindings::v4l2_query_ext_ctrl directly?

ser-io · 2026-07-16T17:15:22Z

+        maximum: 0,
+        step: 1,
+        default_value: 0,
+        flags: 0,


this control is write and read, why not declaring those flags?

ser-io · 2026-07-16T17:15:44Z

+        maximum: 0,
+        step: 0,
+        default_value: 0,
+        flags: bindings::V4L2_CTRL_FLAG_READ_ONLY | bindings::V4L2_CTRL_FLAG_WRITE_ONLY,


can you write to class control?

ser-io · 2026-07-16T17:22:49Z

    /// Lens facing configuration.
    lens_facing: LensFacing,
+    /// Currently selected test pattern.
+    current_pattern: u32,


Use an enum rather than "u32". Have the first value (0) named undefined as this struct needs to be proto compatible, and defined Pulse as 1.

ser-io · 2026-07-16T17:24:13Z

-        } else if id == CID_LENS_FACING {
-            return Ok(self.lens_facing_query_ext_ctrl());
+
+            let mut name = [0 as ::std::os::raw::c_char; 32];


define a method in ControlDef for this.

ser-io · 2026-07-16T17:27:27Z

+                dims: [0; 4],
+                ..Default::default()
+            })
+        } else {


Avoid these else clauses as they are not helpful in this context. Return return Err(libc::EINVAL); directly.

changyeon-jo force-pushed the migrate_tpg branch 3 times, most recently from 8515fd4 to 09bda08 Compare July 10, 2026 23:51

changyeon-jo marked this pull request as draft July 10, 2026 23:59

changyeon-jo force-pushed the migrate_tpg branch from 5591022 to 2e26171 Compare July 11, 2026 00:09

changyeon-jo marked this pull request as ready for review July 11, 2026 00:49

ser-io requested changes Jul 13, 2026

View reviewed changes

Comment thread base/cvd/cuttlefish/host/commands/vhost_user_media/emulated_camera_mplane/src/device.rs

changyeon-jo force-pushed the migrate_tpg branch from d6069c2 to 78b00e3 Compare July 14, 2026 00:52

ser-io requested changes Jul 14, 2026

View reviewed changes

changyeon-jo requested a review from ser-io July 14, 2026 19:10

ser-io requested changes Jul 14, 2026

View reviewed changes

changyeon-jo force-pushed the migrate_tpg branch 6 times, most recently from e5be4b9 to 5f9e6e7 Compare July 15, 2026 04:58

Run cargo fmt on baseline emulated_camera_mplane

d6b351a

Format pre-existing files inside `emulated_camera_mplane` to comply with the standard Rust formatting style. This eliminates formatting noise in subsequent functional commits. Bug: b/490699808 Bug: b/502639876

changyeon-jo force-pushed the migrate_tpg branch from 5f9e6e7 to 5325a81 Compare July 15, 2026 05:06

ser-io requested changes Jul 15, 2026

View reviewed changes

changyeonjo added 2 commits July 15, 2026 16:15

changyeon-jo force-pushed the migrate_tpg branch 2 times, most recently from 442de97 to 5acc779 Compare July 15, 2026 17:48

changyeonjo added 2 commits July 15, 2026 18:08

changyeon-jo force-pushed the migrate_tpg branch from 5acc779 to b23512b Compare July 15, 2026 18:08

ser-io requested changes Jul 16, 2026

View reviewed changes

Uh oh!

Conversation

changyeon-jo commented Jul 10, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ser-io left a comment

Choose a reason for hiding this comment

Uh oh!

changyeon-jo commented Jul 14, 2026

Uh oh!

ser-io commented Jul 14, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

changyeon-jo commented Jul 15, 2026

Uh oh!

ser-io left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ser-io commented Jul 15, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants