Repository navigation
fix(rtc): use the right plane layout for I420A, I422 and NV12 frames - #864
Open
ANDROIDHASSAN wants to merge 1 commit into
Open
ANDROIDHASSAN wants to merge 1 commit into
ANDROIDHASSAN wants to merge 1 commit into
Conversation
_get_plane_length sized I420A and NV12 chroma as chroma_width * chroma_width instead of chroma_width * chroma_height, so a frame received from the FFI (VideoStream) was copied with the wrong length: past the end of the buffer for a landscape frame, short for a portrait one. _get_plane_infos started the I422 V plane one U plane too late, at the end of the buffer, which get_plane(2) and capture_frame then read from.
ANDROIDHASSAN
requested review from
changt,
cloudwebrtc,
lukasIO and
xianshijing-lk
as code owners
October 14, 2026 20:20
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two helpers in
video_frame.pycompute the layout of a packed frame, and three formats were off:_get_plane_lengthsized the chroma of I420A and NV12 aschroma_width * chroma_widthinstead ofchroma_width * chroma_height._from_owned_infocopies that many bytes out of FFI memory for every frame aVideoStreamdelivers in that format, so a landscape frame is copied past the end of its buffer and a portrait one comes back short. For 640x480: NV12 512000 bytes instead of 460800, I420A 819200 instead of 768000._get_plane_infosstarted the I422 V plane atu.data_ptr + u.size + u.size, one U plane too late, which is the end of the buffer.get_plane(2)reads from there, and_proto_infosends that pointer to the FFI onVideoSource.capture_frame.The fix is the three formulas; I420, I444 and I010 already matched.
Tests in
tests/rtc/test_video_frame.py:test_planes_are_packed_and_match_the_frame_length: for every planar format and 640x480, 480x640 and 5x3, the plane sizes match the format, the planes follow each other with no gap, and_get_plane_lengthequals their total. Fails onmainfor I420A, I422 and NV12 at every size.test_frame_from_the_ffi_copies_exactly_the_frame[I420A|NV12]:_from_owned_infoon a 640x480 buffer copies exactly the frame (the buffer is allocated larger, somainover-reads safely and the test just fails).test_get_plane_returns_each_i422_plane:get_plane(0..2)returns the Y, U and V bytes of an I422 frame. Fails onmain.Ran
tests/rtcagainst the publishedlivekitwheel withvideo_frame.py(andmain'saudio_mixer.py) swapped in: the new tests are the only difference betweenmainand this branch (test_agc/test_apm/test_resamplerfail identically on both here, from missing local fixtures).ruff format --check,ruff check(0.15.4) andmypyon the module pass.Scope: left alone, each separate from these formulas:
VideoFrame.__init__does not checklen(data)against_get_plane_length, so a short buffer can still be over-read byget_plane/capture_frame;_from_owned_infoassumes one packed buffer rather than readinginfo.components;get_planeaccepts a negative index.