[Android/Quest 2] 8192x4096@59.94 HEVC decodes at 8-10 fps in both backends; one EOS pass clears it permanently

[ondevChrisD]

Unity version

2022.3.60f1 LTS, URP, Linear, Vulkan, SinglePassMultiview

Unity editor platform

Windows

AVPro Video edition

Core

AVPro Video version

3.4.2 (native 3.4.2-core)

Device hardware

Meta Quest 2, Snapdragon XR2 Gen 1, Adreno 650, Android 14 / API 34, OS UP1A.231005.007.A1

Which Android OS version are you using?

14

Unity Graphics API

Vulkan

Video API

media3/ExoPlayer

Texture format

BGRA

Audio output

System Direct

Any other Media Player component configuration required to reproduce the issue.

Option	Value
Loop	on
forceEnableMediaCodecAsynchronousQueueing	on
minBufferMs / maxBufferMs	5000 / 50000
bufferForPlaybackMs / bufferForPlaybackAfterRebufferMs	5000 / 5000
startWithHighestBitrate	on
prioritiseTimeOverSize	off
preferredMaximumResolution	No Preference
allowUnsupportedVideoTrackVariants	off
videoOutputMode	Texture

Alternatives we also tested, all of which reproduce identically:

• Video API = MediaPlayer instead of media3/ExoPlayer
• Texture format = YCbCr420_OES instead of BGRA
• videoOutputMode = XRCompositionLayer — decoder surface straight to the XR swapchain, so no BGRA conversion and no Unity texture at all

Media is opened with OpenMedia(path, autoPlay: true) from local storage.

Which output component(s) are you using?

Apply to Mesh

Any other component configuration required to reproduce the issue.

Output: Apply To Mesh, onto a single inside-out sphere. The material uses a custom equirect shader rather than an AVPro one; stereo is handled by eye-mode material state.

XR stack: OpenXR loader 1.14.3, com.unity.xr.meta-openxr 1.0.4, SinglePassMultiview, URP, Linear colour space. Display runs at 71.978 Hz (app-measured).

Instrumentation: once per Unity render frame we read AVPro's texture frame counter and count new decoded frames delivered per second. We refer to this below as texFps. It measures frames AVPro delivered to Unity, not display cadence. No sampling, no averaging.

The issue

Summary

An 8192×4096 @ 59.94 fps stereo-180 HEVC Main 10 file is delivered by AVPro at 8–10 fps on Meta Quest 2, while our Unity render loop holds a flat 72.0 fps with zero late frames. The failure is bistable: every OpenMedia starts collapsed, usually recovers after one to three loop wraps, and can relapse. Even the recovered state is only 42–46 fps on a 59.94 fps stream — we have never seen this clip decode at its native rate under AVPro on this device.

We have closed the content axis, the API axis and the output-path axis by measurement, and we have a decisive control: Meta's Horizon Media Player plays the identical file on the identical headset through the identical decoder component, flawlessly, at 90 Hz.

Most importantly we have found a reproducible trigger that clears the defect, which we think gives you a tractable handle on it. That is section 1, deliberately first.

Related: #2391 (closed could-not-reproduce — we believe this is the same defect and we can reproduce on demand) · #1138 (8K on Windows: after a seek it plays for ~1 s, then freezes and drops frames for 1–2 s before running normally — the same "a discontinuity buys a couple of seconds" shape we measure here, closed as stale) · #2313, #1774, #596, #547 (8K on Quest, unresolved)

1. The EOS trigger — the most actionable finding

We can clear the defect deterministically, from the app, without touching AVPro:

1. OpenMedia(path, autoPlay: false)
2. wait for this clip's metadata (MetaDataReady)
3. SeekFast(duration − ~600 ms)
4. Play(), let playback run off the end and take the natural loop wrap
5. do not seek back to 0 — a seek is a flush and undoes it

After that, the same clip plays at 41.8–45.4 texFps from the first second, with no collapsed pass at all, for the rest of the session. Verified repeatedly on device.

A mid-stream seek does not produce this. It yields 2–3 seconds of good frames and then re-collapses, exactly like a fresh open. The operative event is reaching end-of-stream, not the flush.

This is a one-way latch from a bad state into a good one, triggered by a specific, cheap, repeatable action — which is why we are leading with it.

2. The failure shape

State	texFps	appFps	Worst frame	Late frames
Healthy	42–46	72.0	14–18 ms	0
Collapsed	8–10	72.0	14–18 ms	0

The render loop is idle during the collapse. This is not a Unity-side stall.

A representative eight-pass run on a 59.00 fps variant:

pass 0  collapsed
pass 1  clean
pass 2  clean
   -- reopen --
pass 3  collapsed
pass 4  collapsed
pass 5  collapsed
pass 6  clean
pass 7  clean
pass 8  clean

3. The matched pair — this rules out a throughput wall

Clip	Macroblocks × rate	MediaCodecInfo probe	Kernel H/W is overloaded	Healthy	Collapsed
8192×4096@59.94	7,856,455	maxFpsAtThisSize 59.3 → not supported	logged every open	42–46	8–10
7680×3840@59.94	6,904,090	maxFpsAtThisSize 67.5 → SUPPORTED	never logged	42–46	8–10

Both states are numerically identical at both resolutions, across a 14% difference in pixel count. Neither state is pixel-throughput-bound, and the driver's admission check is not the trigger — the 7680×3840 clip clears it and collapses anyway.

4. Backend bisection

Video API switched to MediaPlayer (verified in our log, not the Inspector), same build, 100% battery / 28 °C:

texFps: 13.9, 48.0, 46.4, 31.8, 8.9, then flat 8-10

Identical to the media3/ExoPlayer curve. The MediaCodecInfo: NoSupport [sizeAndRate.cover, 8192x4096@59.94] lines disappear on this path; the behaviour does not.

5. Output-path bisection

We also ran the clip in XRCompositionLayer output mode, where the decoder surface is handed straight to the XR swapchain — no BGRA conversion, no Unity texture, no sphere sampling. Identical behaviour. We separately tested Texture format = YCbCr420_OES. Also identical. So nothing in the texture consumer or the colour conversion is involved.

6. The control: Meta Horizon Media Player, same file, same headset

Captured from logcat while Horizon played the same original file:

	Horizon Media Player	AVPro
Decoder component	OMX.qcom.video.decoder.hevc	OMX.qcom.video.decoder.hevc
nativeWindow	8192×4096	8192×4096
Colour format	0x7fa30c06 → 0x7fa30c09 (YCbCr_420_TP10_UBWC)	identical
MediaCodecInfo consulted	never	yes (ExoPlayer path)
Kernel H/W is overloaded	zero across five sessions	every open
Output buffers	16	17 (20 s clip), 19 (144 s original)
Graphics API	Vulkan, no OES	Vulkan, no OES
Playback	correct, 90 Hz	8–10 fps

Because the component, surface geometry and pixel format are identical on both sides, no property of the file can account for the difference, and because both AVPro backends fail identically, no property of the backend choice can either.

7. What we have already falsified — please do not re-derive

Hypothesis	How it was killed
Memory pressure	Collapses with 2.5–2.8 GB free
Byte rate	151 → 31.6 Mbps (4.8×) changed nothing
Bit depth	8-bit Main / NV12 UBWC identical to 10-bit Main10 / TP10 UBWC
Declared frame rate	A 59.00 variant probes SUPPORTED, zero overload lines, still collapses
Resolution / macroblock ceiling	7680×3840 is 12% under the ceiling and collapses; 8192×4096@30 plays perfectly
Codec / API choice	Both AVPro Android backends identical
Texture path	XRCompositionLayer and OES both identical to BGRA
Seek as a remedy	Buys 2–3 s, exactly like a fresh open
Container / timestamps	Clean NVENC re-encodes collapse identically
Display cadence, capture cost, audio-clock sync, GOP length	Tested and cleared separately

8. What we think is happening, and what we are asking for

We do not claim to know the mechanism. But the combination of (a) the kernel registering operating rate 30 on a 59.94 stream (see the Logcat field), (b) both AVPro backends failing where Meta's direct-MediaCodec client succeeds, and (c) AOSP's own documented behaviour for decoders configured without low latency, points at configure-time MediaCodec keys that AVPro does not appear to set.

• AOSP, low-latency decoding in MediaCodec — with the feature off, "the decoder can use power optimizations that may result in decoded frames being returned later than strictly necessary, and the decoder may power down when stalled"; with it on, the decoder "must return decoded frames as soon as possible … and must stay awake." https://source.android.com/docs/core/media/low-latency-media
• FFmpeg, adding operating_rate to its MediaCodec decoder — "The codec wants to know whether the usecase is realtime playback or full-speed transcoding… The codec runs faster when operating_rate higher than framerate." https://www.mail-archive.com/ffmpeg-devel@ffmpeg.org/msg172972.html
• Android MediaFormat — KEY_PRIORITY 0 is realtime, meaning the codec shall support the given performance configuration at realtime; KEY_FRAME_RATE is only authoritative for pacing when priority is realtime and KEY_OPERATING_RATE is absent. https://developer.android.com/reference/android/media/MediaFormat
• Qualcomm MediaCodec vendor extensions — getSupportedVendorParameters() enumerates device support; static params go through MediaFormat before configure(), dynamic ones through setParameters(). https://www.qualcomm.com/developer/blog/2023/07/building-media-rich-android-apps-mediacodec-and-vendor-extensions
• ALVR configures its Android decoder with operating-rate = 2147483647, priority = 0 and vendor.qti-ext-dec-low-latency.enable = 1 — the same combination we are asking about.
• Counter-evidence, in fairness: ExoPlayer issue #4496 reports that setting KEY_OPERATING_RATE did not fix a 60 fps frame-drop problem, so this is not a guaranteed fix. Dropped Frames on DASH Widevine 60fps dynamic MPDs google/ExoPlayer#4496

Our questions

1. Does the Android backend set MediaFormat.KEY_OPERATING_RATE at configure()? If it derives the value from the stream frame rate, does that survive for local MP4? Our kernel dump shows operating rate 30 while frame rate 59 arrived correctly, which suggests the key is either never set, or set below ExoPlayer's assumedMinimumCodecOperatingRate threshold.
2. Does it set KEY_LOW_LATENCY (API 30+)? AOSP's description of the off state matches our measured symptom closely.
3. Does it set KEY_PRIORITY = 0?
4. Does it query getSupportedVendorParameters() and apply vendor low-latency extensions where present?
5. What determines the output buffer count? Horizon runs this stream with 16, we see 17 and 19. At 8192×4096 TP10 UBWC each buffer is roughly 100 MB. Is the count derived from stream reorder depth, and can it be bounded?
6. Is there anything in the native plugin that could produce a bistable delivery rate independent of pixel count, and that latches permanently good after an EOS? The two states are numerically identical at 7680×3840 and 8192×4096, which rules out a bandwidth explanation on our side.
7. Can you reproduce [Android] Loading and Unloading High-Resolution video sequentially #2391 with this evidence? Our symptom matches it closely and we can reproduce on demand on retail hardware.

What would help us most, in order

1. Expose KEY_OPERATING_RATE, KEY_LOW_LATENCY and KEY_PRIORITY as PlatformOptionsAndroid fields. We cannot reach them — the Android backend ships as prebuilt binaries. With them exposed we can test all three combinations on device in an afternoon and report back with numbers, at no cost to you.
2. Set sensible defaults for high-resolution content: operating rate high (or Short.MAX_VALUE), priority realtime, low latency on, with the Qualcomm vendor extension applied where getSupportedVendorParameters() reports it.
3. A debug build with those keys set, if you would rather not commit to defaults. We will run it and send logs.
4. Consider a "trust the device" path that does not gate on MediaCodecInfo for this content class. Meta's own player never consults it and plays the same file correctly.

9. What we are shipping meanwhile

We ship the EOS prime described in section 1, gated to clips above a decode-work threshold (ceil(w/16) × ceil(h/16) × fps ≥ 5,000,000), because it costs roughly 1.5 seconds at open and ordinary content must not pay for it. That threshold was derived empirically: everything that plays cleanly on this device is at or below 3.9M, everything that collapses is at or above 6.9M.

We mention this so you know the workaround exists, and because it means we can demonstrate the defect and its cure side by side on the same build with a single config flip — which should make verification on your end straightforward.

We are happy to run instrumented builds, add logging, share source clips.
We have a rig set up for this and can turn tests around quickly.

Media information

All clips are stereo-180 side-by-side, local files, single video track, encoded with NVENC out of DaVinci Resolve.

Clip	Codec	Size	Rate	Bitrate	Duration
Primary repro	HEVC Main 10 yuv420p10le L6.1	8192×4096	59.94	~96 Mbps	144.92 s
NVENC 20 s cut	HEVC Main 10	8192×4096	59.94	~151 Mbps	20 s
Under-ceiling control	HEVC Main 10	7680×3840	59.94	—	60.33 s
Declared-rate control	HEVC Main 10	8192×4096	59.00	—	20 s
Bit-depth control	HEVC Main yuv420p (NV12 UBWC)	8192×4096	59.94	—	20 s
Byte-rate control	HEVC Main 10	8192×4096	59.94	31.6 Mbps	20 s
Known-good reference	HEVC Main 10	8192×4096	30	—	—

The primary repro plays correctly in Meta's Horizon Media Player on the same headset.

Meta's published guidance lists 8192×4096 / 60 fps as the hardware decode limit for this class of headset, so this content sits at the documented ceiling — but see section 3 of the issue, where a clip comfortably under the ceiling fails identically.

Per your usual request not to attach media here, the sample clips and the full logs are being emailed to unitysupport@renderheads.com referencing this issue number. Available: a bug/control pair, five device captures, and the original source clip on request.

Logcat output

**Kernel video driver, at codec open.** The instance table prints only when the admission check fails, which is why it appears for 8192×4096 and not for 7680×3840:


msm_vidc: err : 00000003: h265d: H/W is overloaded. needed: 7857448 max: 7833600
msm_vidc: err : 00000001: h264d:    1| 960| 720|  23|  30|   R
msm_vidc: err : 00000002: h264d:    1| 960| 720|  23|  30|   R
msm_vidc: err : 00000003: h265d:    1|8192|4096|  59|  30|   R
msm_vidc: err : 00000003: h265d: msm_vidc_check_session_supported: MBPS limit exceeded


Columns are codec, width, height, frame rate, operating rate. Our session is registered with **frame rate 59 but operating rate 30** on a 59.94 fps stream. Note the frame rate did reach the driver, so this is not a case of the frame rate being unknown. The two h264 sessions are foreign, permanently resident OS sessions — they survive reboot and are not ours.

**ExoPlayer path**, on every open of the 8192×4096 clip:


MediaCodecInfo: NoSupport [sizeAndRate.cover, 8192x4096@59.94]


These lines disappear when Video API is switched to MediaPlayer. The collapse does not.

**Our own app instrumentation**, after the EOS prime described in section 1 (`vfps` is the stream rate AVPro reports, `texFps` is new frames delivered per second):


hz=72.0 vfps=59.94 texFps=41.8 stalled=False playing=True size=8192x4096
hz=72.0 vfps=59.94 texFps=44.0 stalled=False playing=True size=8192x4096
hz=72.0 vfps=59.94 texFps=43.8 stalled=False playing=True size=8192x4096
hz=72.0 vfps=59.94 texFps=45.4 stalled=False playing=True size=8192x4096


Full logcat and app logs for all five captures are being emailed rather than attached here, as they contain client content filenames.

[Chris-RH]

Thank you for the emailed file.