Skip to content

fix(mediaplayer): rtsp default and video output documentation - #992

Open
towneh wants to merge 1 commit into
BasisVR:developerfrom
towneh:docs/mediaplayer-video-output
Open

fix(mediaplayer): rtsp default and video output documentation#992
towneh wants to merge 1 commit into
BasisVR:developerfrom
towneh:docs/mediaplayer-video-output

Conversation

@towneh

@towneh towneh commented Aug 1, 2026

Copy link
Copy Markdown
Collaborator

Summary

The package README documented protocols, codecs, delivery, audio and sync, then stopped at
the point where a decoded frame reaches a screen. That left the shader contract behind
letterboxing undocumented: FitInside scales the bar axis past 1 on purpose, and only
Basis/Media Player Video renders those out-of-range UVs black. Put the video on any other
material and the frame texture's clamped edge texel is smeared across the bar instead, with
nothing in the docs to say why.

This adds a Video output (screens and UI) section covering the two sinks, the shader and
why the screen material matters, the aspect modes and which are safe on an arbitrary
material, display aspect and its transform-scale caveat, projection, orientation, picture
controls, and what a custom screen shader has to implement.

It also brings several existing claims back in line with the code:

  • the directly-playable extension list was missing .m4v, .m4a, .webm, .opus and
    .mp3, understating what loads without a resolver
  • MP3 had no row in the supported-URL table
  • the Windows bullet named only AAC, omitting MP3 and Opus
  • audio-only playback listed WAV alone
  • AnalysisFeed, which lets per-source analysers such as AudioLink read the stream back,
    was undocumented
  • the RTMP known limit named RTSPT as a primary path, contradicting the transport table

Known limits gains an entry for the projection modes that set a shader keyword no bundled
shader implements, and for the picture controls that need a shader declaring them.

One behaviour change: BasisMediaPlayerStreaming now defaults to rtsp:// rather than
rtspt://. VRCDN's own guidance is that rtsp:// is the better default, being truer to the
RTSP spec, and the player already negotiates UDP first and falls back to RTP interleaved
over the TCP control channel on refusal, a socket error or the no-data timer, remembering a
host that fails UDP so later loads go straight to TCP. rtspt:// still pins TCP-interleaved
for hosts or networks where UDP never works. The prefabs serialise empty URLs, so this
affects a freshly added component only.

Required checks

All boxes below must be ticked before this PR can merge. If a check is genuinely N/A, tick it anyway and explain under Notes.

  • Tested — I built and ran this locally. The change works in the editor and (where relevant) in a built player.
  • Transform access is combined and limited — In hot paths, transform reads/writes go through TransformAccessArray or are otherwise batched. I have not added per-frame transform.position / transform.rotation / transform.localPosition calls inside loops. Whenever I need both position and rotation, I use the combined APIs — SetPositionAndRotation / SetLocalPositionAndRotation for writes, GetPositionAndRotation / GetLocalPositionAndRotation for reads — instead of two separate property accesses; the combined call does one local-to-world matrix traversal instead of two.
  • Addressables used for asset/memory loading — Any new asset loads go through Addressables. No new Resources.Load, no direct asset references that pull large content into memory on scene load.
  • No new GetComponent / AddComponent where avoidable — Where unavoidable, the result is cached on a field, and any GetComponent<T> is replaced with TryGetComponent<T>(out var x) — bare GetComponent will be denied. TryGetComponent is the modern API (Unity 2019.2+) and skips the Editor-only GC allocation GetComponent causes when a component is missing: Unity wraps the null return in a managed "fake null" object so its overloaded == operator can still detect destroyed C++ objects, and constructing that wrapper allocates; TryGetComponent returns a bool plus out parameter and never builds the wrapper. None of these calls run inside Update, LateUpdate, FixedUpdate, jobs, or other per-frame code paths.
  • Per-frame work is scheduled through BasisEventDriver — Any new per-frame work hooks into BasisEventDriver rather than adding standalone Update / LateUpdate / FixedUpdate callbacks on a MonoBehaviour.
  • Anything added to BasisEventDriver is bulletproof, or guarded by try/catchBasisEventDriver runs the single per-frame tick that drives the whole framework (network apply, local player sim, blendshapes, JigglePhysics, nameplates, and more) as one sequential chain. An unhandled exception anywhere in that chain aborts the rest of the tick, so every step after the throwing one is silently skipped for that frame. New work added to the driver must either be guaranteed not to throw, or be wrapped in a try/catch that contains the failure and surfaces it through BasisDebug — logged once / rate-limited, never every frame (see the existing HVRBasisBuiltInAddresses.Simulate() guard for the pattern). Expect this to be scrutinized closely in review.
  • Considered jobification — I asked whether this work can be moved to a Unity Job (Burst-compiled where possible). If it can, it is. If it cannot, the reason is in Notes.
  • No needless { get; set; } properties or access lockdowns — Public fields are fine; Basis is meant to be read and modified freely, so don't wall things off private/internal without a real reason. Don't wrap a field in { get; set; } when the accessors do nothing — property accessors have a real performance cost vs direct field access, and the lead maintainer prefers plain fields (or a method / setter-only property when only the setter needs logic) over a noop-getter pair. For .Instance singletons, callers reassigning Type.Instance is allowed; if that would break your code, log a warning or throw — don't block the assignment. Locking down access is not your call.
  • Camera access goes through BasisLocalCameraDriver — Code that needs the local camera (transform, projection, rig data, etc.) pulls it from BasisLocalCameraDriver rather than looking one up itself. Don't roll a separate camera discovery path.
  • Logging uses BasisDebug — All new logging calls go through BasisDebug.Log / BasisDebug.LogWarning / BasisDebug.LogError (with an appropriate LogTag) instead of UnityEngine.Debug.Log / Debug.LogWarning / Debug.LogError. BasisDebug routes through Basis's tagged, color-coded logger and respects the project-wide LoggingDisabled toggle so logging can be killed at runtime; bare Debug.Log calls bypass that and will be denied.
  • No scene-wide discovery for dependencies — New code is architected so it does not need FindObjectOfType / FindObjectsOfType / GameObject.Find / FindGameObjectsWithTag to locate what it depends on. References are wired in — registered through an existing manager/driver, injected at init, or passed in by the caller — rather than discovered by scanning the scene at runtime. If a scene scan is genuinely unavoidable, justify it under Notes.
  • No allocations in hot paths — Per-frame code (Update / LateUpdate / FixedUpdate, simulation loops, jobs, anything called once per frame or more) does not allocate. No new on reference types, no LINQ, no string concatenation/interpolation, no boxing, no foreach over interface-typed collections. Allocate once at init and reuse the buffer.
  • No debugging in hot paths — No log calls of any kind on per-frame paths, including BasisDebug. Hot-path logging floods the console and incurs cost on every frame regardless of whether the message is filtered out downstream. If a hot-path log is needed while iterating, gate it behind #if UNITY_EDITOR and remove (or leave gated) before merge.
  • Hot-path collection access is optimized — Cache .Count (lists) / .Length (arrays) into a local int before the loop instead of re-reading the property each iteration. Prefer T[] (with a separate length int when the array is over-sized) over List<T> where the data is hot — Unity's mono BCL doesn't expose CollectionsMarshal.AsSpan(List<T>), so a list can't be fed into Span<T> / unsafe paths cleanly. Where the perf justifies it, drop into Span<T> / ref locals / Unsafe.As / unsafe pointer code to skip bounds checks and copies, and call out the invariants you're relying on under Notes so reviewers can sanity-check them.

Testing details

Tick the platforms you actually tested on. Leave the rest unticked — these are informational and do not block merge.

  • Windows
  • Linux
  • Android
  • iOS
  • macOS

Input / control mode coverage:

  • Tested in VR (note headset under Notes)
  • Tested in desktop / non-VR mode
  • Tested with phone controls (mobile touch input)
  • N/A — change does not touch player/XR/input code

Where applicable, confirm these flows still work after your changes:

  • Hot-switching (desktop ↔ VR mode swap at runtime)
  • Avatar swapping
  • Server swapping (joining / leaving / changing servers)
  • N/A — change does not touch any of the above

Notes

This is a documentation change plus one default string value in an example component, so
most required boxes are N/A and ticked as such: no new runtime code, no transform access,
no asset loading, no GetComponent, no per-frame work, no logging, no scene discovery, no
hot paths.

On Tested — the docs half carries no runtime risk. The rtsp:// default has not been
exercised against VRCDN on this branch; the negotiation path it relies on is pre-existing
and unchanged, so the behavioural difference is that a freshly added component now probes
UDP once before settling rather than pinning TCP from the start. BasisMediaPlayer.CurrentTransport
reports which transport won, and the Console logs it once per load.

The documentation describes current behaviour, including three gaps it would otherwise
paper over: the Equirect360 / VR180 / Fisheye modes set a BASIS_PROJ_* keyword that
no bundled shader implements, so they render flat; the Picture controls need a shader
declaring the _Basis* floats, which Basis/Media Player Video does not, so only the UI
path's Brightness currently applies; and DisplayAspectOverride at 0 derives the display
aspect from the renderer's local bounds, which exclude transform scale.

The streaming example now defaults to rtsp://. VRCDN's own guidance is that it
is the better choice: it follows the RTSP spec more closely, and the player
already negotiates UDP first, falling back to RTP interleaved over the TCP
control channel on refusal, a socket error or the no-data timer. A host that
fails UDP is remembered, so later loads go straight to TCP. rtspt:// pins
TCP-interleaved and never probes, which is still what you want on a host or
network where UDP never works. The prefabs serialise empty URLs, so this
changes a freshly added component only.

The README covered protocols, codecs, delivery, audio and sync but stopped at
the point where a frame reaches a screen, which left the shader contract behind
letterboxing undocumented. FitInside scales the bar axis past 1 on purpose, and
only the Basis/Media Player Video shader renders those out-of-range UVs black.
On any other material the frame texture's clamped edge texel is smeared across
the bar instead, and nothing said why.

The new section covers the two output sinks, the shader, the aspect modes and
which of them are safe on an arbitrary material, display aspect and its
transform-scale caveat, projection, orientation, picture controls, and what a
custom screen shader has to implement: the texture ST carries aspect,
stereo-eye selection and flips, while Equirect360, VR180 and Fisheye need their
own mapping keyed off BASIS_PROJ_EQUIRECT, BASIS_PROJ_VR180 or
BASIS_PROJ_FISHEYE.

Several existing claims are also brought back in line with the code:

- the directly-playable extension list was missing .m4v, .m4a, .webm, .opus
  and .mp3, so it understated what loads without a resolver
- MP3 had no row in the supported-URL table
- the Windows bullet named only AAC, omitting MP3 and Opus
- audio-only playback listed WAV alone
- AnalysisFeed, which lets per-source analysers such as AudioLink read the
  stream back, was undocumented
- the RTMP limit named RTSPT as a primary path, contradicting the transport
  table

Known limits gains an entry for the projection modes that set a shader keyword
no bundled shader implements, and for the picture controls that need a shader
declaring them.
@towneh
towneh requested a review from dooly123 August 1, 2026 20:59
@towneh towneh added the documentation Improvements or additions to documentation label Aug 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant