Skip to content

About

SuperSplat with the MPEG GS4 (V-PCC) and GS5 (G-PCC) Gaussian splat web decoders and a player mode, used by the MPEG GSC demo

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

SuperSplat with the MPEG GS4 and GS5 decoders

This is SuperSplat (PlayCanvas, MIT licence) extended with web decoders for the two MPEG Gaussian splat coding solutions:

  • GS4: the Gaussian splat profile of V-PCC (ISO/IEC 23090-5), reference software mpeg-pcc-tmc2. The video is decoded with the browser's HEVC decoder (WebCodecs), with the HM reference decoder compiled to WebAssembly as a fallback.
  • GS5: the Gaussian splat profile of G-PCC (ISO/IEC 23090-9), reference software mpeg-pcc-tmc13-gs, decoded by TMC13-GS compiled to WebAssembly (branch v3.0-rc2-wasm, a git submodule of this repository).

It also has a player mode to embed it in a web page. The MPEG GSC demo (mpeg.expert/gsc/demo.html) runs this player; its page and tools are in gsc-demo.

Local Development

Install Node.js 18 or later, then:

  1. Clone the repository with its submodule:

    git clone --recurse-submodules https://github.com/MPEGGroup/SuperSplat-Integration.git
    cd SuperSplat-Integration
  2. Install the dependencies and build:

    npm ci
    npm run build

    The WebAssembly decoders are committed prebuilt in static/lib/ (gpcc/ for GS5, vpcc/ for GS4), so no Emscripten is needed for this step. To rebuild the GS5 decoder, see the next section.

  3. Start a local web server and open http://localhost:3000:

    npm run develop

    When the sources change, SuperSplat is rebuilt automatically: refresh the browser to see the changes.

GS5 (G-PCC) decoder in WebAssembly

Please refer to the README.md for a detailed description.

  1. Install dependencies:

    npm run init:gpcc-wasm
  2. Build TMC13-GS as WASM module:

    npm run build:gpcc-wasm
    npm run build
  3. Run Local Server

    npm run serve

Player mode and MPEG GSC bitstreams

SuperSplat can be embedded in a web page as a viewer (no editor UI): index.html?mode=player.

File extensions: .gs4 GS4 (V-PCC Gaussian splat profile, V3C sample stream, also .v3c), .gs5 GS5 (G-PCC Gaussian splat profile, TLV, also .gpcc). A .bin file is identified from its first bytes. GS4 needs the HEVC decoder of WebCodecs (Chrome, Edge, Safari).

URL options: load=<url>, info=1 (overlay: codec, size, Gaussians, bits/Gaussian, decoding time), rot=x,y,z (orientation in degrees, default 0,0,0: the MPEG GSC contents are y-up), fit=<factor> (framing distance, default 1.6), syncCamera=1.

The page drives the player with postMessage (see src/player-mode.ts): gsc:load {url, filename?, orientation?, keepCamera?}, gsc:setCamera {pose, speed?}, gsc:getCamera, gsc:resetCamera, gsc:syncCamera {enabled}, gsc:setInfo {visible}; the player posts gsc:ready, gsc:loading, gsc:loaded {filename, codec, bytes, numSplats, fetchMs, decodeMs, totalMs}, gsc:error, gsc:camera {pose} (while the user moves the camera, to synchronise two players).

<iframe id="player" src="supersplat/index.html?mode=player&info=1"></iframe>
<script>
  const player = document.getElementById('player').contentWindow;
  window.addEventListener('message', (e) => {
    if (e.data.type === 'gsc:ready') player.postMessage({ type: 'gsc:load', url: 'cricket_player_r02.gs5' }, '*');
  });
</script>

Encoding GS4 and GS5 bitstreams

  • GS4: build mpeg-pcc-tmc2 (tag release-26.1 or later) and encode one frame with PccGs4AppEncoder. The web decoder expects the packed-attribute layout below: copy cfg/gs4/gs-pack-attributes.cfg to cfg/gs4/gs-pack-attributes-cols8.cfg and change packedMapColumnsCount : 4,4 to packedMapColumnsCount : 4,8, then:

    git clone https://github.com/MPEGGroup/mpeg-pcc-tmc2.git && cd mpeg-pcc-tmc2 && git checkout release-26.1
    cmake -S . -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build --config Release --parallel 8
    ./bin/PccGs4AppEncoder \
      --uncompressedDataPath=/path/to/frame_0000.ply \
      --configurationFolder=cfg/gs4/ \
      --config=cfg/gs4/gs-main-common.cfg \
      --config=cfg/gs4/gs-pack-attributes-cols8.cfg \
      --config=cfg/gs4/rate/r01/encoder.cfg \
      --startFrameNumber=0 --frameCount=1 \
      --compressedStreamPath=/path/to/content_r01.gs4 \
      --shMappingMode=1 --implicitPacking=1 --gsdTransform2dPresentFlag=1 --dataAdaptiveShTransformEnabled=1 \
      --adaptivePcaEnabled=0 --gsdPcaDcFlag=1 --enableShNonUniformQuantization=1 \
      --enableScaleNonUniformQuantization=1 --shPredictionEnabled=0 --enableScalePowerTransform=1 \
      --enableOpacityNonUniformQuantization=1 --enableShDimensionReduction=0 --keepIntermediateFiles=0
    ./bin/PccAppDecoder --compressedStreamPath=/path/to/content_r01.gs4 \
      --reconstructedDataPath=/path/to/content_r01_decoded.ply --computeMetrics=0 --computeChecksum=0
  • GS5: build mpeg-pcc-tmc13-gs (tag release-v3.0) and use the CTC configurations generated by scripts/gen-cfg.sh.

The exact settings used for the MPEG GSC demo (per content and rate) are in the tools/ folder of gsc-demo.

MPEG integration and authors

SuperSplat is developed by PlayCanvas (MIT licence, see LICENSE). The MPEG GS4 / GS5 integration was contributed by:

  • Haining Yu: GS4 (V-PCC) web decoder: V3C parsing and GS4 reconstruction in WebAssembly, HEVC video decoding with WebCodecs, first GS4 web demo.
  • Jongseok Lee: GS5 (G-PCC) web decoder: TMC13-GS compiled to WebAssembly and its integration.
  • Marius Preda, co-authored with Claude Opus 5.5 (Anthropic): GS4 and GS5 in one build, player mode and its postMessage API, HM WebAssembly fallback and HEVC Main 10, viewing zones, GS5 colour fix of the WebAssembly decoder, the MPEG GSC demo (mpeg.expert/gsc/demo.html) and this public release.

About

SuperSplat with the MPEG GS4 (V-PCC) and GS5 (G-PCC) Gaussian splat web decoders and a player mode, used by the MPEG GSC demo

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages