Skip to content

JsRuntimeHost has no streaming body surface: Response.body, a constructible Response, Blob.slice()/stream() and the Streams globals are missing, so SPZ and progressive 3D loaders cannot run #1990

Open

Description

@matthargett

Summary

#1825 tracks the SPZ loader symptom (ReadableStream is not defined, 3 scenes). This issue tracks the platform gap behind it, which is wider than one global and affects every large-asset loader, not only SPZ.

The fetch/Blob stack has no streaming surface at all: Response has no body and is not a constructor, Blob has no slice() or stream(), and there are no ReadableStream, DecompressionStream or TextDecoderStream globals. Bodies are fully buffered before the promise resolves. The first byte of an asset becomes visible only after its last byte has arrived, and loaders written against the standard streaming APIs throw.

For large 3D assets this removes the pipeline that makes them loadable at all on mobile: header sniffing before the payload lands, decode of chunk n overlapped with download of chunk n+1, GPU upload of early chunks while later ones are still in flight, and progressive display of a splat scene or glTF. Instead the JS thread sits idle during the download, then does the entire decode in one burst.

What is missing (BabylonNative master @ e79a4e3, Dependencies/JsRuntimeHost)

  • Response is a plain object exposing text/arrayBuffer/json/blob/clone only; no body, no bytes(): Fetch.cpp#L251-L334. There is no global Response constructor, so new Response(...) is a ReferenceError.
  • Blob exposes size/type/text/arrayBuffer/bytes only; no slice(), no stream(): Blob.cpp#L17-L21. File likewise: File.cpp#L46-L48.
  • No Streams polyfill of any kind under Polyfills/ (AbortController, Blob, Console, Fetch, File, Performance, Scheduling, TextDecoder, TextEncoder, URL, WebSocket, XMLHttpRequest).
  • Full buffering before resolve is documented as the contract: Fetch/Readme.md#L18.

Who breaks

  • SPZ (Niantic's compressed splat container, now the common interchange for splat scenes): Babylon.js's non-WASM decode path builds a ReadableStream, pipes it through new DecompressionStream("gzip"), and reads the result with new Response(stream).arrayBuffer() (splatFileLoader.pure.ts#L448-L458). Here that throws, the catch falls through to the PLY parser, and the file is mis-parsed.
  • WebRequest.FetchAsync's XHR fallback wraps the result in new Response(request.response, ...) (webRequest.ts#L138); it is used by the scene loader's HEAD probe (sceneLoader.ts#L691), audio loading and the flow-graph parser.
  • Streamed splat LODs download whole files per LOD with no partial consumption (gaussianSplattingDownloadManager.ts#L310); with a streamed body the decoder could start on the first chunk and the renderer could show the coarse LOD while fine ones arrive.
  • glTF/GLB, KTX2 and USDZ readers that inspect a header or table of contents before committing to the full payload have no way to do so.

Proposed shape

  • response.body: a ReadableStream whose chunks come from UrlLib's progress/data callbacks with backpressure honored by the transport. Reads should settle from the transport's completion callback, not from a timer that polls on the JS thread; the goal is to keep this pipeline off the CPU as much as possible so the GPU keeps its headroom.
  • bytes(), and a constructible Response accepting BufferSource | Blob | ReadableStream | string, so new Response(stream).arrayBuffer() works as the standard streaming-to-buffer idiom.
  • Blob.slice() as an offset/length view over a shared immutable backing (zero copy), and Blob.stream().
  • The streams primitives (ReadableStream, WritableStream, TransformStream, DecompressionStream/CompressionStream, TextDecoderStream) as an optional polyfill module, so hosts that bring their own can skip it.

PRs BabylonJS/JsRuntimeHost#208, #209, #210 and #211 (opened against the now-archived JsRuntimeHost repository) implement most of this (WHATWG Streams, Blob slicing and streaming, Headers plus streamed Response bodies, Compression Streams) and can serve as the reference for the contract above even if the final implementation differs.

Checked the tracker first: #1825 covers the SPZ symptom only; JsRuntimeHost#98 (Implement fetch API) and JsRuntimeHost#213 (blob: URL resolution) predate the gap described here. Related: #GraphQL: Repository was archived so is read-only (createIssue).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions