SparkRing is a vLLM-based inference-serving stack with low-latency collective communication for switchless clusters of NVIDIA GB10-based devices.
SparkRing supports pairs, four-node rings, and six-node rings (in dev).
The collective communication stack combines SIRCL, RoCEnante, and patched NCCL. The high-speed data fabric needs no external Ethernet or InfiniBand switch; administration and vLLM api serving occur over a node/s 10Gbe NIC.
Four- and six-node rings use a virtual mesh built on custom RoCE RDMA routing and hardware forwarding in the ConnectX network ASICs. This creates paths between nodes that aren’t directly connected, carrying traffic over the existing ring cables without routing it through host CPUs. The result is mesh connectivity over a physical ring.
The repository provides setup guides, launch tooling, model profiles, reproducible benchmarks, and test results.
SparkRing is experimental. This repo is changing rapidly.
- Choose a profile and check the prerequisites.
- For a shared-image GLM four-node ring, follow the mesh host setup guide. Other four-node profiles use the bootstrap guide. Two-node profiles include their own direct-link setup.
- Follow the profile's quickstart, then run the validation checks.
KV is approximate total token capacity. The shared TP4 figure is for DCP1;
the TP2 figure is a reference estimate. — means no capacity
is recorded. Startup reports the actual capacity, which is separate from the
per-request Context limit.
| Model / predictor | Serving stack | Transport | Layout | Context | Sequences | KV (tokens) | Guide |
|---|---|---|---|---|---|---|---|
| GLM-5.3 Flash NVFP4-Spark · native MTP3 | Shared SparkRing image | Mesh + dual-domain NCCL | TP4/DCP1; DCP4 option | 1M | 16 | ~2.3M | Quickstart |
| GLM-5.3 Flash NVFP4-Spark · MTP3 + SparkCache | Shared SparkRing image | Mesh + dual-domain NCCL | TP4/DCP1 | 1M | 16 | ~2.3M | Cache profile selection |
| GLM-5.3 Flash NVFP4-Spark · MTP3, switched | Shared SparkRing image | Operator-selected NCCL links | TP4/DCP1 | 1M | 16 | — | Switched quickstart |
| GLM-5.2 EXL3 3.5-bpw | SparkRing vLLM/ExLlamaV3 build | SIRCL + NCCL | TP4/DCP4 | 1M | 16 | ~1.2M | Quickstart |
| DeepSeek-V4-Flash-0731 | SparkRing vLLM/B12X image | Patched NCCL | TP4/DCP1 | 1M | 32 | ~1M | Quickstart |
| Qwen3.8-27B EXL3 K5/K6 | SparkRing vLLM/ExLlamaV3 build | Patched NCCL | TP4/DCP1 | 1M | 64 | — | Quickstart |
| DeepSeek-V4-Flash-Vision-Exp with DSpark (research-only) | Anemll image / MiaAI-Lab recipe | SparkRing patched NCCL | TP4 | 1M | 48 | — | Quickstart |
The Vision-Exp artifact contract identifies the Anemll image, MiaAI-Lab recipe, and SparkRing transport separately. Contributor-reported results are linked from the guide; independent reproduction of the selected artifacts is not claimed.
The shared-image GLM profiles enable continuation coalescing and token-sharded mHC. Ring profiles use hardware-forwarded mesh paths and NCCL across both host PCIe domains. Their source integration is implemented; full shared-image serving qualification remains research-only. Switched deployments are provided as-is and have not been validated on switched hardware. Shared-image GLM ring deployment requires the managed-mesh setup.
| Model / predictor | Serving stack | Transport | Layout | Context | Sequences | KV (tokens) | Guide |
|---|---|---|---|---|---|---|---|
| GLM-5.3 Flash NVFP4-Spark · native MTP3 | Shared SparkRing image | Adaptive RoCEnante + dual-domain NCCL | TP2/DCP1 | 256K | 8 | ~1M | Quickstart |
| DeepSeek-V4-Flash-0731 | SparkRing vLLM/B12X image | Patched NCCL | TP2/DCP1 | 1M | 32 | ~1M | Quickstart |
| Qwen3.8-27B EXL3 K5/K6 | SparkRing vLLM/ExLlamaV3 build | Patched NCCL | TP2/DCP1 | 1M | 32 | — | Quickstart |
The NVFP4-Spark pair uses 8.75 GiB KV per rank, one DAC, both host PCIe domains, coalescing and mHC. SparkCache is unsupported for this TP2 composition. Shared-image GPU qualification remains research-only. The NVFP4-Spark reference deployment reported 1,050,118 KV tokens; that is capacity, not a tested maximum workload.
See the profile index for evidence scopes and SparkCache compositions for persistent-cache support.
Qwen with SparkCache is unsupported; six-node profiles are research-only.
These guides retain their pinned configurations and evidence for reproduction. For deployment with the shared image, use the matching two- or four-Spark entry above.
| Profile | Layout | Retained guide | Replacement |
|---|---|---|---|
| NVFP4-Spark MTP3 cache/checkpoint mesh | TP4/DCP4 | Pinned cache/checkpoint setup | Shared-image MTP3 profiles |
| NVFP4 with BF16 DFlash2 | TP4/DCP1, DCP2 or DCP4 | Pinned DFlash2 setup | Shared-image native MTP3 |
| NVFP4-Spark MTP3 with 5 GiB KV per rank | TP2/DCP1 | Pinned TP2 setup | Shared-image NVFP4-Spark TP2 |
| Original NVFP4 MTP3 with 6.75 GiB KV per rank | TP2/DCP1 | Pinned original-NVFP4 setup | Shared-image NVFP4-Spark TP2 |
The retired 5 GiB TP2 configuration has a recorded blue-video recognition issue.
DFlash2 uses a separate draft checkpoint with CC BY-NC-ND 4.0 terms.
| Package / runtime | Profile | Details |
|---|---|---|
ghcr.io/fujitsupolycom/sparkring |
Shared GLM TP2/TP4 image; profile selects topology and optional cache | Source build and profile verification |
gb10-vllm-serving |
Profile-specific images, including DeepSeek | Packages |
Anemll dspark-vllm-gx10 |
DeepSeek-V4-Flash-Vision-Exp with the MiaAI-Lab recipe | Image, recipe, and transport provenance |
Use the exact digest in the selected quickstart. Images sharing a package name are not interchangeable; a model-neutral name does not qualify every profile. Retired profiles retain their image references in their linked guides. The shared-image publication record contains the download digest and profile verification scope.
Each row links the exact measured configuration. These records include retired profiles and are not benchmark results for the shared-image build.
Decode is sustained aggregate output at temperature 1.0. Results attempt to reflect real world use-case numbers in all instances unless otherwise noted. structured data sweeps,temperature 0 and/or other out-of-spec configurations are not provided or recommended
| Profile | Decode context | Prefill | C1 decode | C8 decode | Highest C at this context | Coding peak |
|---|---|---|---|---|---|---|
| GLM-5.3 NVFP4-Spark · native MTP3 + mesh · 4 Sparks | 8K | 2,703 (8K scout) | 48.2 | 168.8 | C16: 231.3 | — |
| GLM-5.3 NVFP4-Spark · DFlash2 exact request-batch graphs · 4 Sparks | 16K | 2,717 (16K scout) | 43.05 | 134.3 | C16: 187.0 | — |
| GLM-5.3 NVFP4 · DFlash2/B12X-KDA DCP4 · 4 Sparks | 16K | 2,649 (16K scout) | 37.97 | — | C4: 90.36 | — |
| GLM-5.2 EXL3 3.5-bpw · 4 Sparks | 16K | 671 (16K) | 20.15 | 64.13 | C8: 64.13 | 25.39 |
| DeepSeek-V4-Flash DSpark · 2 Sparks | 16K | 1,926 (16K) | 58.36 | 162.69 | C32: 307.13 | 59.31 |
| DeepSeek-V4-Flash-0731 · 4 Sparks | 16K | 2,488 (16K) | 68.84 | 265.16 | C32: 508.11 | 95.77 |
| Qwen3.8-27B EXL3 K5/K6 · 2 Sparks | 16K | 1,367 (16K) | 29.50 | 142.20 | C16: 184.39 | 39.95 |
| Qwen3.8-27B EXL3 K5/K6 · 4 Sparks | 16K | 1,964 (16K) | 35.07 | 191.02 | C8: 191.02 | 48.46 |
See full results and the mesh validation report for repeat counts, accuracy checks, settings, and limitations.
Architecture · SIRCL · RoCEnante · Mesh prerequisites
Validation · Deployment tooling · Contributing · Discussions
| Path | Purpose |
|---|---|
spark_transport/ |
Communication backends and vLLM adapters |
runtime/ |
Pinned images, builders, and serving profiles |
scripts/ |
Preflight, deployment, and validation tools |
recipes/ |
Machine-readable serving recipes |
performance/ |
Measurement methods, evidence, and receipts |
docs/ |
Operator guides and architecture |
integrations/lil/ |
Companion lifecycle and deployment integration |
Built on vLLM, NVIDIA NCCL, B12X, ExLlamaV3, and the local inference community. Luke and Local Inference Lab's RoCEnante implementation and vLLM integration underpin the adapted mesh communication. See third-party notices.
SparkRing code is Apache-2.0. Model weights and bundled components retain their own terms; review the selected model cards and third-party notices before deployment.