Skip to content

Fused Vimoo 1.0 and 1.0-Lite blocks (0.17.0) - #27

Merged
iamwavecut merged 1 commit into
mainfrom
feat/vimoo-family
Nov 9, 2026
Merged

iamwavecut merged 1 commit into
mainfrom
feat/vimoo-family

Conversation

@iamwavecut

Copy link
Copy Markdown
Owner

Adds the fused block family for Vimoo 1.0 and Vimoo 1.0 Lite (VimooTransformer3DModel, the Vimoo-AI Diffusers fork) and sets the version to 0.17.0.

Changes

  • orbitquant.fused.families.vimoo: the 40 dual-stream blocks. Per stream, LayerNorm and the (1 + scale) * x + shift modulation run in the 8-bit activation prologue, Q|K|V is one GEMM, Q/K RMSNorm and the interleaved video RoPE are one kernel, and both output projections and the feed-forward output add their gated residual in the GEMM epilogue. The feed-forward input GEMM applies GELU in its epilogue: tanh for standard checkpoints, exact (erf) for Lite.
  • The transformer sorts the valid text tokens of each sample first. The fused blocks attend over [video | valid text] only, and padded text rows leave the block unchanged.
  • Lite (numerical_profile="lite"): residual streams stay in the dtype the transformer hands over (FP32), the separately stored output and feed-forward biases are added in the epilogues, and the observed first frame of image-to-video sampling reads row 1 of two-row modulation tables (mod_index/gate_index).
  • Standard: the direct patch embedding hands the blocks a column-major video stream. The blocks work on row-major copies.
  • EPILOGUE_GELU_ERF (exact GELU) in the INT8 GEMM and in the BF16 fallback.
  • FusedRuntime.linear(..., bias=...) adds a bias stored outside the group.
  • matmul_int8_scaled_with_triton rejects an out without unit column stride. Before, such an output was written as if it were row-major.
  • Feed-forward rows run in chunks of 32768, which bounds the [rows, 12288] intermediate of long videos.
  • GPTQ reports an error of 0 for a projection whose calibration inputs are all zero. Before, the relative error was 0/0 (NaN), and a NaN in a report makes invalid JSON. The codes do not change: GPTQ already sets the weights of such inputs to 0. Vimoo-1.0 image-to-video has one such projection (the feed-forward output of the first block's video stream: every GELU input is below -5.7).
  • The version is 0.17.0.

Checks

Parity on captured transformer calls, nearest rounding, against the eager OrbitQuant model and the BF16 original:

Model, GPU Video tokens, batch Fused vs eager W4A8 Eager W4A8 vs BF16 Fused vs BF16 Time per forward: BF16 / fused cuDNN / fused INT8 attention
Lite T2V+I2V, RTX PRO 6000 8.3k, 2-3 1.8-2.6 % 15-26 % 15-26 % 2.34 / 0.73 / 0.68 s (step 0)
Vimoo-1.0 T2V, RTX 5090 14.3k, 2 1.5 % 6.5-6.8 % 6.6-6.9 % 5.59 / 1.49 / 1.09 s
Vimoo-1.0 I2V, RTX 5090 14.3k, 2 2.7-4.6 % 15-20 % 15-21 % 5.63 / 1.59 / 1.20 s

GPTQ rounding lowers the quantization error further (release artifacts: mean group error 0.0094 -> 0.00054 for the Lite transformer).

  • tests/test_fused_vimoo.py: groups, layouts, padding and both profiles against the eager block (CUDA), with a column-major video input.
  • tests/test_gptq.py::test_projection_without_input_reports_zero_error.
  • ruff check ., pytest -ra, scripts/run_paper_methodology_checks.sh, scripts/run_hf_compat_checks.sh --mode all: pass locally.
  • scripts/run_cuda_kernel_checks.sh: pass on an RTX 5090 (Torch 2.12.1 + CUDA 13.0, Triton 3.7.1).

Adds orbitquant.fused.families.vimoo for VimooTransformer3DModel (Vimoo-AI
Diffusers fork), exact-GELU GEMM epilogue, FusedRuntime.linear bias override,
a unit-stride check on INT8 GEMM outputs, and a finite GPTQ error report for
projections whose calibration inputs are all zero.
@iamwavecut
iamwavecut merged commit 6521227 into main Nov 9, 2026
3 checks passed
@iamwavecut
iamwavecut deleted the feat/vimoo-family branch November 9, 2026 16:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant