Repository navigation
Fused Vimoo 1.0 and 1.0-Lite blocks (0.17.0) - #27
Merged
Merged
Conversation
Adds orbitquant.fused.families.vimoo for VimooTransformer3DModel (Vimoo-AI Diffusers fork), exact-GELU GEMM epilogue, FusedRuntime.linear bias override, a unit-stride check on INT8 GEMM outputs, and a finite GPTQ error report for projections whose calibration inputs are all zero.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the fused block family for Vimoo 1.0 and Vimoo 1.0 Lite (
VimooTransformer3DModel, the Vimoo-AI Diffusers fork) and sets the version to 0.17.0.Changes
orbitquant.fused.families.vimoo: the 40 dual-stream blocks. Per stream, LayerNorm and the(1 + scale) * x + shiftmodulation run in the 8-bit activation prologue, Q|K|V is one GEMM, Q/K RMSNorm and the interleaved video RoPE are one kernel, and both output projections and the feed-forward output add their gated residual in the GEMM epilogue. The feed-forward input GEMM applies GELU in its epilogue: tanh for standard checkpoints, exact (erf) for Lite.[video | valid text]only, and padded text rows leave the block unchanged.numerical_profile="lite"): residual streams stay in the dtype the transformer hands over (FP32), the separately stored output and feed-forward biases are added in the epilogues, and the observed first frame of image-to-video sampling reads row 1 of two-row modulation tables (mod_index/gate_index).EPILOGUE_GELU_ERF(exact GELU) in the INT8 GEMM and in the BF16 fallback.FusedRuntime.linear(..., bias=...)adds a bias stored outside the group.matmul_int8_scaled_with_tritonrejects anoutwithout unit column stride. Before, such an output was written as if it were row-major.[rows, 12288]intermediate of long videos.Checks
Parity on captured transformer calls, nearest rounding, against the eager OrbitQuant model and the BF16 original:
GPTQ rounding lowers the quantization error further (release artifacts: mean group error 0.0094 -> 0.00054 for the Lite transformer).
tests/test_fused_vimoo.py: groups, layouts, padding and both profiles against the eager block (CUDA), with a column-major video input.tests/test_gptq.py::test_projection_without_input_reports_zero_error.ruff check .,pytest -ra,scripts/run_paper_methodology_checks.sh,scripts/run_hf_compat_checks.sh --mode all: pass locally.scripts/run_cuda_kernel_checks.sh: pass on an RTX 5090 (Torch 2.12.1 + CUDA 13.0, Triton 3.7.1).