chore(deps): update dependency flashinfer-python to v0.6.14

[renovate]

This PR contains the following updates:

Package	Change	Age	Confidence
flashinfer-python	==0.6.12 → ==0.6.14

---

Release Notes

flashinfer-ai/flashinfer (flashinfer-python)

v0.6.14

Compare Source

Manual Intervention Needed ⚠️

Please modify your install instructions as you bump the version to 0.6.14.

pip install flashinfer-python
 
pip install flashinfer-cubin --index-url https://flashinfer.ai/whl     # this is the difference
 
pip install flashinfer-jit-cache --index-url https://flashinfer.ai/whl/cu129

# OR 
pip install flashinfer-jit-cache --index-url https://flashinfer.ai/whl/cu130

See also #​3808

Highlights

This release pushes FlashInfer's coverage onto Blackwell RTX PRO and DGX Spark silicon, lands the CuTe-DSL rewrite of the gated-delta-rule (GDN) kernels behind Qwen3.5/3.6, and makes those kernels production-ready inside vLLM.

Gemma 4 and W4A16 on Blackwell RTX Pro and DGX Spark

Blackwell SM12x parts (RTX PRO 6000, DGX Spark / GB10) previously trailed Blackwell GB200 NVL72 (and B300 SM103) on attention shape and quantization coverage. This release closes the gap for Gemma 4 and weight-only-quantized inference: head_dim=512 attention for Gemma 4's global layers now runs on SM120/121, the FMHAv2 prefill path gains head_dim=256/512 and sliding-window masking on SM120, the new mm_bf16_fp4 W4A16 GEMM is tuned for DGX Spark (completing W4A16 across both dense GEMM and MoE), and gated tanh-GELU brings Gemma 4 MoE onto the CUTLASS backend.

• #​3576
• #​3518
• #​3597
• #​3501

GDN / gated delta rule: CuTe-DSL overhaul (Qwen3.5/3.6)

The gated-delta-rule kernels behind the Qwen3.5/3.6 family were rewritten from CUTLASS C++ to CuTe-DSL (#​3491). The rewrite eliminates the C++ JIT compilation pain reported by customers and establishes the base for context-parallel delta-rule kernels — covering SM90 prefill and its context-parallel variant, SM120 prefill, and a ~20–25% GDN prefill speedup from mainloop efficiency work.

• #​3477
• #​3481
• #​3479
• #​3742

GDN production-ready in vLLM

Two gaps blocked GDN serving in vLLM (#​3602); both are now resolved. GDN kernels are compilation batch-size agnostic, so a single compiled cubin is reused across batch shapes instead of recompiling on every new batch size at inference time, and a new BF16 state recovery/decode kernel writes SSM state into preallocated space to supply the MTP-compatible spec-decode path vLLM needs.

• #​3649
• #​3502

DeepSeek-class sparse MLA on Blackwell, FP8 KV on Hopper

New sparse-MLA paged-attention kernels extend the DeepSeek-V4 (d_qk=512) and DeepSeek-V3.2 / GLM-5.1 (d_qk=576) families onto SM120/121 through the existing flashinfer.mla APIs, with DSv4 coverage broadened to 8/16/32 head counts. On Hopper SM90, native FP8 KV cache support eliminates SGLang's per-layer cast workaround, saving an HBM round-trip per layer on DeepSeek-V3/V4 while staying bit-identical to the BF16 path.

• #​3395
• #​3545
• #​3694

This release pushes FlashInfer's coverage onto Blackwell RTX PRO and DGX Spark silicon, lands the CuTe-DSL rewrite of the gated-delta-rule (GDN) kernels behind Qwen3.5/3.6, and makes those kernels production-ready inside vLLM.

What's Changed

• fix: use explicit .ptr on scalar struct fields in monolithic MLA decode to get cleaner logs by @​jdebache in #​3458
• fix: One sided MOE A2A warp token policy hangs in some cases, disable… by @​djns99 in #​3371
• test: correct XQA NVFP4 skip reason to SM120/SM121 by @​deng451e in #​3510
• fix XQA NVFP4 head dim by @​Njuapp in #​3534
• tests: split test_trtllm_gen_attention.py into prefill / decode / decode-xqa shards by @​bkryu in #​3162
• docs(misc): expose DiT/RoPE norms; new RSTs for GDN decode/prefill/Mamba by @​kangbintNV in #​3446
• perf: cache cudaGetDeviceProperties in CudaDevice (fmha_v2) by @​aws-jiadingg in #​3522
• FMHAv2 on SM120 for head_dim 256/512 + sliding-window masks by @​dbari in #​3518
• feat: add FlashInfer Trace Apply by @​yongwww in #​3240
• Fix smem race in FilteredTopK overflow refinement by @​awgu in #​3529
• Extend Require Workspace Size and Disable Overlapping Criritcal Workspace Sections w/ 'auto' backend + autotuning for MLA decode by @​Vinnie6167 in #​3465
• perf(sampling): Optimize top_k_top_p_sampling_from_logits/from_probs for large-vocab small-k sampling by @​bkryu in #​3461
• Use an environment variable to control the number of reserved SMs for overlapping in TRT-LLM fused MoE by @​jinyangyuan-nvidia in #​3483
• Remove excessive CuteDslMoEWrapper memory allocation by @​nvjullin in #​3404
• docs: close v0.6.13 doc-check gaps + fix(moe) misleading topk_indices ICHECK message by @​kangbintNV in #​3546
• Unified MoE API: MoELayer with cross-backend NVFP4 autotune by @​aleozlx in #​3093
• Support smaller DSv4 sparse MLA head counts by @​PerkzZheng in #​3545
• feat(bench): add GDN routines to flashinfer_benchmark.py by @​bkryu in #​3572
• Add BF16 MoE SwiGLU OA params by @​ruoqianguo in #​3532
• perf(gemm): update mm_fp4 b12x SM120 NVFP4 dense GEMM kernel by @​yichengj0 in #​3560
• feat: add sm90 delta rule dsl prefill by @​guangyunh-nv in #​3477
• fix: headDim=512 GQA decode (#​3343) by @​djmmoss in #​3393
• Add Qidi Sang to CODEOWNERS by @​saltyminty in #​3593
• [feat] Add gated tanh-GELU (GeluTanh) activation to CUTLASS fused MoE (GEMMA 4) by @​jhaotingc in #​3501
• [chore] Add jiahanc to GDN commiter by @​jiahanc in #​3600
• fix(ci): skip TRT-LLM Gen BF16 MoE SwiGLU test outside SM10.x by @​ruoqianguo in #​3580
• fix(mla): warn when backend='auto' falls back to non-Blackwell kernel on SM>=100 by @​elwhyjay in #​3405
• [fix] Fix SM100 GDN prefill hang by @​jiahanc in #​3581
• feat: improve gated delta rule benchmark script by @​guangyunh-nv in #​3616
• feat: delta rule work with zero length sequence by @​guangyunh-nv in #​3536
• feat(cutile): introduce cuTile backend (mm_bf16 + bmm_bf16 + gemm_fp8_nt_groupwise) by @​yifeis-nv in #​3426
• chore: remove leftover cpp srcs from #​3477 by @​guangyunh-nv in #​3613
• feat: add mxfp8 quant to moe a2a combine by @​IwakuraRein in #​3376
• fix: scope trtllm-gen block_size guard to its own backend by @​elwhyjay in #​3428
• feat(attention): add SM120 sparse MLA kernels by @​lucifer1004 in #​3395
• test: match test output scaling with cutedsl kernel (scale before bf16 cast) by @​saltyminty in #​3596
• fix(sampling): fix shared-memory race and out-of-range token id in SamplingFromLogitsKernel by @​kahyunnam in #​3624
• feat(attention): head_dim=512 support for attention prefill & decode for Gemma 4 on SM120/121 by @​bkryu in #​3576
• [fix] explicitly validate grouped_gemm_nt_masked layout contract to prevent NaN contamination by @​kahyunnam in #​3574
• fix(gemm): relax b12x FP4 K constraint from 128 to 32 (TMA alignment) by @​yichengj0 in #​3646
• Add MXFP8 MoE SwiGLU OA parameters by @​ruoqianguo in #​3504
• Add support for non-multiple VEC_COLS in fused_rmsnorm_silu for bf16/fp8 by @​xueweilnvidia in #​3417
• fix(topk): eliminate multi-CTA radix top-k stream hangs on SM120/SM121 by @​waynehacking8 in #​3615
• feat: add sm120 delta rule dsl prefill by @​guangyunh-nv in #​3479
• Add SM120 NVFP4 attention JIT path by @​tiffany940107 in #​3640
• Add MXFP8 MoE GEMM entry (cute SM120 backend) by @​CarstyYou in #​3562
• feat: Add BF16_FP4 GEMM with cuDNN and CuTe-DSL backends for SM120/121 for W4A16 workloads by @​bkryu in #​3597
• perf(gdn): make GDN kernels compilation batch-size agnostic (support dynamic batch shapes for vLLM integration) by @​kahyunnam in #​3649
• feat: add sm90 cp delta rule dsl by @​guangyunh-nv in #​3481
• fix(sampling): terminate top-p search at adjacent float bounds by @​djmmoss in #​3623
• fix: fix tinygemm barrier bug by @​yweng0828 in #​3630
• feat(gdn): BF16 state recovery/decode kernel with per-request K and f… by @​ameynaik-hub in #​3502
• feat(benchmark): add MLA --mla_is_var_seq / --mla_cute_dsl_impl knobs by @​lunarz-dev in #​3695
• Enable SwapAB in mm_fp4 cute-dsl backend when M is not a multiple of 8. by @​b8zhong in #​3667
• fix(moe): Fix unbounded weight-cache growth in b12x MoE by @​bkryu in #​3709
• Add Relu2 + ungated MoE to CuteDSL MoE by @​b8zhong in #​3642
• trtllm_batch_decode_with_kv_cache_mla trtllm-gen backend cum_seq_lens_q support by @​saltyminty in #​3238
• fix: pass XQA NVFP4 scale-factor strides by @​Njuapp in #​3608
• feat(attention): FP8 KV cache support for Hopper SM90 MLA by @​qsang-nv in #​3694
• [GDN] sm100: support more state dtype by @​Observer007 in #​3715
• feat: cuTile Grouped MXFP8 Quantization by @​philipphack in #​3657
• Accept uint8 workspaces in CUTE DSL MLA decode by @​leejnau in #​3599
• tests: split test_trtllm_gen_fused_moe.py into shards by @​feih-nv in #​3635
• perf(moe): Enhance CuteDSL NVF4 MOE Perf by @​liyuhannnnn in #​3564
• fix(mamba): reject SM120/SM121 in SSDCombined with a clear error by @​waynehacking8 in #​3668
• docs(gemm): add missing .rst entries for mm_bf16_fp4 and prepare_bf16_fp4_weights by @​kangbintNV in #​3710
• docs: add missing parameter entries to docstrings and env vars by @​kangbintNV in #​3627
• rename back by @​aleozlx in #​3730
• docs: document scale_major_mode param and FLASHINFER_AUTOTUNE_DIR env var by @​kangbintNV in #​3696
• feat: add mxfp4/nvfp4 quant to moe a2a combine by @​IwakuraRein in #​3643
• chore: fix enable_pdl for trtllm-gen routing and finalize kernel by @​IwakuraRein in #​3588
• Bug fix (gdn): Layout contract fix from #​3649 by @​kahyunnam in #​3693
• Ameyn/fix fp32 mtp pool out indices by @​ameynaik-hub in #​3490
• Prune moe tests by @​aleozlx in #​3733
• Fix the output allocation consistency of trtllm-gen MoE APIs by @​b8zhong in #​3678
• feat(comm): Support per-token LoRA Info in MoE a2a comm payloads by @​JyChang012 in #​3375
• docs(gdn): document missing ssm_state_indices param in gated_delta_rule_mtp by @​kangbintNV in #​3725
• feat(moe): enable DSFp8 + LoRA delta path by @​zetacat in #​3708
• Improve GDN prefill perf by ~20-25% (mainloop efficiency) by @​jhjpark in #​3742
• test: fix CUDA OOM in batch-prefill custom-mask test on 24GB CI GPUs by @​waynehacking8 in #​3609
• Remove pagesize 16/32 assertion from xqa nvfp4 sm120 by @​Njuapp in #​3724

New Contributors

• @​deng451e made their first contribution in #​3510
• @​Njuapp made their first contribution in #​3534
• @​aws-jiadingg made their first contribution in #​3522
• @​awgu made their first contribution in #​3529
• @​ruoqianguo made their first contribution in #​3532
• @​yichengj0 made their first contribution in #​3560
• @​jhaotingc made their first contribution in #​3501
• @​elwhyjay made their first contribution in #​3405
• @​yifeis-nv made their first contribution in #​3426
• @​lucifer1004 made their first contribution in #​3395
• @​waynehacking8 made their first contribution in #​3615
• @​tiffany940107 made their first contribution in #​3640
• @​CarstyYou made their first contribution in #​3562
• @​feih-nv made their first contribution in #​3635
• @​liyuhannnnn made their first contribution in #​3564
• @​JyChang012 made their first contribution in #​3375
• @​zetacat made their first contribution in #​3708

Full Changelog: flashinfer-ai/flashinfer@v0.6.13rc2...v0.6.14

v0.6.13

Compare Source

What's Changed

• Run high-likelihood OOM culprits separately, record memory usage and test duration for analysis by @​dierksen in #​2961
• fix(autotuner): differentiate file cache entries by runner specific kernel parameters by @​qiching in #​3367
• feat: integrate cute-dsl Blackwell GQA decode into BatchDecodeWithPagedKVCacheWrapper by @​richardmcai in #​3360
• Fix returning reference to temporary in moe gemm by @​benbarsdell in #​3332
• Enable compression of GPU device binaries by @​benbarsdell in #​2949
• feat: MNNVL Allreduce quant fusion and performance optimization by @​timlee0212 in #​3385
• fix(norm): widen address arithmetic to int64 for large contiguous inputs > 2**31 elements by @​bkryu in #​3392
• MLA Decode Autotuning Across TRTLLM-Gen and CuTe Backends by @​Vinnie6167 in #​3355
• profiler: group perfetto traces by SM, one row per block by @​Edenzzzz in #​3038
• Make cute dsl mxfp8/nvfp4 quantizer bitwise exact by @​zianglih in #​3387
• Extend autotuner delay kernel length by @​yanqinz2 in #​3373
• Enable smaller tile N for SM100 Cute-DSL NVFP4 GEMM by @​b8zhong in #​3403
• test: align test_fmha_v2_prefill SM gating with is_sm12x_supported by @​leonardHONG in #​3182
• feat(autotuner): enable per-op autotune bypass for faster framework warmup by @​qiching in #​3396
• bench: Unify PDL behavior, add missing norm routines, and misc improvements by @​bkryu in #​3435
• feat(kda): add recurrent KDA decode kernel with per-K gating by @​djmmoss in #​2572
• Add mHC post mapping and pre big-fuse kernels by @​jmydurant in #​3285
• [Feat] Add num_heads < 128 support for mla decode kernel by @​Observer007 in #​3309
• Fix cross-warp race in checkpointing SSU kernel (mamba) by @​ishovkun in #​3439
• feat(trace): add check callbacks to trace templates by @​yyihuang in #​3330
• Add BGMV MoE CUDA kernels for multi-LoRA by @​taehokim20 in #​3249
• replace deprecated APIs: cute.make_fragment and cute.core.ThrMma by @​brandon-yujie-sun in #​3430
• [ci feat] Support /bot run tests/ to scope CI test runs by @​kahyunnam in #​3422
• perf(attention): Speed up FP8 KV-cache prefill (FA2 BatchPrefill) by repacking K/V to BF16 in shared memory by @​bkryu in #​3485
• Ep api design -- Adding the actual code and tests by @​Anerudhan in #​3453
• Optimize mxfp8 quantization on sm100 by @​IwakuraRein in #​3289
• Fix silent no-op autotuning for cuBLAS bmm_fp8 and cuDNN bmm_fp8/mm_fp4 by @​bkryu in #​3437
• fix(quantization): nvfp4_quantize(backend='cuda') silently corrupts scale factors when global_scale is not float32 by @​bkryu in #​3497
• feat(moe): add SWIGLUSTEP activation to CUTLASS fused MoE by @​bkryu in #​3492
• Support LSE buffers in TRTLLM API by @​saltyminty in #​3410
• NFC: replace deprecated API: cute.make_fragment by @​brandon-yujie-sun in #​3473
• feat(moe): write routing_replay_out from custom routing kernels by @​jdebache in #​3382
• Add CuTe DSL NVFP4 quantization with 4over6 FP16 scoring by @​zianglih in #​3448
• fix intermittent exit 141 (SIGPIPE) in test resource summary by @​yongwww in #​3498
• add_cudnn_mxfp8 by @​yanqinz2 in #​3489
• fix: use routedScalingFactor to initialize mRouteScale by @​yweng0828 in #​3499
• fix: Simple approach to restore support for bias for fp4 block scale types by @​djns99 in #​3416
• docs(infra): document env vars, refresh SM list, fix stale paths in CLAUDE.md by @​kangbintNV in #​3440
• docs(quant/sampling/activation): canonicalize quantization.rst, stub fp4_quantization.rst by @​kangbintNV in #​3447
• docs(gemm): document GEMM + grouped_mm public surface; canonicalize aliases by @​kangbintNV in #​3442
• Optimize MoE routing top-k reduction and non-power-of-two sorting by @​jiahanc in #​3476
• [bugfix] TorchDistBackend.bcast uses global rank instead of local rank by @​xuanyu-mistral in #​3418
• docs(comm): structural RST refactor for MoeAlltoAll/DCP/Mixed comm by @​kangbintNV in #​3445
• [docs] Backfill missing docstrings and decorators across kernels by @​kangbintNV in #​3456
• docs(attention): backfill missing/stale Attention/POD/cuDNN/CuteDSL APIs; restore single_prefill_with_kv_cache_return_lse by @​kangbintNV in #​3441
• docs(moe): close fused_moe / trtllm_*_moe / CuteDSL MoE doc gaps by @​kangbintNV in #​3443
• docs(comm): NumPy-style docstrings + Deprecated leads for 21 STALE comm APIs (no decorator changes) by @​kangbintNV in #​3444
• bump version to 0.6.13 by @​aleozlx in #​3513

New Contributors

• @​richardmcai made their first contribution in #​3360
• @​jmydurant made their first contribution in #​3285
• @​taehokim20 made their first contribution in #​3249
• @​brandon-yujie-sun made their first contribution in #​3430
• @​xuanyu-mistral made their first contribution in #​3418

Full Changelog: flashinfer-ai/flashinfer@v0.6.12rc3...v0.6.13

---

Configuration

📅 Schedule: (UTC)

• Branch creation
  • At any time (no schedule defined)
• Automerge
  • At any time (no schedule defined)

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

---

• If you want to rebase/retry this PR, check this box

---

This PR was generated by Mend Renovate. View the repository job log.