Skip to content

SPIR-V: fuse contractable multiply-adds into fma on devices with an FMA - #1010

Draft
vchuravy wants to merge 1 commit into
mainfrom
vc/spirv-fmuladd
Draft

vchuravy wants to merge 1 commit into
mainfrom
vc/spirv-fmuladd

Conversation

@vchuravy

@vchuravy vchuravy commented Oct 13, 2026 •

Copy link
Copy Markdown
Member

SPIR-V can only carry LLVM's contract fast-math flag with SPV_KHR_float_controls2, which few drivers support. Without it, the back-ends drop the flag and mark every kernel ContractionOff, so Julia's muladd reaches the driver as a separate OpFMul + OpFAdd and is never fused. On PoCL, y[i] = muladd(a[i], x[i], y[i]) compiles to vmulps + vaddps (JuliaGPU/KernelAbstractions.jl#865).

Julia's own contraction can't help here: since JuliaLang/julia#55802 (1.12), LowerSIMDLoop only marks the fmul as contract and leaves the fusing to the backend. On older Julia, CombineMulAdd did the same.

This PR:

  • adds a fma::Bool = false field to SPIRVCompilerTarget, for front-ends to set when the device has a fused multiply-add (OpenCL's CL_FP_FMA);
  • with it set, contract_muladd! (in finish_ir!) rewrites an fadd/fsub and an fmul that both allow contraction into llvm.fmuladd, if the fmul has no other use (as LLVM's instruction selection does). Both back-ends translate llvm.fmuladd to OpenCL's fma:
    • fadd (fmul a, b), c → fmuladd(a, b, c)
    • fsub (fmul a, b), c → fmuladd(a, b, -c)
    • fsub c, (fmul a, b) → fmuladd(-a, b, c)

Additions that may be reassociated are left alone. The back-ends emit reassoc as FPFastMathMode Fast, which already lets the driver contract them (PoCL fuses @fastmath x * y + z either way). Fusing them would also keep the driver from vectorizing @simd reductions, since fma can't carry the flag: a @simd dot product on PoCL went from 8-wide vector code to a scalar vfmadd231ss chain before this exclusion.

Why opt-in: OpenCL's fma must be correctly rounded, so a device without a fused multiply-add has to emulate it, which is slow.

Results on PoCL

AMD Ryzen 9 5950X (AVX2 + FMA), via JuliaGPU/KernelAbstractions.jl#892:

kernel fma = true fma = false
muladd(a[i], x[i], y[i]) vfmadd231ps (ymm), bit-identical to fma vmulps + vaddps
muladd(a, x, -y), muladd(-y, x, a) vfmsub213ps / vfnmadd213ps, bit-identical unfused
a * x + y (no contract) unfused unfused
@simd dot product vectorized, same code as fma = false vectorized

Testing

A new "contraction" testset, for both back-ends, checks:

  • both negated forms, vectors, and the result with and without fma;
  • that the rewrite leaves alone a product with another use, an addition without contract, and @fastmath code.

The SPIR-V tests pass locally (733/733).

Related:

🤖 Generated with Claude Code

… FMA

SPIR-V can only carry the `contract` fast-math flag with
SPV_KHR_float_controls2, which few drivers support. Without it, the
back-ends drop the flag and mark every kernel `ContractionOff`, so
Julia's `muladd` reaches the driver as a separate multiply and add, and
is never fused: on PoCL, `y[i] = muladd(a[i], x[i], y[i])` compiles to
`vmulps` + `vaddps`.

Add a `fma` field to `SPIRVCompilerTarget`, for front-ends to set when
the device has a fused multiply-add (OpenCL's `CL_FP_FMA`). Then
`contract_muladd!` fuses an `fadd`/`fsub` and an `fmul` that both allow
contraction, if the `fmul` has no other use, into `llvm.fmuladd`, which
the back-ends translate to OpenCL's `fma`. On PoCL, that compiles to a
vectorized `vfmadd231ps`, bit-identical to `fma`.

Additions that may be reassociated are left alone: the back-ends emit
those as `FPFastMathMode Fast`, which already lets the driver contract
them, and fusing them would keep the driver from vectorizing `@simd`
reductions, as `fma` can't carry the flag.

Assisted-by: Claude Code (Opus 5.5)
@codecov

codecov Bot commented Oct 14, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.72727% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 87.44%. Comparing base (6099ebc) to head (c367b88).
⚠️ Report is 9 commits behind head on main.

Files with missing lines Patch % Lines
src/spirv.jl 97.72% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1010      +/-   ##
==========================================
- Coverage   89.50%   87.44%   -2.07%     
==========================================
  Files          31       31              
  Lines        6905     7118     +213     
==========================================
+ Hits         6180     6224      +44     
- Misses        725      894     +169     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant