Repository navigation
VORTEXM4: packing dominates SME2 level-3 at small n #6146
Description
On VORTEXM4 (the SME2 level-3 kernels from #6074), packing dominates the run time of SYMM, SYRK, SYR2K, TRMM and TRSM at n <= 512.
Measured on an M4 Pro, one thread, sizes m=512, n=1024, column-major:
| copy routine | GB/s | SME2 fp32 flops in the same time per element |
|---|---|---|
sgemm_oncopy (generic gemm_ncopy_16.c, B side) |
28 | ~215 |
sgemm_otcopy (generic gemm_tcopy_16.c, B side) |
36 | ~165 |
sgemm_incopy (gemm_ncopy_sme2.c, A side) |
25 | ~245 |
sgemm_itcopy (gemm_tcopy_sme2.c, A side) |
49 | ~125 |
strmm_iltncopy (trmm_tcopy_sme2.c) |
43 | ~140 |
Each packed element is used for about 2n flops, so at n = 256 packing costs about as much as the arithmetic. A sampling profile at n = 256 (one thread) shows sgemm_otcopy + memmove at about 70% of SSYRK, and sgemm_oncopy + strmm_iltncopy at about 80% of STRMM. Compared with Accelerate on the same machine, STRMM is at 0.09x / 0.20x and SSYRK at 0.18x / 0.36x at n = 64 / 256.
Possible work, all in kernel/arm64/ and KERNEL.VORTEXM4:
- NEON (or ZA-transpose) B-side copies for
S/DGEMMONCOPYandS/DGEMMOTCOPYwith the generic tail layout (16, then 8, 4, 2, 1 columns), which the symm/trmm/trsm OUTER copies also use. - A faster transposing A-side copy (
gemm_ncopy_sme2.c), for example with the ZA transposition ofs2_pack_tinsme2_gemm_impl.h. - Replace the per-row
memcpyofs2c_rows()insme2_gemm_copy.hwith inline copies of up to 64 elements.
Activity