gemv

Several optimization methods of half-precision general matrix vector multiplication (HGEMV) using CUDA core.

An implementation of SGEMV with performance comparable to cuBLAS.

PCCX architecture, ISA, implementation and verification documentation. Open-source semiconductor project operated by Altifigence.

Matilda is a library to repeatedly multiply a constant matrix with a variable vector

Configurable digital near-memory computing platform for GEMV mapping, NoC communication, and endpoint reduction research.

Highly optimized DGEMV on CPU with both serial and parallel performance better than MKL and OpenBLAS.

High-performance FP32 GEMV for AMD RDNA 3 (gfx1100), with reproducible performance and numerical analysis.

Reusable PCCX v002 IP-core sources and compatibility contracts; operated by Altifigence.

Measure and visualize why LLM inference is slow: bottleneck analysis, model dissection, KV-cache, GEMM/GEMV, quantization, and memory-bound decoding.

Hand-written CUDA kernels for batch-1 LLM decode: FP16 GEMV and online softmax at the measured memory-bandwidth limit, INT4/INT8 dequant GEMV turning compression into throughput. Nsight-profiled, correctness-gated.

Software-FP8 (E5M2) weight-only GEMV: 4 codes per int32, decoded in-register inside the K-loop — 2x decode bandwidth on GPUs without FP8 hardware

FP8 (E4M3) GEMM, GEMV and fused AdamW for NVIDIA GPUs without native FP8 (sm_50–sm_87, Maxwell to Ampere): exact products, FP32 accumulation, C API and PyTorch bindings incl. torch._scaled_mm and Float8Tensor

🧮 CereMath is a library of Machine Learning kernels for the Wafer Scale Engine (WSE) from Cerebras. Made as a BSc thesis project

FP8 (E5M2 and E4M3, any pairing) GEMM, GEMV and fused AdamW for NVIDIA GPUs without native FP8 (sm_50–sm_87, Maxwell to Ampere): exact products, FP32 accumulation, TE HYBRID backward, C API and PyTorch bindings incl. torch._scaled_mm and Float8Tensor

InfiniTensor 训练营作业：OpenCL Q8_0 量化 GEMV 优化（RTX 4090 D 实测）

Benchmark workbench for MAX (Mojo) LLM decode kernels vs llama.cpp / cuBLAS / FlashInfer and the memory roofline on consumer NVIDIA GPUs (sm_86/sm_89). A public record, not a competing kernel library.

NVFP4 GEMM, GEMV, linear layers and quantization for NVIDIA GPUs without native FP4 (sm_50–sm_90): every NVIDIA NVFP4 variant (ModelOpt, Transformer Engine 1x16 / 16x16, 4over6, swizzled scales) bit for bit, exact block sums, accumulation, C API and PyTorch bindings

A quantized GEMV kernel ladder measured against its own bandwidth roof. 12 CUDA kernels plus Triton for batch-1 LLM decode over ggml q8_0/q4_0, measured on 5 NVIDIA GPUs from sm_60 to sm_89.

OpenCL INT4 GEMV optimization for Qualcomm Adreno 540 / Snapdragon 835 mobile LLM inference

C++/CUDA inference engine for Qwen2.5-Coder-0.5B, written from scratch: INT4 GEMV/GEMM kernels, Flash-Decoding, a 24-layer decode engine and lossless speculative decoding. Verified against Hugging Face; benchmarked against bandwidth ceilings, PyTorch and llama.cpp.