Ternary (BitNet b1.58-style) weight GEMM on SM120 (Blackwell) tensor cores.
Status: COMPLETE — bounded spike, all experiments run on the me4me RTX 5060 Ti (GPU 0, SM120, cc 12.0), CUDA 13.3, driver 610.43.03. Every headline path is a NATIVE SM120 tensor-core route (no CPU/emulated fallback). All exactness claims verified entry-by-entry against an independent pure-Python integer oracle.
Ternary {-1,0,+1} weights can use SM120 tensor cores losslessly, and the fastest exact route is NVFP4 (FP4 E2M1), not INT8:
- Exactness: INT8 IMMA, NVFP4 E2M1, FP8 E4M3, and a custom SIMT 2-bit-packed kernel all reproduce the integer oracle exactly on every output entry — 8,912,896 exact comparisons across 2 shapes × 2 seeds × 4 paths, 0 mismatches.
- Throughput (4096³, resident, median of 60 launches): NVFP4 163.7 TMAC/s = 2.32× INT8 IMMA (70.5), 1.81× FP8 (90.2), 6.79× FP16 (24.1), 242× the SIMT 2-bit reference (0.68).
- Accuracy trade-off (activation quantization, weights stay exact): relative L2 error ≈ 1.1% (int8), 2.6% (FP8 E4M3), 15.3% (FP4 E2M1). FP4 buys throughput at a real activation-precision cost.
- Negative result: MXFP4 (
VEC32_UE8M0) is unsupported on SM120 (cuBLASLt heuristicINVALID_VALUE); the native FP4 route is NVFP4 (VEC16_UE4M3). Ternary weights give no INT8 tensor-core speedup over dense INT8 (1.008×) — the ternary win is storage/bandwidth, not INT8 compute.
| Experiment | Where | Count | Outcome |
|---|---|---|---|
| cuBLASLt narrow-precision support probes | server, GPU 0 | 5 probe programs | NVFP4/FP8/INT8 supported & exact; MXFP4 unsupported |
| Correctness vs integer oracle | server, GPU 0 | 2 shapes (256³,1024³) × 2 seeds × 4 paths | EXACT, 0/8,912,896 mismatches |
| Negative controls | server, GPU 0 | wrong FP4 scale + overflow-K guard | wrong scale diverges (63070/65536); overflow rejected pre-launch |
| Activation-quantization error | server, GPU 0 | 2 shapes × 2 seeds × 3 quant paths | int8 1.1% / FP8 2.6% / FP4 15.3% rel L2 |
| Resident timing | server, GPU 0 | 1024³,2048³,4096³ × 6 routes × 60 launches | FP4 fastest; tables in RESULTS.md |
| Transfer-inclusive offload | server, GPU 0 | 1024³ × 60 launches | 1.077 ms (57× slower than resident) |
| CPU oracle self-test + unit tests | local & server, stdlib | 9 tests | PASS |
- Bounded spike, not a full BitNet inference study: GEMM only, no end-to-end model, no training, no non-linear layers.
- Timing is resident repeated launches (operands reused on-device) — no streaming claim. A separate transfer-inclusive offload boundary is reported for one shape.
- Exactness is for ternary weights; activation quantization (int8/FP8/FP4) is approximate and its error is measured, not hidden.
- FP4 exactness bound requires
6·K ≤ 2^23(FP32 accumulate); INT8/SIMT requireK·127 < 2^31. Both checked before launch; study shapes are far inside. - Single GPU (RTX 5060 Ti 16 GB), single CUDA/driver version. No multi-GPU, no other architectures.
- NVFP4 scale-tensor internal swizzle is characterized only for uniform scales (sufficient for ternary weights at scale 1.0); non-uniform per-block scales were not validated.
requant-network/gpu-fp4-research— FP4/FP8 GEMM accuracy.requant-network/gpu-exact-tensorcore— exact modular FP4/INT8 tensor-core arithmetic (balanced base-13 encoding). This study is different: ternary weights, not modular fields.
cuda/ternary_bench.cu native SM120 driver (INT8 IMMA, NVFP4, FP8, SIMT 2-bit, FP16/INT8 baselines)
oracle.py independent stdlib integer oracle + data generator + verifier + quantization
run.py server orchestrator (gen + GPU runs + verify + finalize)
tests/test_oracle.py CPU-only stdlib test suite (9 tests)
experiments/ cuBLASLt support probes + PROBE_FINDINGS.md
results/*.json measured results (UTC timestamp, GPU/CUDA/driver, source SHA-256)
METHODS.md mathematical contract + timing boundaries
RESULTS.md tables of measured numbers
FINAL_REPORT.md conclusions, negative results, next decisions
# CPU oracle self-test (stdlib python3, no GPU):
python3 oracle.py selftest
python3 tests/test_oracle.py
# On the server (build, then full pipeline):
/usr/local/cuda/bin/nvcc -std=c++17 -O3 -arch=sm_120 cuda/ternary_bench.cu -o ternary_bench -lcublasLt -lcublas
python3 run.py <driver_version> # e.g. 610.43.03Every GPU run is pinned to CUDA_VISIBLE_DEVICES=0 and serialized under
flock /tmp/requant-new-studies-gpu.lock. See METHODS.md for the exactness contract.