low-bit

An innovative library for efficient LLM inference via low-bit quantization

Phonon: open speech recognition models (Phonon-2, Phonon-1) — CLI, CPU and CUDA images

Get more intelligence from every bit. Better quantization formats and smarter calibration let larger, stronger models run smoothly on the hardware you already own.

Compute-for-memory research lab for running larger LLMs on memory-constrained consumer GPUs. Official website exvram.pro

QuantFace: Towards Lightweight Face Recognition by Synthetic Data Low-bit Quantization

admm for cnn layerwise weight low bit quantization

Memory-efficient PyTorch optimizers with Adaptive Log-Space quantization, fused CUDA kernels, and Adafactor, CAME, APOLLO support.

Efficient low-bit KV-cache compression research with honest metadata accounting.

2.898-BPW Qwen3-8B with direct-packed CPU/CUDA inference

Pure-Julia CPU inference engine for BitNet b1.58 ternary LLMs

A local testbed for 1-bit and ternary neural networks, BitNet research, and low-bit LLM inference.

Quantization toolkit for large language models: run and store huge models on small VRAM. Inspired by Unsloth's dynamic quantization.

Some investigations I made with AMD cloud credits: training language models from scratch, exploring low-bit and byte-level architectures, and running ablation studies.

qpeft: quantization-aware training (QAT) and PEFT for LLMs in PyTorch. EfficientQAT, QA-LoRA and PEQA for Hugging Face models at 2/3/4 bits: train scales, zero-points or LoRA adapters, then merge into a low-bit integer model (GPTQ layout) that is tested to equal the trained one. Pure-torch and torchao backends.

Microscaling (MX) low-bit quantization formats, implemented and benchmarked on VLMs.

Compute-for-memory research lab for running larger LLMs on memory-constrained consumer GPUs.

저비트(3·4비트) 가중치 양자화 오차의 저랭크 복원(W ≈ Q + LR)을 NumPy로 단순화해 구현 — RTN·QER·보존-복원 랭크 분할·대리지표. 논문 재현 아님. 합성 행렬 결과, 실제 LLM 가중치·perplexity는 진행 중.