a new KV-cache eviction mechanism using single-hop, drift-free rotation for up to 4.6x speedup than continuous re-rotation, no accuracy cost.
-
Updated
Sep 19, 2026 - Jupyter Notebook
#
kv-cache-optimization
Here are 6 public repositories matching this topic...
[NeurIPS 2025] NSNQuant: A Double Normalization Approach for Calibration-Free Low-Bit Vector Quantization of KV Cache
quantization kv-cache huggingface-transformers kv-cache-compression neurips-2025 llm-inference-optimization kv-cache-optimization
-
Updated
Jun 29, 2026 - Cuda
Intent-aware KV execution prototype for agentic long-context inference: semantic block selection, dynamic scoring, KV quantization modeling, speculative prefetch simulation, CPU references, and future Triton/CUDA kernels.
cost-model triton memory-bandwidth gpu-kernels mixed-precision prefetching inference-optimization kv-cache sparse-attention long-context paged-attention semantic-routing agentic-ai block-sparse-attention kernel-research block-attention semantic-attention kv-quantization paged-kv kv-cache-optimization
-
Updated
May 29, 2026 - Python
Transformer attention, worked through numerically; from self-attention and RoPE to KV cache, MQA, and GQA.
nlp key-value transformers pytorch attention attention-mechanism nlp-machine-learning attention-model attention-is-all-you-need multi-query attention-mechanisms pytorch-implementation multi-query-attention kv-cache transformers-models grouped-query-attention key-value-cache kv-cache-optimization kv-cache-management
-
Updated
Aug 18, 2026 - Python
MeritKV: Proactive KV cache admission for Large Language Models. Reduces prefill latency, energy consumption, and extends effective cache capacity by selectively caching tokens based on long-term utility.
caching pytorch reproducibility admission-controller huggingface kv-cache llm llm-serving vllm sglang llm-serving-systems kv-cache-optimization
-
Updated
Sep 23, 2026 - Python
0ns zero-copy, autograd-free hybrid guide layer (Pre-Transformer Packet Rectifier). Uses viscous Burgers' & Vorticity under FNG V3 to pre-rectify high-order skewness & stream clean tensor manifolds straight into LLM Attention.
zero-copy transformer high-performance-computing fluid-dynamics forward-only pybind11 plugin-architecture pinn burgers-equation jax vorticity cuda-kernel hardware-software-codesign llm backpropagation-free vram-optimization kv-cache-optimization xla-compiler autograd-free
-
Updated
Jul 25, 2026 - Python
Add this topic to your repo
To associate your repository with the kv-cache-optimization topic, visit your repo's landing page and select "manage topics."