LoopVQ: Sub 1-bit KV Cache Quantization for Looped Transformers

Minsoo Cheong, Sungjoo Yoo

LoopVQ: Sub 1-bit KV Cache Quantization for Looped Transformers: 9 upvotes on Hugging Face Daily Papers, #22 of 38 papers on 2026-11-05. Day-by-day upvote history.

Upvotes over time

It had 8 upvotes by its first day, 9 upvotes by the next day, 11 upvotes by day 3, 9 upvotes by day 7, and has 9 now. It ranked #117 of 207 papers in its week and #177 of 340 in its month. Its code repository has 3 GitHub stars.

Looped transformers are a parameter-efficient alternative to standard transformers for language modeling. By reapplying the same layers across multiple loop steps, recent looped language models match the performance of non-looped models several times their size. While this architecture effectively reduces parameter memory, each loop step still stores its own keys and values, so the key-value (KV) cache grows linearly with the number of loop steps and becomes the main memory bottleneck. We observe that the KV cache of looped models is highly redundant along the loop axis and that the loop steps differ substantially in their sensitivity to quantization. Based on these observations, we propose LoopVQ, a KV cache vector quantization method for looped transformers that codes channels of multiple loop steps jointly, and fine-tunes the codebooks to match the output of the full-precision model. At 0.5 bits per element, LoopVQ outperforms existing KV cache vector quantization methods for standard LLMs on general, long chain-of-thought reasoning, and long-context retrieval tasks with Ouro and Huginn models. Its design is orthogonal to these methods, and combined with TaSQ, it retains much of the BF16 accuracy even at 0.25 bits per element. We also implement LoopVQ in SGLang, where it nearly halves the decode latency at a 32K-token context and raises the peak serving throughput by up to 4.3times and 6.1times on the 1.4B and 2.6B Ouro models compared to the BF16 baseline.

Paper page on Hugging Face · arXiv

Data: hysts-bot-data/daily-papers-stats and the Daily Papers API. Open data: tardellirs/paper-pulse-data. Sister project: Model Pulse, the download history of every model on the Hub.