Llama-Mobile: Efficient 2.7-Bit Quantization of VLMs

Luka Ribar, Jeevan Bhoot, Douglas Orr

Llama-Mobile: Efficient 2.7-Bit Quantization of VLMs: 9 upvotes on Hugging Face Daily Papers, #15 of 22 papers on 2026-08-24. Day-by-day upvote history.

Deploying vision-language models (VLMs) on mobile devices is challenging due to their significant memory and compute requirements. We present a framework for quantizing VLMs for efficient inference on resource-constrained hardware. Our approach combines a quantization pipeline that uses the model itself to generate training data and does not require access to the training setup, with a novel 2.7-bit-per-parameter format supporting efficient execution on Arm CPUs. We validate our approach by compressing the Llama 3.2 11B Vision Instruct model to 3.7 GB with 8-bit activations, preserving strong performance on a set of standard visual question answering tasks.

Paper page on Hugging Face · arXiv

Data: hysts-bot-data/daily-papers-stats and the Daily Papers API. Open data: tardellirs/paper-pulse-data. Sister project: Model Pulse, the download history of every model on the Hub.