From Large Language Models to Efficient System One Models via Adaptive Computation
Minh-Anh Nguyen
From Large Language Models to Efficient System One Models via Adaptive Computation: 5 upvotes on Hugging Face Daily Papers, #23 of 34 papers on 2026-11-13. Day-by-day upvote history.
Upvotes over time
It had 2 upvotes by its first day, 3 upvotes by the next day, 6 upvotes by day 3, 5 upvotes by day 7, and has 5 now. It ranked #102 of 167 papers in its week and #323 of 520 in its month. Its code repository has 1 GitHub stars.
Transferring LLMs into System One decision models typically replaces the language modeling head with a decision head while retaining the full backbone, making every decision pay for all layers. Yet it remains unclear how much depth such decisions require. We find that decision information is concentrated in intermediate layers, with early layers building essential features and the final six layers removable without loss, while independently trained smaller models largely mimic the full model rather than developing specialized decision capabilities. These findings suggest that efficiency requires not simply smaller backbones, but shared computation across different depths. Building on this, we propose , which jointly trains a full model with prefix sub-models that share its final layers and decision head through self-distillation. A lightweight gate reads the hidden state at each exit to determine whether additional computation is needed, allowing one model to operate at multiple layers. Across seven decision benchmarks, our approach matches or outperforms independently trained models at every depth, while achieving better accuracy-efficiency trade-offs through adaptive computation.
Paper page on Hugging Face · arXiv
Data: hysts-bot-data/daily-papers-stats and the Daily Papers API. Open data: tardellirs/paper-pulse-data. Sister project: Model Pulse, the download history of every model on the Hub.