One-shot Entropy Minimization

Zitian Gao, Lynx Chen, Joey Zhou, Bryan Dai

One-shot Entropy Minimization: 5 upvotes on Hugging Face Daily Papers, #37 of 66 papers on 2025-05-30. Day-by-day upvote history.

We trained 13,440 large language models and found that entropy minimization requires only a single unlabeled data and 10 steps optimization to achieve performance improvements comparable to or even greater than those obtained using thousands of data and carefully designed rewards in rule-based reinforcement learning. This striking result may prompt a rethinking of post-training paradigms for large language models. Our code is avaliable at https://github.com/zitian-gao/one-shot-em.

Paper page on Hugging Face · arXiv

Data: hysts-bot-data/daily-papers-stats and the Daily Papers API. Open data: tardellirs/paper-pulse-data. Sister project: Model Pulse, the download history of every model on the Hub.