Test-Time Scaling for Diffusion Language Models with Frequency-Aware Remasking

Bowen Zuo, Yue Yu, Dongruo Zhou, Yinglun Zhu

Test-Time Scaling for Diffusion Language Models with Frequency-Aware Remasking: 8 upvotes on Hugging Face Daily Papers, #18 of 39 papers on 2026-11-11. Day-by-day upvote history.

Upvotes over time

It had 8 upvotes by its first day, 8 upvotes by the next day, 8 upvotes by day 3, 8 upvotes by day 7, and has 8 now. It ranked #63 of 133 papers in its week and #190 of 340 in its month.

Diffusion language models (DLMs) generate text through iterative denoising, where decisions about which tokens to commit or remask strongly affect output quality. Existing certainty-aware strategies rely on instantaneous confidence or entropy, but these local signals can be misleading on difficult reasoning problems. We formulate DLM test-time scaling as adaptive allocation of both inference compute and decoding strategies. Our two-stage approach uses a fast base sampler to solve easy questions, then applies frequency-aware remasking to the remaining hard questions. This strategy tracks how often each token's prediction changes across denoising steps and preferentially commits frequently changing tokens, encouraging samples to explore alternative reasoning paths rather than collapse onto the model's preferred trajectory. Experiments with LLaDA-8B-Instruct and Dream-7B-Instruct show consistent improvements over a strong adaptive allocation baseline, with absolute accuracy gains of up to 3.73 percentage points (LLaDA) and 7.33 percentage points (Dream) on MATH-500, and absolute coverage gains of up to 11.39 percentage points (LLaDA) and 10.98 percentage points (Dream) on HumanEval. On questions unsolved by the base certainty-aware strategy after 100 samples, frequency-aware remasking recovers solutions for 3.75% to 18.75% of examples.

Paper page on Hugging Face · arXiv

Data: hysts-bot-data/daily-papers-stats and the Daily Papers API. Open data: tardellirs/paper-pulse-data. Sister project: Model Pulse, the download history of every model on the Hub.