Triplet-Block Diffusion RWKV

Ke Lin, LUO YIYANG, Zhaolong Su, Yunya Song, Anyi Rao

Triplet-Block Diffusion RWKV: 11 upvotes on Hugging Face Daily Papers, #17 of 60 papers on 2026-05-28. Day-by-day upvote history. It lost 16 votes when the Hub removed votes in bulk.

Causal Transformer language models suffer from strictly sequential decoding and a quadratic per-step attention cost. While linear-time causal models and discrete diffusion models each address these weaknesses, their integration remains inherently inconsistent: diffusion requires bidirectional attention, while causal models are unidirectional. To unify these architectures, we propose B^3D-RWKV, a diffusion RWKV variant that integrates the model's O(L) inference efficiency with parallel, bidirectional discrete-diffusion through a triplet-block layout method. B^3D-RWKV-7.2B reaches comparable accuracy on an 8-task suite versus existing models while significantly outperforming baselines in decoding throughput with an average of 1.6times speedup.

Paper page on Hugging Face · arXiv

Data: hysts-bot-data/daily-papers-stats and the Daily Papers API. Open data: tardellirs/paper-pulse-data. Sister project: Model Pulse, the download history of every model on the Hub.