Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads

Tianle Cai, Yuhong Li, Zhengyang Geng, Hongwu Peng, Jason D. Lee, Deming Chen, Tri Dao

Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads: 60 upvotes on Hugging Face Daily Papers, #2 of 8 papers on 2024-01-22. Day-by-day upvote history.

Paper page on Hugging Face · arXiv

Data: hysts-bot-data/daily-papers-stats and the Daily Papers API. Open data: tardellirs/paper-pulse-data. Sister project: Model Pulse, the download history of every model on the Hub.