Efficient Monotonic Multihead Attention

Ma, Anna Sun, Siqi Ouyang, Hirofumi Inaguma, Paden Tomasello

Efficient Monotonic Multihead Attention: 7 upvotes on Hugging Face Daily Papers, #15 of 17 papers on 2023-12-07. Day-by-day upvote history.

We introduce the Efficient Monotonic Multihead Attention (EMMA), a state-of-the-art simultaneous translation model with numerically-stable and unbiased monotonic alignment estimation. In addition, we present improved training and inference strategies, including simultaneous fine-tuning from an offline translation model and reduction of monotonic alignment variance. The experimental results demonstrate that the proposed model attains state-of-the-art performance in simultaneous speech-to-text translation on the Spanish and English translation task.

Paper page on Hugging Face · arXiv

Data: hysts-bot-data/daily-papers-stats and the Daily Papers API. Open data: tardellirs/paper-pulse-data. Sister project: Model Pulse, the download history of every model on the Hub.