arxivcs.CV2026-06-25
DnA: Denoising Attention for Visual Tasks
Ron Campos, Subhajit Maity, Xin Li, Srijan Das, Aritra Dutta
The softmax activation in multihead attention (MHA) is the de facto standard for attention-based models in visual perception tasks. However, standard softmax can produce noisy attention patterns that dilute relevant features and degrade its performance. In this paper, we propose…