arxivcs.LGcs.CL2026-07-20
L1 Augmented Attention as an Improved Vector Similarity Metric
Scaled dot product attention conflates directional alignment and vector magnitude, limiting its effectiveness as a similarity metric in Transformer models. We introduce L1 augmented attention, a simple and computationally parallelizable modification that subtracts a learned, head…