Ayoub Ghriss

Kernelized Linear Attention: Breaking the Capacity Wall with Symmetric Cones

Ayoub Ghriss, Sourav Chakraborty

Linear attention promises constant-time recurrent inference but degrades sharply on associative recall. We formulate attention recall as a spherical-packing problem and introduce Kernelized Linear Attention Activations (KATA), a framework whose feature maps are derived from first…Read more