arxivcs.LGcond-mat.dis-nn2026-07-07
Fingerprint, Not Blueprint: How Positional Schemes Set the Default Spectral Algebra of Attention
The pre-softmax score of an attention head is a bilinear form $score(i,j) = x_i^T M x_j$ in a learned operator $M = W_q^T W_k$. Because M is generally non-symmetric, hence non-normal, it has a complex eigenspectrum and non-orthogonal eigenvectors, the regime where non-Hermitian a…