openalexZenodo (CERN European Organization for Nuclear Research)2026-07-23
The correlation geometry of learned representations: a mean-field baseline with a weak anomalous excess, and its cross-domain universality
The hidden states of large language models carry a power-law correlation along the token axis, C(r) ∝ r^{−α}, whose exponent is stable across model scale and across architectures that share little else. We ask what this apparent “attraction” between nearby positions in representa…