arxivcs.LGcs.AI2026-07-20
An Analysis of Residual-Stream Geometry Across Transformer Depth
Sunit Bhattacharya, Ravi Shankar Kolli
We propose a transition-centred geometric analysis of transformer residual streams. Relative displacement measures how \emph{far} representations move between consecutive layers, and orthogonal Procrustes analysis separates each transition into a rigid rotation and a non-rigid re…