arxivcs.LGcs.AI2026-07-07
At-Grok Is Not Converged:A Measurement-Validity Audit for Grokking Representation Metrics
On modular arithmetic, a network's embedding keeps compressing for tens of thousands of steps after it has already generalized. Reading effective rank at the grokking transition overstates the converged value by 3-5x on an MLP, and by 1.3-1.5x on a transformer trained to converge…