Lai Shun Chan

The Grokked Illusion: True Equilibrium Mitigates Catastrophic Forgetting

Xiaotian Zhang, Lai Shun Chan, Yue Shang, Entao Yang, Ge Zhang

While neural networks are typically evaluated by their training and test performance, these metrics do not reveal how robust a learned representation is. Recent studies have shown that solutions occupying larger volumes in parameter space, as quantified by Boltzmann entropy, ofte…Read more