Jamie Simon

Muon learns balanced solutions in matrix factorization without slow saddle-to-saddle dynamics

Mark Rhee, Jamie Simon, Dhruva Karkada

Matrix factorization (i.e., problems of the form $\min_{\mathbf{P},\mathbf{Q}} \|\mathbf{M}^\star - \mathbf{P}^\top\mathbf{Q}\|_\mathrm{F}^2$) is a minimal learning problem that exhibits both nonlinear parameter dynamics and representation learning. In this setting, we study how…Read more