Ali Parviz

Reassessing Muon for Matrix Factorization

Ali Parviz, Gal Mishne, Alex Cloninger

Muon has recently emerged as a strong optimizer for large-scale deep learning, where it reshapes gradient updates through approximate orthogonalization and has been reported to outperform Adam and AdamW in large language model training. Its empirical success has motivated a growi…Read more