Maria Smirnova

Sign compression for Muon: SignMuon, MuonSign, and the Limits of Error Feedback

Maria Smirnova, Alexey Kravatskiy

SignMuon compresses the Muon update to one bit per parameter by taking its elementwise sign, providing the most direct way to run a matrix-aware optimizer under an extremely low communication budget. It outperforms SignSGD in practice, yet it can ascend even on a linear function.…Read more