Randy Goebel

Does Role Specialization Matter for Explanation Faithfulness in Mixture-of-Experts?

Yeji Kim, Housam Babiker, Mi-Young Kim, Randy Goebel

Mixture-of-Experts (MoE) architectures have recently been extended with role-based mechanisms for interpretability. This is typically done by assigning semantic roles to individual expert components, for example roles like synergy, redundancy, and uniqueness in multimodal setting…Read more