Grégoire Cattan

Riemannian Geometry for Pre-trained Language Model Embeddings

Szczepan Konior, Alexandre Quemy, Przemysław Klocek, Bartłomiej Sobieski, Grégoire Cattan

Understanding the geometric structure of pre-trained language model embeddings matters for interpretability and safety. We ask whether sentence-level classification signal lives in the Riemannian geometry of contextual token embeddings, and probe it by extracting per-token pullba…Read more