Tiberiu Musat

Invariant Learning Dynamics of Transformers in Inductive Reasoning Tasks

Tiberiu Musat, Tiago Pimentel, Nicolas Zucchet, Thomas Hofmann

We present a theoretical framework to explain the emergence of inductive reasoning abilities in Transformer language models. While previous works on Transformer learning dynamics have so far been mostly tied to specific tasks, we study a generalized class of inductive tasks that…Read more