Ram Janarthan

When transformers learn "impossible" languages, what do they learn?

Ram Janarthan, Coleman Haley, Sharon Goldwater

Recent work suggests that transformer language models show a bias towards human languages over unnatural ("impossible") languages argued to be unacquirable by humans. However, this literature has largely based these claims on differences in sample efficiency and test-set perplexi…Read more