CORTEXA
← Browse

Jerry Liu

2 papers indexed

arxivcs.LG2026-07-10

MLPs are Hebbians: Constructing Efficient Fact-Storing MLPs for Transformers

Roberto Garcia, Jerry Liu, Ronny Junkins, Sabri Eyuboglu, Atri Rudra, Christopher Ré

Large language models (LLMs) store factual knowledge in their parameters. While recent work has shown that this knowledge resides in MLP layers, existing constructive and mechanistic interpretability models of fact-storage in LLMs fail to explain the surprising empirical phenomen…

View free PDFSource page