Chenrui Liu

A Unified Framework for In-Context Learning with Causal and Masked Language Models

Chenrui Liu, Chuanlong Xie, Falong Tan, Yicheng Zeng, Lixing Zhu

In-context learning (ICL) has emerged as a central capability of pretrained language models, yet its theoretical analysis has focused primarily on causal language models trained by left-to-right autoregressive prediction, such as GPT-style models. Masked language models instead r…Read more