Jaeho Lee

PARTREP: Learning What to Repeat for Decoder-only LLMs

Andikawati P Widjaja, Yongjun Kim, Hyounghun Kim, Jaeho Lee

While decoder-only LLMs excel at a vast array of natural language tasks, it suffers from an asymmetric information flow induced by causal attention: later tokens are richer in contextual grounding than earlier ones. A simple and effective remedy is prompt repetition -- just appen…Read more