arxivcs.CLcs.IRcs.LG2026-07-11
Transforming LLMs into Efficient Cross-Encoders via Knowledge Distillation for RAG Reranking
Shreeya Dasa Lakshminath, Shubhan S
Cross-encoders achieve high reranking accuracy in Retrieval-Augmented Generation (RAG) pipelines but impose quadratic inference costs that limit real-time deployment. We address this by fine-tuning LLaMA 3 (8B) as a drop-in reranker using a two-stage pipeline: supervised fine-tun…