CORTEXA
← Browse
arxivcs.CLcs.IRcs.LG2026-07-11

Transforming LLMs into Efficient Cross-Encoders via Knowledge Distillation for RAG Reranking

Shreeya Dasa Lakshminath, Shubhan S

Cross-encoders achieve high reranking accuracy in Retrieval-Augmented Generation (RAG) pipelines but impose quadratic inference costs that limit real-time deployment. We address this by fine-tuning LLaMA 3 (8B) as a drop-in reranker using a two-stage pipeline: supervised fine-tuning on a custom query-document relevance dataset via the Unsloth framework with LoRA adapters, followed by 4-bit quantization for efficient inference. The resulting model replaces the cross-encoder in a dual-retriever RAG pipeline combining BM25 and dense vector search. Evaluated on a domain-specific question-answering benchmark using the RAGAS framework, our fine-tuned LLaMA 3 reranker achieves gains of 14% in answer relevancy, 16% in context precision, 19% in answer similarity, and 21% in answer correctness over the cross-encoder baseline, while reducing inference overhead through 4-bit quantization. These results demonstrate that instruction-tuned LLMs can be adapted into accurate, efficient rerankers without the quadratic complexity of traditional cross-encoders.

View free PDFSource page

Related papers

arxivcs.CLcs.AIcs.DLcs.IRcs.LG2026-07-10

Automatic Thematic Indexing of Large Literary Corpora: A Machine Learning Approach to Voltaire's Complete Works

Miguel Arana-Catania, Gillian Pink, Glenn Roe

Thematic indexing -- the practice of assigning structured conceptual labels to sections of text -- is essential to scholarly access in large-scale literary and historical editions, yet it remains a largely manual, labour-intensive process. This paper explores the application of m…

View free PDFSource page
arxivcs.IRcs.AIcs.CLcs.LG2026-06-29

SABER-Math: Automated Benchmark for Information Retrieval Evaluation in Mathematics

Nikolay Georgiev, Maria Drencheva, Kseniia Ibragimova, Ivo Petrov, Dimitar I. Dimitrov, Martin Vechev

As agentic AI systems tackle more complex mathematical tasks, they increasingly rely on information retrieval (IR) to search problem databases, theorem libraries, and educational resources. However, choosing the right retriever remains difficult, as it is infeasible to directly i…

View free PDFSource page
arxivcs.IRcs.AIcs.CLcs.LG2026-06-29

ARMOR: Adaptive Retriever Optimization for Low-Resource Telecom Question Answering

Heshan Fernando, Quan Xiao, Yan Xin, Tianyi Chen

Telecom question answering (QA) is a challenging setting for retrieval-augmented generation (RAG): evidence is fragmented across standards, papers, encyclopedic resources, and web documents, and answers often hinge on technical tables, equations, and specialized protocol language…

View free PDFSource page
arxivcs.AIcs.CLcs.HCcs.IRcs.LG2026-06-26

DysLexLens: A Low-Resource LLM Framework for Analysing Dyslexic Learners Insights from Online Forums

Dana Rezazadegan, Atie Kia, Phongpadid Nandavong, Dominique Carlon, Jeremy Nguyen, Abhik Banerjee, et al.

Dyslexic learners increasingly use artificial intelligence (AI) tools to support reading, writing, organisation, and study-related tasks. However, their lived experiences with these tools remain largely underexamined. This paper proposes DysLexLens, a low-resource LLM framework,…

View free PDFSource page
arxivcs.CLcs.AIcs.IRcs.LG2026-06-27

The strength of clinical evidence is recoverable from language model representations but not from their stated grades

Soroosh Tayebi Arasteh

Large language models (LLMs) increasingly summarize clinical evidence, where a claim's weight depends on how strongly it is supported. Yet these models convey confidence poorly, and properties they never state, such as truth, are often readable from their activations. Whether a c…

View free PDFSource page