arxivcs.PFcs.AIcs.LG2026-07-20
SALT: Salience-Aware Lexical Trie for Long-Context Compression
Oteo Mamo, Hyunjin Yi, Joydhriti Choudhury, Shangqian Gao, Weikuan Yu
As large language models (LLMs) process increasingly longer prompts, computation and KV-cache memory costs have emerged as major bottlenecks in inference systems. Existing input-level prompt compression methods address this, but rank each sentence by a scalar relevance score, tre…