Siddharth Pal

2 papers indexed

arxivcs.LGcs.AIcs.CLcs.IR2026-07-11

Context by Distinct Information: An Auditable Dirichlet-Process Working Memory for Long, Redundant Context Streams

Context engineering decides what information a model carries forward, and current designs meter it in tokens: compressing the past into a bounded recurrent state, keeping a key-value entry for every token, or imposing a fixed budget through a window or eviction rule. All three ma…

View free PDFSource page

arxivcs.LGcs.AIcs.CLcs.NE2026-07-10

Remembering Distinct Items, Not Tokens: A Learnable Dirichlet-Process Cache Between State-Space Models and Attention

Siddharth Pal, Viktoria Rojkova

Fixed-state sequence models compress an unbounded past into a bounded state, which caps their associative recall at roughly the state dimension; attention escapes the cap by keeping a key-value entry for every token, at quadratic compute and a cache that grows with the sequence.…

View free PDFSource page