arxivcs.SDcs.AIcs.NI2026-07-02
Metronome: Bound the Cache, Keep the Beat for Real-Time Interaction Model Serving
Real-time interaction models -- Moshi, MiniCPM-o, Qwen-Omni -- turn serving into a periodic real-time task: on every frame a session ingests streaming audio and must respond by a recurring wall-clock deadline, while its KV cache grows monotonically and stays pinned for the whole…