arxivcs.DCcs.AI2026-07-09
SMetric: Rethink LLM Scheduling for Serving Agents with Balanced Session-centric Scheduling
Jiahao Wang, Kaizhan Lin, Kaixi Zhang, Jinbo Han, Xingda Wei, Sijie Shen, et al.
LLM scheduling is critical to serving, yet it remains unclear how well existing designs fit agentic serving--with LLM requests issued by agents instead of humans. This shifts the workload in two ways: (1) agents act only on complete responses, making the cluster's tokens per seco…