Stephen Xia

Depth Exploration for LLM Decoding

Weisi Yang, Zipeng Sun, Stephen Xia

Autoregressive LLM decoding evaluates every generated token through the full layer stack, even though many tokens become predictable at intermediate depths. Existing lossless depth-adaptive methods exploit this redundancy by choosing a single non-final exit depth and verifying it…Read more