arxivcs.ARcs.AIcs.CLcs.DCcs.LGcs.PF2026-07-21
BaseRT: Advancing Best-in-Class LLM Inference with Apple M5 Neural Accelerators
Fabian Waschkowski, Prabod Rathnayaka, Lukas Wesemann
Apple's M5 generation introduces a redesigned GPU architecture in which every core carries a dedicated Neural Accelerator: on-die matrix units exposed through the Metal~4 tensor API. We show that BaseRT, our native Metal inference runtime for large language models on Apple Silico…