FlexServe: A Fast and Secure LLM Serving System for Mobile Devices with Flexible Resource Isolation

arXiv·medium signal

Device-side LLMs have grown explosively for their privacy and availability advantages over cloud serving, and FlexServe targets fast, secure on-device inference with flexible resource isolation. For builders shipping local-first or edge AI, it addresses the dual problem of latency and isolation that on-device deployment introduces.