Laya got an MLX runtime the day after it shipped: 7.4 ms per decision on an M3 Max, 395 q/s multilingual

GitHub (mizorewww/laya-mlx)·medium signalXBlueskyLinkedInCopy link

laya-mlx (created Sept 19, 549 stars) is a native Apple Silicon inference runtime for Laya typed decision models that skips PyTorch and Transformers entirely. It reports 13.4 ms median end-to-end for short English questions and 7.4 ms on the multilingual checkpoint, with batched throughput of 146.8 q/s and 395.0 q/s respectively on an M3 Max with 40 GPU cores, peak memory of 687.6-943.6 MiB per question, and all three checkpoints (Laya 421M, Laya Multilingual 322M, Laya Typed-Decisions 421M) matching the upstream selected answer on 63/63 validation questions in both FP32 and FP16. A Core ML sibling from the same author claims ~5 ms. Same-day runtime ports are now the tell that a small-model release has actually landed.