AMD Releases PACE — CPU-Optimized LLM Inference Hitting 380 Tokens/Sec on Llama 3.1 8B

AMD Developer·medium signal

AMD released PACE (Platform Aware Compute Engine) on April 8, an optimization framework for LLM inference on 5th Gen EPYC CPUs. PACE adapts execution to specific NUMA topology and cache hierarchy, using a unified scheduler for prefill and decode phases. Combined with AMD's PARD (Parallel Draft Models), it achieves ~380 tokens/sec on Llama 3.1 8B at batch size 1. For organizations running inference on CPU-only infrastructure — common for privacy-sensitive workloads and edge deployments — this is practical for squeezing more tokens out of existing hardware.