vllm-project/semantic-router: System-Level Intelligent Router for Mixture-of-Models — 3.6K Stars

GitHub·medium signal

The vLLM project's semantic-router operates between the real world and AI models, capturing signals from requests and responses to intelligently route workloads across model instances for cloud, data center, and edge deployments. Built in Go (42.5%), Python (18.5%), TypeScript (15%), and Rust (11.9%), it includes real-time hallucination detection and PII leak prevention. The mixture-of-models (MoM) approach brings collective intelligence to LLM systems — a production-grade answer to the multi-model routing problem.