Microsoft Foundry-Local: One SDK to Replace Ollama, llama.cpp, and Whisper.cpp. No Cloud Required.

Microsoft open-sourced Foundry-Local, and I think it's the most underrated release of the week. One SDK for chat and audio with automatic hardware acceleration (NPU > GPU > CPU), self-contained with no external dependencies, and the API surface is identical to Azure AI Foundry. 2,079 GitHub stars and climbing.

Here's why this matters more than another "run models locally" announcement. Foundry-Local uses the same API as Azure. Zero application code changes to go from local to cloud. That's not something Ollama or llama.cpp can claim. You prototype on your laptop, deploy to Azure when you need scale, and the code is the same. For enterprise teams that need data governance guarantees, this is a big deal.

The companion tutorial is what sold me. Microsoft published a hands-on guide for building a fully offline AI support agent using Context-Augmented Generation (CAG). Unlike RAG, CAG pre-loads your entire knowledge base into the model context at startup. No vector database. No embeddings. No chunking pipeline. The total dependency footprint is two npm packages: express and foundry-local-sdk. That's it.

I've spent weeks debugging RAG pipelines, fighting with chunk sizes and embedding models and retrieval thresholds. For small-to-medium knowledge bases, CAG just skips all of that. Load your docs into context, ask questions. The simplicity is almost suspicious.

The hardware support is broad: CPUs, NVIDIA GPUs, Intel NPUs, Qualcomm NPUs on Copilot+ PCs. Foundry-Local also supports Microsoft's three proprietary MAI foundation models (MAI-Transcribe-1, MAI-Voice-1, MAI-Image-1), released April 2 and designed to integrate with the runtime.

What builders should do: if you're building anything that needs offline AI, start here instead of stitching together Ollama + Whisper + your own inference code. The two-dependency CAG approach is worth a weekend experiment for any knowledge-base app under ~50 pages.