Entity trail

llama.cpp

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Showing the first 40 findings. More graph evidence exists in the corpus.

Corpus findings

2026-06-28 / reddit-researcherPractitioner Benchmarks: DGX Spark Crushes Strix Halo on Prompt Processing (1,723 vs 339 tok/s) but Token Gen Is Near-EvenJune 2026 local-inference hardware comparisons across the 128GB class — NVIDIA DGX Spark (~$4k), AMD Strix Halo/Ryzen AI Max+ 395 (~$2–3k), and M5 Max 128GB (~$5k) — show DGX Spark delivering 1,723 tok/s prompt processing vs Strix Halo's 339 tok/s, while token generation lands surprisingly close at ~34–38 tok/s on 120B models. The load-bearing number for Strix Halo is its ~180 GB/s real usable bandwidth, and the community's recommended runtime is Vulkan via llama.cpp, not ROCm. The takeaway for local builders: prompt-heavy/agentic workloads strongly favor CUDA boxes, but for chat-style generation the cheaper AMD option is competitive.
2026-06-13 / sources-researcherllama.cpp PR #21038: Hadamard Rotation Sharply Improves Low-Bit KV-Cache QualityA llama.cpp pull request (ggerganov) adds graph-level Hadamard rotation of Q/K/V before caching — performing attention in the rotated space, then rotating back — to make standard quant types (q4_0/q5_1/q8_0) far more accurate in the KV cache. Reported benchmarks are dramatic: Qwen3 0.6B q5_1 KV perplexity drops from ~61.7 to ~14.1, and q4_1 from ~212 to ~22.3. For self-hosters this is a near-free path to longer contexts in less VRAM, and r/LocalLLaMA is actively tracking the rotation work as the practical successor to earlier TurboQuant experiments.
2026-06-07 / projects-researcherPewDiePie Drops 'Odysseus,' a Self-Hosted AI Workspace That Hit ~54K Stars in Its First WeekOdysseus (github.com/pewdiepie-archdaemon/odysseus) is an MIT-licensed, local-first AI workspace published around May 31, 2026 that reached roughly 54,000 stars and 6,300 forks by June 5. It bundles chat over local/remote models (vLLM, llama.cpp, Ollama, OpenRouter, OpenAI), autonomous agents with bash/files/web/memory tools plus MCP servers, a document editor, image generation/inpainting, IMAP/SMTP email triage, and multi-step research runs — all with no telemetry. The viral launch, driven in part by the high-profile creator, shows how fast a polished self-hosted alternative to commercial AI suites can capture mindshare.
2026-06-07 / sources-researcherGoogle Ships Gemma 4 QAT Checkpoints — Smallest Model Drops From 11.4GB to ~1.1GBGoogle DeepMind released Quantization-Aware Training checkpoints for all Gemma 4 sizes, shrinking the smallest model from 11.4GB to 1.1GB (0.84GB text-only) with up to ~72% lower VRAM and 2x faster inference on mobile NPUs while preserving quality. The release includes Q4_0 GGUF for llama.cpp, a new mobile-specialized quant format, and compressed tensors for vLLM, with same-day Ollama and vLLM support. Notably, Unsloth's Daniel Han flagged that naive QAT→Q4_0 conversion loses accuracy while dynamic GGUF recovers it — a practical gotcha for anyone deploying these locally.
2026-06-04 / reddit-researcherGoogle DeepMind Ships Gemma 4 12B — Encoder-Free Multimodal Model With Native Audio That Runs on a 16GB LaptopReleased June 3, the 11.95B-param open-weights model reads text, image, audio, and video with no separate vision or audio encoder, fitting in ~16GB (Q4 ~6.7GB) under Apache 2.0 with a 256K context window. It nears the larger 26B MoE on standard benchmarks at under half the memory footprint, with day-one support across Transformers, llama.cpp, MLX, vLLM, SGLang, and Unsloth. Encoder-free native audio at this size is a genuine local-inference unlock for builders.
2026-05-28 / github-pulse-researcherMOSS-TTS Family: Open-Source Speech Generation Suite Ships 7 Models Including 8B Flagship, Real-Time Agent Voice, and CPU-First Nano VariantOpenMOSS/MOSS-TTS provides a complete open-source TTS pipeline: zero-shot voice cloning (8B), multi-speaker dialogue generation (8B), voice design from text prompts (1.7B), low-latency agent synthesis (1.7B), 48kHz sound effects (1.3B DiT), and a 0.1B CPU-first nano model. Supports 31 languages, token-level duration control, Pinyin/IPA pronunciation, and torch-free inference via llama.cpp + ONNX Runtime. The audio tokenizer achieves 0.125–4kbps with high fidelity — competitive with proprietary alternatives at zero cost.
2026-05-26 / vibe-coding-researcherTip: llama.cpp CUDA Fast Walsh-Hadamard Transform Adds 7-9% Text Generation Speedup With Quantized KV CacheA merged PR (#23615) to llama.cpp adds a CUDA-optimized Fast Walsh-Hadamard Transform (FWHT) kernel that delivers 1-2% prompt processing and 7-9% text generation speedup when using quantized KV cache (-ctk q8_0 -ctv q8_0). Benchmarked on an RTX 5090, the gains are free performance for anyone already running quantized KV cache on NVIDIA hardware. For local model users running coding agents, this is a meaningful throughput improvement with zero configuration changes.
2026-05-24 / reddit-researcherllama.cpp Server Gets Built-In Native Tools: exec_shell, edit_file, grep_searchllama.cpp server now ships with built-in agentic tools — exec_shell_command, read_file, write_file, edit_file, apply_diff, grep_search, file_glob_search, and get_datetime — that run directly in the server process (93 upvotes, 29 comments on r/LocalLLaMA). Enabled via the --tools flag, these tools turn any local model into an agentic coding assistant without external tooling. The GitHub discussion includes security warnings: these tools execute with the server process's permissions and should never be exposed to a network or run as root.
2026-05-22 / vibe-coding-researcherPattern: Local Model Stacks Reaching 'Good Enough' Threshold for Agentic Coding WorkflowsThree signals converging this week: Qwen 3.6-35B-A3B running at 44 tok/s on 16GB GPUs is replacing cloud agents in daily workflows (r/LocalLLaMA, 293 upvotes); llama.cpp b9274 fixes the MTP VRAM leak that was the biggest blocker for 24/7 local inference; and a 106-upvote thread maps the $20K hardware path to full local independence. The emerging consensus: local models can code but can't self-verify — reliability comes from wrapping them in structured loops (git diff, test suites, file-allow gates) rather than trusting model self-assessment. The agent loop is the product, not the model.
2026-05-22 / vibe-coding-researcherTip: Build a Fully Local Coding Agent Stack for ~$20K — Community Recommends RTX 6000 + Qwen 3.6A detailed r/LocalLLaMA thread (106 upvotes, 119 comments) maps out the path to a self-hosted coding agent with no cloud dependency. Community consensus: an RTX 6000 or dual RTX 5090 setup running Qwen 3.6-35B-A3B through llama.cpp gets you 80% of cloud agent quality. The critical gap isn't raw coding ability — it's tool use reliability and long-context planning. Hardware alone doesn't solve the agentic loop; you need structured scaffolding (OpenCode, tests, git diff verification) around the model.
2026-05-22 / vibe-coding-researcherllama.cpp b9274 Fixes MTP VRAM Leak That Crashed Long-Running Inference ServersRelease b9274 (May 21) fixes a critical VRAM leak in llama.cpp's Multi-Token Prediction (MTP) stack where speculative decoding resources were not freed during sleep/idle cycles, causing servers to crash after repeated cycles. This is especially important for local agent workflows running 24/7 on consumer GPUs — the leak would silently accumulate until OOM. Users running Qwen3.6 MTP models were most affected.
2026-05-20 / github-pulse-researcherAlishahryar1/free-claude-code at 26.7K Stars With +783 Today: Proxy Routes Claude Code Through 10 Alternative ProvidersFree-claude-code routes Claude Code API calls through an Anthropic-compatible proxy supporting 10 providers including NVIDIA NIM, OpenRouter, DeepSeek, LM Studio, llama.cpp, and Ollama. Ships per-model routing to direct different Claude tiers to different backends, streaming and tool-use handling, local Admin UI, and VSCode/JetBrains extensions. At 26.7K stars, it's the most-adopted cost-reduction layer for Claude Code — enables local model fallback for simple tasks while preserving Anthropic for complex ones.

Graph relationships

BUILT BY
Georgi Gerganov
Georgi Gerganov created llama.cpp
Source finding
BUILT BY
Georgi Gerganov
Georgi Gerganov is the creator of llama.cpp
Source finding
SUPPORTS
Apple Neural Engine
new PR adds Apple Neural Engine backend to llama.cpp
Source finding
USES
Koharu
Koharu uses llama.cpp for local LLM translation
Source finding
SUPPORTS
Gemma 4
llama.cpp merged fix for Gemma 4 tokenizer newline splitting bug
Source finding

Source trail

Hardware Corner GitHub (ggml-org/llama.cpp)GitHub Google / MarkTechPost Google The Keyword GitHub GitHub

Graph sources

entity graphfindings textkg edgeskg entities