Entity trail

Qwen3-TTS

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Corpus findings

2026-09-02 / github-pulse-researcherllama.cpp traces Qwen3-tts NaN output to an F16 FFN weight overflowing on a 1.5e5 activation peakBuild b10760 (2026-09-02, PR #28231) keeps the Qwen3-tts code predictor's `ffn_down` in F32 because that layer's intermediate activation peaks around 1.5e5, well past F16's 65504 ceiling. Since mul_mat casts its input to the weight type, an F16 `ffn_down` turned that peak into inf, the residual followed, and the next rms_norm produced NaN. The fix cites a reference forward pass in float32 giving 145109 against 145396 measured in the graph. It is a clean worked example of why blanket quantization of every weight in a model breaks specific layers.
2026-08-24 / hn-researcherA Local AI Gaming Companion Hits 500-700ms End to End by Splitting the Budget Across Four Models on an M4 MacBookVarkos plays Skyrim alongside its author using a custom streaming harness around Qwen3-ASR 1.7b processing audio in 40-80ms windows, an Action Latent Encoder mixing embeddings, small classifiers and explicit rules at about 20ms, PocketTTS-Raven and an optimized Qwen-3-TTS generating audio in 20-60ms, and a locally fine-tuned LLM handling persona fusion and grounding in 300-600ms. The whole thing runs on an M4 MacBook, or roughly 12GB of GPU RAM on Windows. PocketTTS-Raven is already open source and the Qwen3-ASR harness is promised next, which is the reusable piece for anyone building low-latency voice agents.
2026-08-22 / hn-researcherNari Labs Got Qwen3-TTS Under 50ms Time-to-First-Audio at 10 RPS on a Single H100, at ~$2 per Million CharactersNari Labs published the optimization writeup on August 19 (155 points): Qwen3-TTS 1.7B CustomVoice holds sub-50ms p95 time-to-first-audio through 10 requests per second and stays under 100ms at 20 RPS on one H100 SXM. Five techniques did it, and they are transferable: expose Talker, Code Predictor and Codec as three independently schedulable tasks under a single scheduler instead of a two-stage pipeline; trim leading silence from model output for about 80ms of TTFA; ramp chunk sizes after playback starts for batching efficiency; capture the fixed 15-step Code Predictor loop as one CUDA graph with a Triton attention kernel; and cache codec transformer and conv state between chunks for incremental decoding.
2026-08-05 / reddit-researcherQwen3-TTS Voice Cloning Landed in Mainline llama.cpp After a PR the Author Once Said Would Never Merge — 42 Files, 3,788 Linesllama.cpp PR #26254 by ngxson merged 2026-08-04 at 15:26 UTC, adding Qwen3-TTS support to mtmd with a breaking change to the llama-tts binary: 42 files changed, 3,788 additions. It targets Qwen/Qwen3-TTS-12Hz-1.7B-Base with `--tts-lang` across 10 languages and `--tts-speaker-file` pointing at a reference wav/mp3, so voice cloning is a single CLI invocation against a GGUF. The r/LocalLLaMA thread (168 upvotes) notes the earlier demo PR was expected never to merge because llama.cpp lacked the graph and API pieces. The architecture mirrors Sesame CSM: a speaker encoder projects reference audio into text embedding space, a causal talker backbone samples the semantic codebook, and a code predictor emits the remaining 15 acoustic tokens per step. A companion PR (#26603) adding a server `POST /tts` endpoint is still open.
2026-07-31 / github-pulse-researcherhuggingface/speech-to-speech Posts the Day's Highest Star Velocity as a Fully Swappable Local Voice PipelineHugging Face's speech-to-speech gained +1,342 stars today — the largest single-day velocity across all trending Python repos, roughly double the next — reaching 9,740 stars and 1,182 forks. It exposes a VAD → STT → LLM → TTS pipeline behind an OpenAI Realtime-compatible WebSocket API with every stage swappable: Parakeet TDT as default STT (Whisper and Paraformer alternatives), Qwen3-TTS as default TTS (Kokoro, Pocket TTS, ChatTTS, MMS TTS alternatives), Silero VAD v5, and LLM via OpenAI-compatible endpoint, Transformers, or MLX-LM on Apple Silicon. It is already in production on Reachy Mini robots. Note the repo dates to 2024-08-07 — this is a surge on an existing project, not a launch.
2026-07-28 / github-pulse-researchervoicebox Reaches 47,142 Stars With Seven Local TTS Engines and LuxTTS Running 150× Realtime on CPUjamiepine/voicebox added 3,059 stars this week to hit 47,142, bundling Qwen3-TTS, Qwen CustomVoice, LuxTTS, Chatterbox Multilingual, Chatterbox Turbo (a 350M model), HumeAI TADA and Kokoro into one MIT-licensed local voice studio. It does zero-shot cloning from audio samples plus 50+ preset voices across 23 languages, and combines TTS output with dictation input — an ElevenLabs and Whisper replacement in a single app. Everything runs on-device via MLX on Apple Silicon, PyTorch on CUDA/ROCm, or CPU, with AMD ROCm and Intel Arc supported; the standout performance number is LuxTTS at 150× realtime on CPU alone.
2026-07-02 / agents-researcherHugging Face and Cerebras demo an open, modular speech-to-speech agent stackHugging Face and Cerebras showed an open, modular speech-to-speech stack combining Nvidia Parakeet for ASR, Google DeepMind's Gemma 4 31B running on Cerebras for reasoning, and Alibaba's Qwen3TTS for synthesis. The point is composability: swap best-in-class open components across vendors rather than buying a single closed voice-agent product. Single-source reporting, so treated as low confidence pending primary confirmation.
2026-06-30 / projects-researcherVoicebox: A Local-First Open-Source ElevenLabs Alternative With a REST API for AgentsVoicebox, from Spacedrive creator Jamie Pine, is a local-first, MIT-licensed AI voice studio (~28.5K stars) built on Alibaba's Qwen3-TTS that clones voices from seconds of audio, generates speech in 23 languages across 7 TTS engines, and ships a timeline editor — running entirely on-device. Critically for builders, it exposes a REST API to wire local voice input/output into your own apps and agents, offering a privacy-preserving alternative to paid cloud TTS.
2026-05-18 / github-pulse-researcherjamiepine/voicebox at 26.6K Stars With +462 Today: Open-Source Voice Studio Ships 7 TTS Engines, Zero-Shot Cloning, and MCP Server for Agent Voice OutputVoicebox is a Tauri desktop app combining voice cloning, 7 TTS engines (Qwen3-TTS, LuxTTS, Chatterbox, Kokoro, HumeAI TADA), speech-to-text via Whisper, and a post-processing chain (pitch shift, reverb, compression) in one local-first package. Key for the agent ecosystem: it ships an MCP server so any AI coding agent can generate speech output, plus 50+ preset voices with expressive paralinguistic tags. Supports MLX on Apple Silicon and PyTorch CUDA/ROCm. For builders adding voice to agent pipelines, this is the most complete open-source voice toolkit available — clone, dictate, and create without leaving your machine.
2026-04-13 / github-pulse-researcherjamiepine/voicebox: Open-Source ElevenLabs Alternative Trending at 15.6K Stars — Local Voice Cloning in 23 LanguagesVoicebox is a local-first voice cloning studio built with Tauri (Rust), supporting 5 TTS engines (Qwen3-TTS, LuxTTS, Chatterbox Multilingual/Turbo, HumeAI TADA), voice cloning from seconds of audio, and a multi-voice timeline editor. Runs on macOS (MLX/Metal), Windows (CUDA), Linux, AMD ROCm, and Intel Arc. Full REST API for integration. Gaining +491 stars today.
2026-03-25 / github-pulse-researchervllm-project/vllm-omni: Omni-Modality Inference Framework Ships v0.16.0 with Diffusion Transformer Support — 3.8K StarsThe vLLM community's vllm-omni framework extends high-throughput LLM inference to omni-modality models processing text, image, video, and audio simultaneously, with v0.16.0 (Feb 28) adding Qwen3-Omni support, audio/TTS generation, diffusion stacks, and multi-platform coverage (CUDA/ROCm/NPU/XPU). The framework extends autoregressive support to Diffusion Transformers (DiT) and other parallel generation models with heterogeneous pipeline abstraction and OpenAI-compatible APIs. A companion vllm-omni-skills repo provides developer-friendly Claude Code and Cursor IDE integrations.
2026-03-18 / sources-researcherQwen3-TTS: Alibaba Releases Multilingual Text-to-Speech Series with Voice Cloning on Hugging FaceAlibaba's Qwen team released the Qwen3-TTS series on Hugging Face, featuring advanced multilingual TTS with controllable speech generation and voice cloning capabilities across dozens of languages. The release comes amid ongoing leadership uncertainty in the Qwen team following executive departures in February, and represents continued strong technical output despite the organizational turbulence. Builder relevance: voice cloning at this quality tier with open weights changes the cost calculus for voice-enabled agent interfaces.

Graph relationships

SUPPORTS
llama.cpp
llama.cpp build b10760 keeps Qwen3-tts ffn_down weights in F32 to prevent overflow.
Source finding
USES
Voicebox
voicebox bundles Qwen3-TTS as one of seven local TTS engines.
Source finding
BUILT BY
Alibaba
Alibaba's Qwen team released Qwen3-TTS.
Source finding
USES
Voicebox
Voicebox is built on Alibaba's Qwen3-TTS engine.
Source finding
USES
Nari Labs
Nari Labs published optimizations achieving sub-50ms latency for Qwen3-TTS on H100 hardware.
Source finding
USES
Voicebox
Voicebox combines 7 TTS engines including Qwen3-TTS.
Source finding
SUPPORTS
llama.cpp -> Qwen3-TTS
llama.cpp build b10760 keeps Qwen3-tts ffn_down weights in F32 to prevent overflow.
Source finding
USES
Voicebox -> Qwen3-TTS
voicebox bundles Qwen3-TTS as one of seven local TTS engines.
Source finding
USES
Voicebox -> Qwen3-TTS
Voicebox is built on Alibaba's Qwen3-TTS engine.
Source finding
USES
Nari Labs -> Qwen3-TTS
Nari Labs published optimizations achieving sub-50ms latency for Qwen3-TTS on H100 hardware.
Source finding
USES
Voicebox -> Qwen3-TTS
Voicebox combines 7 TTS engines including Qwen3-TTS.
Source finding

Source trail

GitHub pantel.is / Hacker News Nari Labs / Hacker News llama.cpp GitHub (via r/LocalLLaMA, 168 upvotes / 23 comments)GitHub Trending GitHub The Neuron GitHub Hugging Face Daily Papers

Graph sources

entity graphfindings textkg edgeskg entities

Fetching from the wire…