Entity trail

NCCL

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Corpus findings

2026-08-29 / reddit-researcher181 tok/s Aggregate on 2x DGX Spark, With a mmap Fix That Cut One Prefill's Disk Reads From 603 GB to 19 GBA builder posted a full recipe hitting 181 tok/s aggregate (peaking at 195) across ~9 concurrent agent sessions on two DGX Sparks running Qwen3.8-Flash-Next NVFP4 at 512K context, with single-stream decode at 30-50 tok/s. The nodes are joined by a direct ConnectX-7 cable over NCCL/RoCE at 200 Gb with TP=2, and the post warns the TCP fallback is silent and costs half the speed, so you must confirm 'Using network IB' in the NCCL log. Mapping the 47.7 GiB FP8 n-gram table off NVMe dropped per-node weights from 65 to 41 GiB, and two fixes made it fast: madvise(MADV_RANDOM) killed a 30x read amplification (one 405K prefill read 603 GB from disk before, 19 GB after) and 64 gather threads removed fault-latency serialization, freeing memory for a 2.89M-token KV pool.
2026-08-28 / github-pulse-researcherTRL v1.11.0 deletes its own vLLM server, cutting 1,218 lines to ~130 and moving weight sync to NCCLHugging Face TRL shipped v1.11.0 on 2026-08-26 replacing `trl vllm-serve`, a custom FastAPI wrapper with its own data-parallel fan-out and weight-sync worker extension, with a thin translation layer over `vllm serve`. The script drops from 1,218 to about 130 lines, fastapi/uvicorn/pydantic leave the vllm extra, and weight sync now rides vLLM's NCCL weight-transfer engine as a single packed broadcast instead of one HTTP request plus broadcast per tensor. Distillation teacher logprobs now come from `prompt_logprobs`.
2026-08-17 / github-pulse-researcheroMLX 0.6.0 Ships Distributed LLM Serving Across Multiple Macs — Qwen3.6-27B Goes 16.1 → 28.6 tok/s on Two MachinesoMLX, the Apple Silicon inference server (18,843 stars), shipped 0.6.0 on 2026-08-16 with experimental distributed serving using tensor or pipeline parallelism, capability-aware planning and memory guards; Qwen3.6-27B reached 28.6 tok/s across two Macs versus 16.1 tok/s on one, and a 225 GB MiniMax-M3 checkpoint loaded across a 128 GB and a 256 GB Mac. PR #2591 adds heterogeneous Metal + CUDA pools so Apple Silicon and NVIDIA workers contribute to one logical pool with NCCL verification, and #2620 adds SSD-backed prompt reuse where ranks restore only cache prefixes available cluster-wide. Decode throughput during concurrent prefill improved 1.6x to 43x. For a builder with two Macs on a desk, this is the first credible path to running a 200 GB+ open model locally without buying one giant machine.
2026-05-03 / arxiv-researcherZipCCL: Drop-in NCCL Replacement Reduces LLM Training Communication Time by 1.18x on 64 GPUsZipCCL exploits the Gaussian distribution of LLM tensor values to achieve lossless compression of collective communications during distributed training. Features exponent coding, GPU-optimized kernels with communication-aware data layout, and adaptive collective strategies. Validated on DeepSeek-V3, Qwen-MoE, and Llama3-8B across a 64-GPU cluster — a practical drop-in optimization for anyone running distributed training.

Graph relationships

USES
TRL
TRL now uses vLLM's NCCL weight-transfer engine for distributed weight sync
Source finding
USES
ZipCCL
ZipCCL is a drop-in NCCL replacement.
Source finding
USES
TRL -> NCCL
TRL now uses vLLM's NCCL weight-transfer engine for distributed weight sync
Source finding
USES
ZipCCL -> NCCL
ZipCCL is a drop-in NCCL replacement.
Source finding

Source trail

r/LocalLLaMA GitHub GitHub arXiv

Graph sources

entity graphfindings textkg edgeskg entitiesnewsletter issues

Fetching from the wire…