Fetching from the wire…
01
02
03
04
05
06
07
08
Public story · 2026-03-18 · source-backed
Hardware-aware compression that preserves bit-exact model outputs while reducing memory footprint and increasing throughput. Unlike quantization, zero accuracy tradeoff — drop-in safe for production inference pipelines. Substantial memory reduction on standard GPU stacks with measurable throughput gains. arXiv 2603.17435
Each link below shares sources, entities, or timing with this story.
Mobile devices multitask, so the OS evicts weights and KV cache under pressure, forcing slow storage reads or full KV recomputation on the next request. mzCache partitions LLM memory into fine-grained shared buffers so eviction and restoration are partial and concurrent, then...
It compresses fine-grained local features bottom-up into a small set of proxy states per layer, runs global attention only in that compressed space, then decompresses and injects back into a local stream that persists across layers, so information one compression step misses s...
arXiv 2607.27090 targets the hidden tax in Mem0, MemGPT, and Zep: retrieved memory facts get re-prefilled on every single request. It precomputes each fact's KV representation, stores it on-GPU with a semantic embedding, and injects it into vLLM's paged cache via the KV-connec...
Tom's Hardware confirms the first sub-$1000 GPU with enough VRAM for serious local inference. Runs Qwen 3.5 27B at 4-bit at ~13 tok/s single-request. Intel's software stack still trails CUDA, but the hardware price point changes the local inference calculus.
SBW determines green-list membership through independent per-token Bernoulli trials rather than KGW's vocabulary permutation or SynthID's tournament, requiring one comparison per token against a counter-based RNG, which enables single-kernel execution with zero intermediate al...
Load Hijack modifies nothing but router weights in a checkpoint. When a private trigger appears, token-to-expert assignment concentrates on experts co-located on a single GPU, making it a straggler while peers idle (arXiv 2608.10614). Across three MoE families and four corpora...
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.