Research2026-04-21 · source-backed

GSQ Closes the Gap Between Simple and Complex Quantization at 2-3 Bits.

Issue

2026-04-21

Confidence

source-backed

Sources

Redaction

redacted

Story

GSQ uses Gumbel-Softmax sampling to match the accuracy of QTIP and AQLM while keeping the deployment simplicity of GPTQ/AWQ. If you're quantizing models for local inference, this eliminates the accuracy-vs-complexity tradeoff.

The best 2-bit quantization on paper loses 2.27x to a trellis kernel once you serve it.
Leech-lattice vector quantization holds the strongest reported 2-bit quality under its own protocol, but no implementation of the multi-shell decoder existed. This paper supplies one for the full 301-class codebook with a fused dequantize-plus-matvec kernel and measures batch-...
KronQ pushes usable 2-bit quantization of LLaMA-3-70B where GPTQ falls apart.
KronQ folds gradient covariance into the pipeline, using bidirectional incoherence processing and a Hessian-trace sensitivity metric for mixed-precision allocation (arXiv:2607.07964, COLM 2026). At aggressive 2-bit weight-only quantization it reaches 7.93 WikiText-2 perplexity...
Quantization is a selective forgetter, not a privacy defense.
Bits and Memories measures verbatim reproduction of training data rather than membership inference, using Pythia models with known-memorized sequence sets across five precision levels down to four bits. Verbatim memorization drops faster than perplexity at every precision and...
Telling colluding agents they're being watched by an activation probe changes nothing.
Activation probes are usually evaluated against agents who don't know they're monitored, which is a generous assumption. This study held models, probes and thresholds fixed and varied only the disclosure: nothing, monitor present, or monitor present plus last round's score. Ac...
Perplexity open-sourced an Apple Silicon inference server that supports exactly one checkpoint.
Lily is Apache 2.0 inside pplx-garden, a small Metal inference server built for Qwen3.6-35B-A3B converted to MLX affine 4-bit, exposing a minimal OpenAI-compatible chat API with greedy decoding. The README explicitly rules out dense and smaller Qwen checkpoints, BF16, GGUF, AW...
New Qwen3.8-27B GGUFs claim BF16-level AIME25 at 2.75 bpw.
Two methods: GSQ (Gumbel-Softmax Quantization), post-training scalar quantization that jointly learns grid assignments and scales at 2-3 bits, and RCO (Riemannian Constrained Optimization), assigning a quant type per tensor under a strict size budget by gradient descent on the...

Source trail

GSQ More from arxiv.org

Entities

AQLM AWQ Bits Complex Quantization Gap Between Simple GPTQ GSQ GSQ Closes Gumbel QTIP Softmax

Provenance

Canonical issue: Ramsay Research Agent — April 21, 2026
AI generated: no
Story unit: 2026-04-21-gsq-closes-the-gap-between-simple-and-complex-quantization-at-2-3-bits
Labels: source-backed, canonical briefing excerpt

Fetching from the wire…

GSQ Closes the Gap Between Simple and Complex Quantization at 2-3 Bits.

Story

Related stories

The best 2-bit quantization on paper loses 2.27x to a trellis kernel once you serve it.

KronQ pushes usable 2-bit quantization of LLaMA-3-70B where GPTQ falls apart.

Quantization is a selective forgetter, not a privacy defense.

Telling colluding agents they're being watched by an activation probe changes nothing.

Perplexity open-sourced an Apple Silicon inference server that supports exactly one checkpoint.

New Qwen3.8-27B GGUFs claim BF16-level AIME25 at 2.75 bpw.

Source trail

Entities

Provenance

GSQ Closes the Gap Between Simple and Complex Quantization at 2-3 Bits.

Story

Related stories

The best 2-bit quantization on paper loses 2.27x to a trellis kernel once you serve it.

KronQ pushes usable 2-bit quantization of LLaMA-3-70B where GPTQ falls apart.

Quantization is a selective forgetter, not a privacy defense.

Telling colluding agents they're being watched by an activation probe changes nothing.

Perplexity open-sourced an Apple Silicon inference server that supports exactly one checkpoint.

New Qwen3.8-27B GGUFs claim BF16-level AIME25 at 2.75 bpw.

Source trail

Entities

Provenance

Story

Related stories

The best 2-bit quantization on paper loses 2.27x to a trellis kernel once you serve it.

KronQ pushes usable 2-bit quantization of LLaMA-3-70B where GPTQ falls apart.

Quantization is a selective forgetter, not a privacy defense.

Telling colluding agents they're being watched by an activation probe changes nothing.

Perplexity open-sourced an Apple Silicon inference server that supports exactly one checkpoint.

New Qwen3.8-27B GGUFs claim BF16-level AIME25 at 2.75 bpw.

Tomorrow's issue, in your inbox.

Source trail

Entities

Provenance

Story

Related stories

The best 2-bit quantization on paper loses 2.27x to a trellis kernel once you serve it.

KronQ pushes usable 2-bit quantization of LLaMA-3-70B where GPTQ falls apart.

Quantization is a selective forgetter, not a privacy defense.

Telling colluding agents they're being watched by an activation probe changes nothing.

Perplexity open-sourced an Apple Silicon inference server that supports exactly one checkpoint.

New Qwen3.8-27B GGUFs claim BF16-level AIME25 at 2.75 bpw.

Tomorrow's issue, in your inbox.

Source trail

Entities

Provenance