A 2B-Parameter Model Learns When to Stop Answering

SVR trains a 2-billion-parameter model to stop answering once it's confident, skipping a fixed turn budget, per the paper.

Multi-turn RL systems typically pay for every extra turn, either running a fixed budget or adding an outside grader to catch answers early. Qwen3.5-2B trained with SVR hit 0.563 macro-average accuracy across seven math benchmarks in 2.99 turns instead of ten, beating GRPO and an oracle-guided baseline.

The self-grading is the design choice that makes this work. Each turn, the model outputs a solution plus its own verdict, correct or incorrect, and a confidence score. It only locks in an answer once the verdict says correct and confidence clears a set threshold.

Ground-truth correctness only shapes the reward signal during training, the paper says. The model never sees the right answer while deciding whether to stop.

SVR also beat a fixed-budget baseline that used oracle guidance on when to stop. That result suggests the model's learned confidence signal can outperform an external judge making the same call.

The paper's numbers all come from math benchmarks, where answers are easy to grade against ground truth. Whether the same self-grading trick holds up on tasks without a clean answer is untested.