RefineRL: Microsoft Shows 4B Models Can Match 235B Performance via Self-Refinement RL

arXiv / Microsoft Research·medium signal

Microsoft Research introduced RefineRL with a 'Skeptical-Agent' that maintains a skeptical attitude toward its own outputs during competitive programming. After RL training, Qwen3-4B models integrated with Skeptical-Agent outperform 32B models and approach the single-attempt performance of 235B models — a 50x+ model size compression through inference-time self-refinement. Practical evidence that inference-time compute investment can dramatically reduce model size requirements for specific tasks.