4B Vision Model Tops 235B Rival, No Teacher Needed

S²VOPD raised Qwen3.5-4B's fine-grained perception score from 70.7% to 77.4% by making the model its own teacher, a twist on visual on-policy distillation, per the paper.

On-policy distillation usually needs a stronger teacher model or privileged supervision to work at all. S²VOPD skips both. The same model reads a clean image as the teacher and a strongly augmented version of that image as the student. The training signal comes from the gap between the two, with no annotations, rewards, or second model involved.

The gain held up against competition. Qwen3.5-4B outscored every open-source model in the comparison, including the 235-billion-parameter Qwen3-VL, and GPT-5.4 too.

The paper's ablations matter more than the leaderboard number. All four augmentation families it tested helped. Symmetric self-distillation, where the model isn't degraded differently as student and teacher, hurt performance. And augmentations strong enough to erase the image evidence the question depended on produced large gradients that taught the model nothing useful.

The benchmark win is easy to repeat. The ablations are the harder part to copy. Augmentation strength has to sit in a narrow band. It must be strong enough to force learning, but not so strong it erases the answer. Teams that skip that tuning, and just point a model at itself symmetrically, will see this method underperform for them specifically.

The comparison surfaces in the August 17 research briefing, where cutting distillation costs is the throughline.