When Built-in Thinking Helps and Hurts: Constraint-Level Error Shifts in Instruction Following

arXiv·medium signal

Studies Qwen3 models (1.7B–32B) on IFEval and finds that reasoning ('thinking') mode improves math and coding but shifts the type of errors made in instruction following — helping some constraints while hurting others. Quantifies the trade-off rather than assuming reasoning is uniformly better. Actionable for builders deciding when to toggle thinking mode on tasks where strict instruction adherence matters more than raw reasoning.