Training a Weak Model on Expert Trajectories Under Its Own Evolved Harness Regresses It on All Seven Tasks
arXiv 2609.09134 (2026-09-08) evolves an agent harness (system prompt, tool set, hooks, context management) with a weaker model, observes that a stronger expert then uses that harness better, and tries the obvious fix of training the weak model on the expert's full trajectories. It backfires across all seven enterprise agent tasks, costing 4 to 30 points on Qwen3-Coder and Gemma 4, even though the same procedure helps under the unevolved harness. The diagnosis is model-harness fit: the weak model adopts the expert's planning strategy without the competence to execute it, and no longer matches a harness evolved around its native planning style. Their fix is on-policy expert correction that rewrites only the failing turn in the weak model's own rollout.
↳ Follow the thread