Renaming identifiers and adding dead code costs code agents up to 6.7 points, and the simpler scaffold holds up better.

Semantics-preserving transformations of SWE-bench repos produced statistically significant degradation in 6 of 16 model-scaffold-dataset configurations. Rankings flipped by scaffold: Qwen looked robust under mini-SWE agent and brittle under OpenCode. arXiv The simpler scaffold was more robust overall, which argues against assuming a heavier harness buys reliability.