Programmatic tool calling beats JSON in 11 of 14 models

A model that writes code to call tools outperformed one filling out a JSON schema in 11 of 14 models tested on BFCL v4, per arXiv preprint 2608.06370.

That's a problem for teams that built their agent loop around JSON-schema tool calls as the default. The GPT-5.6 family gained 10.6% in accuracy under programmatic tool calling, and the gain held up under parallel execution in 13 of 14 models.

Under context degradation, the JSON baseline's accuracy fell 2.3% on average. Programmatic calling held steadier.

Yes, three of the 14 models still did better with JSON tool calling, so this isn't a clean sweep.

The advantage tracked model capability across release generations instead of fading. A one-off quirk in a single model family would shrink going backward through older, smaller models. This one didn't.

Framework maintainers who keep JSON as their only default are tuning for the weakest model in their fleet, not the one running in production. Worth watching whether the three holdout models share an architecture or training recipe. That distinction would show whether this is a permanent split or a one-time transition.