Microsoft's Agent Lightning learns to tune other coding agents

Microsoft released Agent Lightning v1.0.1 as an installable skill instead of a standalone tool. Point it at an editable agent and a benchmark, and it works through changes to prompts, tools, workflows, models, and reasoning settings. It scores each attempt against accuracy, cost, latency, and reliability. That turns a manual before-and-after comparison into something the agent runs on its own. It matters most for anyone maintaining more than one version of the same agent.

Install runs through gh skill install microsoft/agent-lightning agent-lightning --agent <agent>, targeting Claude Code, Codex, or Copilot.

v1.0.0 released a week earlier and did the plumbing that makes this practical. The codebase dropped to about 3,500 lines, and Kubernetes Job execution got added for running these sweeps at scale. A proxy now trains against a deployed harness without touching the agent's own code.

The release notes don't say how long a tuning pass takes or what a benchmark run costs against a live model API. Without that number, it's unclear whether this saves time or adds a slower loop.

What's still missing from the notes is guidance on picking a benchmark. Optimize against the wrong one and the agent scores well on paper and worse in the workflow you run day to day.