METR Uplift Study Follow-Up: Original 19% AI Slowdown Finding — Now Questioned by Selection Effects

METR·high signal

METR's landmark 2025 study found AI tools caused a 19-20% slowdown for experienced open-source developers completing tasks. The 2026 follow-up finds 30-50% of participants are now self-selecting which tasks to submit, choosing not to submit tasks they 'did not want to do without AI' — invalidating clean random assignment. METR is changing their experiment design because current data 'gives an unreliable signal of the productivity effect.' The self-selection bias likely masks significant AI speedups on tasks developers actually choose to use it for.