Reconciling Impressive AI Benchmark Performance with Limited Developer Productivity Impacts

AI coding agents now complete multi-hour coding benchmarks with roughly 50% reliability, yet a randomized trial found experienced open-source developers took about 19% longer when allowed frontier AI tools than when tools were disallowed.

This talk presents the evidence on the productivity paradox in AI coding, shows the bottlenecks in deployment, and outlines the next steps for understanding AI’s productivity impacts

Speaker: Joel Becker, METR

Registerat weblink to attend in person or via Zoom