Do AI Coding Assistants Really Make Developers Faster?

The rise of AI-powered coding assistants like Cursor and GitHub Copilot has promised a revolution in software development workflows. These tools, built on advanced language models from OpenAI, Google DeepMind, Anthropic, and xAI, are designed to write code, fix bugs, and automate testing. But does the reality live up to the hype for experienced developers?

New Research Challenges Assumptions

A recent study by the non-profit AI research group METR set out to answer this question. The researchers conducted a randomized controlled trial involving 16 seasoned open-source developers. Participants were asked to complete 246 real-world tasks on large code repositories—half the tasks allowed AI tools like Cursor Pro, while the other half did not.

Before starting, developers predicted that AI would help them finish 24% faster. However, the results were surprising: using AI actually increased completion time by 19%. In other words, developers were slower when using AI coding tools.

Why Did AI Slow Down Developers?

• Learning Curve: Only 56% of participants had experience with Cursor, the main AI tool used in the study, though most had used other AI coding assistants previously. All received training on Cursor beforehand.
• Prompting and Waiting: Developers spent significant time crafting prompts and waiting for AI responses, rather than coding directly.
• Complex Codebases: The tasks involved large, intricate repositories—areas where current AI tools often struggle.

Not the Final Word on AI Productivity

The researchers caution against drawing broad conclusions. They note that:

• Other large-scale studies have shown productivity gains of up to 26% using AI coding assistants.
• AI tools are improving rapidly, and results could be different just months from now.
• AI coding tools have recently gotten much better at handling complex, long-horizon tasks.

This study highlights that the impact of AI coding assistants can vary, especially for experienced developers working on challenging projects. The promise of universal productivity boosts may be overstated for now, and it’s important to consider the context and developer familiarity with specific tools.

Potential Risks: Mistakes and Security

Beyond productivity, other studies have found that AI-generated code can introduce bugs and even security vulnerabilities. As with any tool, careful evaluation and oversight are essential.

Conclusion

AI coding assistants are a powerful new resource, but they are not a one-size-fits-all solution. Teams should assess how these tools fit their specific workflows and remain realistic about the pace of productivity gains as the technology continues to evolve.

References

• METR: Early 2025 AI Experienced OS Devs Study (PDF)
• TechCrunch: AI coding tools may not speed up every developer, study shows
• IT Revolution: AI Coding Assistants Boost Developer Productivity by 26%
• The Register: AI developer Devin poor reviews
• TechRepublic: AI-generated code outages
• METR: Measuring AI Ability to Complete Long Tasks