Feeling faster is not the same as being faster

The most useful result in AI-assisted development is still one of the most uncomfortable. In METR's controlled trial, experienced open-source developers working on their own repositories were measurably slower with AI assistance — around 19% — while reporting that they had been roughly 20% faster.

Not slower and knew it. Slower and certain of the opposite.

That is a nearly 40-point swing between measurement and perception, and it should make anyone cautious about how they are currently evaluating these tools. Because most teams are evaluating them by asking people how it is going.

Why the perception gap is structural

It is tempting to read this as developers fooling themselves. That is uncharitable and it misses the mechanism.

AI assistance genuinely removes the parts of the work that feel like effort. Recalling an API signature, writing the boilerplate, producing the first draft of a test file — these are effortful in the moment and highly salient in memory. What replaces them is reviewing, correcting, and re-steering, which is cognitively cheaper per second but takes longer in aggregate and leaves a much fainter trace.

You remember the twenty minutes you did not spend writing the scaffold. You do not remember the forty minutes spent reading generated code closely enough to trust it.

The bottleneck moved, it did not disappear

The follow-up finding matters more than the headline. Analyses through 2026 consistently show that AI tools reduce coding time and shift the bottleneck to review, which can increase overall pull-request cycle time.

This is the classic queueing result and it is worth stating plainly: speeding up one station in a pipeline does not speed up the pipeline. It moves the queue. If your team writes code twice as fast and reviews at the same rate, you have not doubled throughput. You have doubled the size of the review backlog and lengthened the average time a change spends alive.

That is why teams report feeling faster while organisational throughput does not move. Both observations are true at different levels of the system.

What the evidence says actually helps

Three findings are stable enough to act on.

There is a real learning curve. Reported estimates put it in the tens of hours of hands-on use before the effect turns clearly positive — weeks of regular work, not an afternoon. Any evaluation that runs for a sprint and concludes "it didn't help" measured the learning curve, not the tool.

Codebase quality is a multiplier, not a nice-to-have. Test coverage, type coverage, modularity and documentation quality all correlate with net productivity impact. This makes sense: the tool's output is only cheap if verifying it is cheap. In a codebase with no tests, every generated change requires a human to reason about correctness from scratch, and the generation was the easy part anyway.

There is an uncomfortable corollary. The codebases most likely to be handed an AI tool as a rescue — legacy, untested, undocumented — are the ones where it will help least.

Seniority changes what helps. Junior developers appear to benefit more from assistant-style completion; senior developers benefit relatively more from agentic workflows that require steering and system-level judgment. Rolling out one mode to everyone and measuring the average will produce a muddy number that describes nobody.

Measure the pipeline, not the feeling

If you want to know whether this is working for your team, the survey question is the one thing you can be confident will mislead you.

Track cycle time from first commit to merged, not time-to-first-draft. Track review latency and review depth separately — if reviews are getting faster at the same time as more code arrives, that is a warning, not a win. Track change failure rate and rework, because generated code that ships and then needs fixing has moved cost downstream rather than removing it.

None of this argues against the tools. The 2026 evidence suggests the effect is genuinely more positive than a year earlier, and there are careful studies showing real gains under the right conditions. It argues against a specific failure mode: adopting on the strength of how it feels, measuring nothing, and then being unable to explain a year later why delivery has not changed.

The honest position is that these tools are powerful, the effect size is highly context-dependent, and self-reports are the least reliable instrument available. Any of those three would be worth knowing. Together they mean that if you have not measured, you do not know — and the confidence you feel is, on the evidence, not informative.