I’ve been building greenfield products with AI agents for a while now. On a good day, it’s intoxicating. You write a spec, the agent spits out a working feature, and you feel like you’ve unlocked a cheat code. On a bad day — and there are plenty — you’re staring at something the agent produced with absolute confidence that happens to be broken, over engineered or misunderstood.

Here is an example: I was building toward a milestone — a defined functional demo where the agent would walk me through the product’s capabilities live. The agent had a detailed spec. It wrote the features, implemented the logic, built out a full test suite. Everything passed. Green across the board. Ship it, right?

When the time came for a functional demo, I was going to start the application for the first time. Turns out there was no runtime. No entry point. No way to actually run the thing. The agent had built a fully tested product that couldn’t be started. It did everything the spec said. It just never occurred to it that software needs to run.

A human would never miss that. It’s so obvious it wouldn’t even make the spec. But agents don’t have obvious. They have instructions — and what isn’t in the instructions doesn’t exist.

That contradiction — fast but wrong, productive but stupid — started eating at me. So I did what any frustrated engineer would do. I went looking for data.

What I found changed how I think about all of this.

The number that broke my brain

METR — a research group that evaluates AI capabilities — ran a proper randomized controlled trial. 16 experienced open-source developers, 246 real tasks from their own repositories, measured with and without AI tools. Not juniors. Seasoned contributors working on codebases they knew inside out.

They predicted a 24% speedup.

The actual result: they were 19% slower with AI.

But here’s what really got me. Even after completing the tasks — after living through the slowdown — the developers still believed they had been 20% faster. The gap between what they perceived and what actually happened was 39 percentage points. That’s not a rounding error. That’s living in a different reality.

We’re all in this

If this were one study, we could wave it away. It’s not.

LinearB analyzed PR data across thousands of repositories: 67.3% of AI-generated PRs get rejected. Compare that to 15.6% for human-written code. Thats two out of three.

Google’s DORA 2025 report — probably the most respected benchmark we have for software delivery — found 95% AI adoption alongside a 9% increase in bug rates and a 91% increase in code review time. Let that land for a second. Ninety-one percent more time reviewing code, and we’re still shipping more bugs.

Stack Overflow’s 2025 survey? 66% of us cite “AI solutions that are almost right, but not quite” as our top frustration. Positive sentiment dropped to 60%. Nearly half of developers actively distrust AI-generated code.

We’re using AI more while trusting it less.

The autopilot problem

In flight simulator studies, experienced pilots followed incorrect automated recommendations 55-70% of the time — even when their own instruments were giving opposite instructions. The same thing was observed for doctors with decision-support systems.

Developers using AI assistants produced code with more security vulnerabilities while believing their code was more secure (Perry et al., 2023). And here’s the part that really stings: a 2012 meta-review found that automation bias persists even when you warn people, train them to watch for it, and let them experience failures firsthand. “Just be more careful” doesn’t work. The bias is structural, not attitudinal.

Think about what this means for us. We’re all sitting in the cockpit, instruments telling us one thing, the autopilot doing another — and we’re trusting the autopilot. Not because we’re lazy or careless. Because the output looks polished. It compiles. The tests pass. Everything on the surface says “this is fine.”

That’s the trap. The better AI gets at producing plausible-looking output, the harder it becomes to catch the mistakes that matter.

Death by a thousand almost-right answers

If your AI tool is 85% accurate on each step — and that’s generous — a 10-step workflow has about a 20% chance of being entirely correct. The errors aren’t obvious. Wrong field name. Off-by-one boundary. An assumption about nullability that happens to be wrong. The code compiles. The tests pass.

It looks right. It ships. It’s wrong.

This is actually worse than code that clearly breaks, because broken code gets caught. “Almost right” code is a silent killer. It passes every gate and surfaces as a bug report three weeks later, when nobody remembers which AI session generated it.

This is exactly the frustration that sent me down the research rabbit hole in the first place. On my own greenfield projects, I could feel the agents being simultaneously amazing and unreliable — but I couldn’t articulate why, and I couldn’t figure out whether the net effect was positive or negative. The METR study gave me the framework: each interaction feels productive, but the cumulative cost of subtle errors is where the slowdown hides. You don’t feel it in the moment. You feel it in the retro.

What the evidence says actually works

I want to be clear: this is not an anti-AI piece. I build with AI agents every day and I’m not going back. The raw ‘turn it on and go faster’ narrative doesn’t match the data — but the teams that get the governance right are seeing genuinely transformative results.

I see this firsthand in my consulting work. Enterprise teams trying to adopt agent-driven development, struggling with the same patterns — inconsistent output quality, review bottlenecks, subtle bugs that slip through. The teams that crack it share a pattern. They stop treating AI as a faster typist and start treating it as an executor that needs precise instructions and structural guardrails.

What the research points to:

Specs before code. Explicit acceptance criteria. Given/When/Then format. Edge cases covered. Every ambiguity resolved before the agent generates a single line. Because when the spec is wrong, the output is wrong — no matter how capable the model. An analysis of 2,500+ agent configuration files found spec quality to be the single highest-impact factor. Not the model. Not the prompt. The spec.

Small, reviewable changes. Google’s code review research shows that reviewer attention per line drops off a cliff after 200 lines. AI routinely generates 500-line changes. Constrain the output size and review quality goes back up. The 91% review time increase is what happens when you don’t.

Automated verification as a hard gate. Pre-commit hooks, type checking, linting, test suites that actually fail the build. Not nice-to-haves. Gates. When 45% of AI-generated code has security vulnerabilities, your verification pipeline is the last line of defense.

The boring governance work. Context files that capture project conventions. Design systems that take subjective decisions off the agent’s plate. Review gates between planning and implementation. None of this is exciting. All of it is what separates the teams that get real value from the ones producing plausible-looking garbage at unprecedented speed.

The speed isn’t in skipping the planning. It’s in compressing it. Full spec, architecture review, and test strategy in 15 minutes instead of two weeks — then ship it, get it in front of users, and iterate. That’s the real advantage. Not more code. Better decisions, faster.

Two realities, diverging fast

The perception gap means that teams without honest measurement cannot tell whether AI is helping or hurting them. They feel faster. The data says otherwise. And because they feel faster, they never invest in the governance that would make AI actually productive. It’s a trap that reinforces itself.

Meanwhile, the teams that figured this out are compressing build-measure-learn cycles from weeks to days. Not producing more code. Producing more _correct_ code, faster. DORA 2025’s core finding puts it plainly: AI amplifies existing patterns. Strong teams get stronger. Weak teams get weaker.

These two groups are diverging. Fast. And most teams are on the wrong side of it, because the perception gap is invisible from the inside.

I see both realities in my work — the greenfield projects where governance makes agents genuinely powerful, and the enterprise teams drowning in confident, wrong output because nobody set up the guardrails. Same technology. Completely different outcomes.

Which side is your team on? And more importantly — how would you know?

A note on the METR study

A fair question that came up after publishing: the METR trial used early 2025 models (primarily Cursor Pro with Claude 3.5/3.7 Sonnet). Models have improved significantly since. Would the results be different today?

METR themselves address this and believe developers are “likely more sped up” with current tools. They tried to run a follow-up study but hit serious methodological problems as developers refused to participate without AI access, and 30-50% cherry-picked which tasks to submit. METR called the new data “unreliable” and are redesigning the experiment. The productivity question turns out to be genuinely hard to measure.

However the perception gap and automation bias aren’t about model capability, they’re about human cognition. Better models produce more polished output, which makes subtle errors harder to spot, not easier. Philipp Dubach’s March 2026 analysis puts it plainly: 93% adoption, roughly 10% productivity gains. His explanation via Amdahl’s Law — coding is only 25-35% of the job, so even a 100% coding speedup yields 15-25% total improvement — is consistent with everything in DORA 2025 and the Faros AI data.

The structural argument in this essay is model-independent; governance, specs and review capacity. DORA 2025’s core finding is that AI amplifies existing patterns: strong teams get stronger, weak teams get weaker. That’s not going to change with better models. If anything, it gets more true.

What started as personal frustration — why are these agents so productive and so stupid at the same time? — turned into a month of research. The findings are collected in The Agent-Era Handbook, an open-source framework for teams navigating this shift. If this piece hit a nerve, subscribe. Next up: why the hardest part of AI-assisted development has nothing to do with the AI — it’s what happens to your identity when you stop writing code and start directing machines.

Sources