AI Makes Writing Code Cheaper. It Does Not Make Engineering Judgment Cheaper.

AI coding assistants have made one part of software engineering dramatically cheaper: turning an idea into plausible-looking code. But typing was never the expensive part. The expensive parts are deciding what to build, verifying that the code is correct, secure and maintainable, and owning it when it fails at 3 a.m. Those costs haven’t dropped; for many teams they have risen, because there is now much more code to check. This post looks at what the research says, where the bottleneck moves, and how individuals and teams can adapt.

Why It Matters

When the cost of one step in a process drops, the bottleneck moves to the next step. Teams that measure output (lines of code, pull requests, story points) will see impressive numbers while review queues grow, incidents creep up and nobody fully understands parts of their own system. Teams that understand where the cost moved can actually turn cheaper code into faster, safer delivery.

What the Evidence Says

The research so far is mixed, and the differences are instructive:

Both results point the same way: generating code faster doesn’t automatically make delivery faster or safer, and people are poor judges of their own speed.

Where the Cost Moved

Verification Is Now the Bottleneck

AI-generated code is fluent, which makes it look reviewed. It can:

• call APIs that don’t exist
• handle the happy path and miss the edge cases
• introduce subtle security issues
• duplicate logic that already exists elsewhere

Someone has to catch all of that, and reading code carefully is slower than writing it. If review standards drop because the code looks plausible, defects move into production; if they don’t, reviewers become the constraint.

Judgment Is What Remains Scarce

AI is good at producing an answer to the question you asked. It is much weaker at noticing you asked the wrong question. The decisions that matter most (what to build, which trade-offs to accept, where to draw boundaries, what can go wrong in production) still depend on context that lives in people’s heads, in conversations with users and in the history of the system.

Ownership Doesn’t Transfer

Whoever merges a change owns it: they will be asked why it does what it does, and they will debug it during an incident. Code nobody on the team understands is a liability, however fast it was produced. It is technical debt with an interest rate nobody has priced.

Where AI Clearly Helps

How to Adapt as an Engineer

• Write the tests or acceptance criteria first. They turn “looks right” into “is right” and are the cheapest verification there is.
• Keep changes small. Small batches are easier to review and safer to deploy, which is exactly what DORA’s numbers suggest AI erodes.
• Know the fundamentals. You can only catch a wrong answer about the event loop or promise ordering if you know the right one.
• Don’t outsource understanding. If you couldn’t have written it, you can’t review it. Ask the AI to explain, then verify the explanation.

How to Adapt as a Team

• Measure outcomes, not output. More pull requests and lines of code are now nearly free; lead time, change failure rate and customer impact are not (see the metrics that make engineering organizations worse).
• Invest in automated verification. Type checking, tests, linters, static analysis and preview environments scale; human attention doesn’t.
• Protect review capacity. Set a size limit for changes, make review part of everyone’s job, and don’t let review time be the hidden cost of “faster” coding.
• Set clear rules for data and security. Decide which tools may see which code and data, and never paste secrets or customer data into prompts.
• Keep growing people. Junior engineers learn by struggling with problems; if AI removes all the struggle, it also removes the learning. Pair them with experienced reviewers and ask them to explain what they ship.

Going Deeper: Verification as a System

Individual review doesn’t scale with the volume of generated code. Teams that get real gains from AI treat verification as a pipeline, with cheap automated checks catching most problems before a human looks, and every later stage being more expensive than the one before:

How Much Autonomy to Give an AI Agent

Agents that run commands and edit many files raise the stakes. A useful way to decide is to match autonomy to how reversible the agent’s actions are:

The Replit incident above happened at the last level, without those guardrails. Every level below it keeps a human decision between the AI and anything irreversible.

Measuring Whether AI Helps

Run it like any other change to how you work: pick a baseline period, then compare outcome and delivery metrics after adoption, not output.

• Useful signals: change lead time, change failure rate, review time per change, change size, escaped defects and the team’s own survey answers.
• The warning sign: pull requests per week rise while lead time, review queues and failure rates also rise. The bottleneck has moved to verification, as DORA’s data suggests.

Test Yourself: Scenarios

There is no single right answer to these, but there are strong and weak ones. Think through yours, then compare.

Senaryo 1

The 2,000-Line Pull Request

A teammate opens a pull request of about 2,000 lines, mostly generated by an AI assistant. CI is green. The feature is due this week.

Cevabı gösterCevabı gizle

Green CI shows the code compiles and that the tests that exist pass; it says nothing about the tests that don’t exist. Before reviewing line by line:

1. Ask for it to be split into reviewable pieces (data model, API, UI), each one deployable, ideally behind a feature flag. Large batches are exactly what DORA links to lower stability.
2. Ask the author to walk you through it. If they can’t explain a part, it isn’t ready, whoever wrote it.
3. Check the tests first. Do they cover edge cases and failure paths, or only the happy path the AI generated alongside the code?
4. Look for the usual AI failure modes: invented APIs, duplicated helpers that already exist, missing authorization checks, swallowed errors.

On the deadline: shipping a smaller, well-understood slice this week is usually better than shipping an unreviewed large one.

Senaryo 2

“We’re 40% More Productive”

Six months after rolling out AI tools, a dashboard shows pull requests per engineer up 40%. Leadership asks whether engineering got faster.

Cevabı gösterCevabı gizle

Pull request count is an output metric, and AI makes output cheap, so it can’t answer the question. Look at outcomes and flow over the same period:

• Did change lead time drop?
• Did change failure rate and incident count stay flat?
• Are review queues growing?
• Did customer-facing delivery (features adopted, bugs fixed) speed up?
• Change size: more, smaller PRs is healthy; more, larger PRs is a warning sign.

Report the honest picture, including where AI clearly helped (tests, migrations, boilerplate) and where the bottleneck moved. See the metrics that make engineering organizations worse.

Senaryo 3

Fast but Fragile

A junior engineer ships features quickly with AI help, but during an incident they can’t explain how their code works.

Cevabı gösterCevabı gizle

Treat it as a learning gap, not a tooling problem. Agree on one rule: anything you merge, you must be able to explain and debug without the AI. In practice:

• write the tests or acceptance criteria before generating code
• explain their PRs in review (“why this approach?”)
• rewrite one generated piece by hand each week
• pair with a senior reviewer on incident follow-ups

The goal is to keep AI’s speed while making sure the struggle that builds understanding still happens.

Senaryo 4

An Agent for Dependency Updates

You want an AI agent to keep dependencies up to date by opening pull requests automatically.

Cevabı gösterCevabı gizle

It is a good fit: the work is repetitive, and every change is a reviewable diff. Set it up so mistakes are cheap:

• Sandbox: no production credentials, and it can’t merge.
• Small PRs: one dependency (or one group) per PR, with its changelog linked.
• Full CI: the whole test suite, plus a lockfile and license check.
• Human review always for major versions and security-sensitive packages.
• An off switch: one setting turns the whole thing off.

Measure success by how many of its PRs merge without rework.

Key Takeaways

Sources

• Google Cloud and DORA: Impact of generative AI in software development
• METR: Measuring the impact of early-2025 AI on experienced open-source developer productivity (July 2025)
• DORA: 2024 Accelerate State of DevOps report
• InfoQ: Summary of the 2024 DORA report
• eWEEK: AI agent wipes production database, then lies about it (July 2025)
• OWASP: Top 10 for large language model applications
• Martin Fowler: Cannot measure productivity