The Evidence Base

This section establishes the problem set. Each finding below is a specification for something Part II tells you to build, not a reason for hesitation.

Three caveats govern everything. Much of the strongest data comes from vendors selling into the market they measure, and is labeled where that is the case. Almost none of it is causal. And the cleanest instrument—the randomized controlled trial—is losing viability in this domain for reasons discussed in 3.4.

3.1 Adoption Is Saturated; Trust Is Not #

The 2025 DORA research, drawing on approximately five thousand technology professionals, found 90 percent using AI at work with a median of two hours per workday, and more than 80 percent reporting increased personal productivity—against 30 percent reporting little or no trust in AI-generated code.²³ The 2025 Stack Overflow survey found the same shape: 84 percent using or planning to use AI tools, 66 percent naming “almost right, but not quite” as their top frustration, and trust falling eleven points year over year to 29 percent.^24,118 Stack Overflow’s own analysis calls it “a counterintuitive dip in trustworthiness that correlates with higher adoption rates.”

At organizations further along, adoption is effectively total. Spotify reports 99 percent of engineers using AI coding tools weekly and 94 percent reporting increased productivity.¹⁰¹ Uber reports 84 percent using agentic coding tools and 92 percent using agents monthly.¹⁰² Telemetry across five hundred organizations puts utilization above 90 percent industry-wide.¹⁰⁴

Adoption rose while trust fell. That is not irrational; it is what a genuinely useful and genuinely unreliable tool produces. A survey of 3,500 developers and managers captures the same tension from the other side: 99 percent report AI time savings and 68 percent save more than ten hours a week, while 50 percent lose ten or more hours weekly to non-coding work—the saving and the loss are the same magnitude, and 63 percent say leaders do not understand their pain points, up sharply year over year.²¹¹

It is also the signature of a workforce whose burden has shifted from producing to verifying—which is precisely what the rest of this section documents.

3.2 What Acceleration Costs Downstream #

DORA’s 2024 research estimated that a 25 percent increase in AI adoption was associated with a 1.5 percent decrease in throughput and a 7.2 percent decrease in delivery stability.²⁵ The 2025 research reversed the first and not the second: a positive relationship with throughput and product performance, and a continuing negative relationship with delivery stability.²³ Anyone citing the throughput finding as current is citing superseded data; anyone citing the reversal as a clean win is omitting half the sentence.

DORA’s explanation is the load-bearing part: “Without robust control systems, like strong automated testing, mature version control practices, and fast feedback loops, an increase in change volume leads to instability.”²³ AI amplifies what is already there.

The magnitude of the gain is smaller and more context-dependent than marketing suggests. The most-cited figure—55.8 percent—comes from a 2022 vendor study of ninety-five developers writing an HTTP server from scratch, with a confidence interval spanning 21 to 89 percent.²⁶ The largest randomized trial available, 4,867 developers across three field experiments, found a 26 percent increase in completed tasks, concentrated heavily by seniority: 21 to 40 percent for junior and short-tenure developers against 7 to 16 percent for seniors.²⁷ A Google internal trial of ninety-six engineers found 21 percent and said plainly that “our confidence interval is large.”²⁸ Telemetry across four hundred organizations puts median pull request throughput improvement at 7.8 percent, mean 13.1 percent, ninetieth percentile 43.9 percent—a right-skewed distribution in which a few teams do very well and most see single digits.²⁹

And at the organizational level the gains largely stop showing up. Quarterly AI spend across a five-hundred-organization panel rose roughly twenty-eight-fold while the innovation ratio moved from 57 to 58 percent.¹⁰⁴ One telemetry set shows deployments per week down 11.7 percent in a measured subset even as task completion rose.¹¹⁹ Individual-level gains are real and are not aggregating.

3.3 The Absorption Constraint Is Measurable #

This is the finding that should shape policy, and it is the least discussed.

Telemetry from approximately twenty-two thousand developers across four thousand teams, comparing low- and high-adoption periods within the same organizations, found tasks completed per developer up 34 percent and epics up 66 percent—alongside median review time up 500 percent, pull requests merged without review up 31.3 percent, bugs per developer up 54 percent, and the incidents-to-pull-request ratio more than tripling.^30,119 Average pull request size rose 51.3 percent, files touched per developer per month 149.9 percent, and commit-to-production lead time 480.4 percent.

Median review time up five-fold alongside a third more changes merged with no review at all is the empirical signature of a saturated review system. Reviewers are either queuing or waiving, and both failure modes appear simultaneously in the same dataset.

Independent corroboration arrives from every direction. Microsoft’s ten-month agent report records 16.5 review comments per merged agent pull request against 12.4 for human pull requests, with 61 percent of all feedback coming from ten people.¹⁰⁵ Google throttled migrations deliberately to protect reviewers.¹⁰⁷ An eleven-thousand-pull-request academic study found 79.1 percent of a manually inspected merged sample showing no observable reviewer interaction at all.³¹ Concurrency compounds it: across 33,596 agent-authored pull requests, 79.4 percent temporally overlap with other agent pull requests, and the textual conflict rate roughly doubles from 19.8 percent within one agent product to 41.7 percent across products.³²

DORA’s 2026 return-on-investment work names the mechanism directly, calling it “the verification tax imposed by reviewing AI-generated code” and modeling it as one of three causes of a J-curve dip before returns arrive.¹²⁰

3.4 Why the Cleanest Evidence Is Disappearing #

In July 2025 a randomized controlled trial of sixteen experienced open-source developers found that allowing AI tools increased task completion time by 19 percent, while participants forecast a 24 percent speedup beforehand and still believed they had experienced a 20 percent speedup afterward.³³

The February 2026 follow-up matters more and is widely misreported as a reversal. Across fifty-seven developers, 143 repositories, and more than eight hundred tasks, the estimate was −18 percent for returning participants and −4 percent for new recruits, both confidence intervals now crossing zero.³⁴ Negative speedup still means slower; the effect attenuated toward statistical indistinguishability rather than reversing.

The reason the researchers abandoned the design is the important finding. Between 30 and 50 percent of developers declined to submit tasks because they did not want to do them without AI, and concurrent multi-agent workflows made time tracking unreliable. A no-AI control arm is no longer a condition professional developers will accept.

Plan on the assumption that clean causal evidence about AI-assisted productivity is not coming. Internal measurement—instrumented, longitudinal, against your own baseline—is now the only reliable instrument you have. This is not a reason to avoid the transition; it is a reason to build measurement before making the transition, because you will not be able to borrow anyone else’s.

3.5 Quality Does Not Improve On Its Own #

The longitudinal security finding is the most important input to this framework’s design. Across more than 150 models, using eighty coding tasks in four languages, models produce secure code in 55 percent of cases against syntactic correctness above 95 percent—essentially flat for two years, over a period in which capability on software engineering benchmarks improved dramatically.^15,35 By language: Python 62 percent, C# 58 percent, JavaScript 57 percent, Java 29 percent. By vulnerability class the spread is extreme—SQL injection 82 percent and insecure cryptography 86 percent against cross-site scripting 15 percent and log injection 13 percent. Model size has near-zero effect.

The defect distribution is shifting as well as persisting. Vendor telemetry from Fortune 50 environments reports AI-assisted developers producing three to four times more code and roughly ten times more security findings, with privilege escalation paths up 322 percent and architectural design flaws up 153 percent, while syntax errors fell 76 percent and logic bugs fell 60 percent.³⁶ That research is correlational and vendor-published. The structural claim survives the caveats: defects are moving away from the local syntactic errors static analysis catches well, toward the architectural and privilege-boundary flaws it catches poorly.

Independent, non-vendor corroboration exists for one non-functional attribute. The 2026 WebAIM Million study found detected accessibility conformance failures on 95.9 percent of the top million home pages, up from 94.8 percent—reversing a multi-year improvement trend—with average errors per page up 10.1 percent and elements per page up 22.5 percent in a single year, attributed in part to “automated or AI-assisted coding practices.”³⁷

The point is not that agents write bad code. It is that quality attributes which are not specified, oracled, and enforced will degrade, at population scale, without anyone deciding to degrade them.

3.6 Codebases Are Drifting Toward Duplication #

Analysis of 623 million code changes between 2023 and 2026 found refactoring activity falling from 21 percent of changed lines in 2022 to 3.8 percent in 2026, while copy-pasted code rose from 9.4 percent to 15.7 percent.³⁸ Block duplication rose 81 percent, from 40.3 to 73.0 duplicated lines per million changed. Cross-file function calls, a proxy for reuse, fell 35 percent. Error-masking constructs rose 47 percent. Developers are now roughly five times more likely to copy than to refactor, inverting a two-to-one preference for refactoring in 2022.

A study of 304,362 verified AI-authored commits found more than 15 percent of commits from every assistant introducing at least one issue, 24.2 percent of those issues surviving to the latest revision, and security issues persisting at 41.1 percent against 22.7 percent for code smells.³⁹ That study has no human-written control group, so the rate is absolute rather than comparative—anyone citing it as proof that AI writes worse code is overreaching.

The mechanism is not mysterious. Generation is locally optimal and globally indifferent: a model asked to implement a function implements it, rather than noticing that something similar exists three modules over. Reuse and consolidation are now architectural decisions that must be designed for and enforced, because they will not emerge.

3.7 What Agents Actually Achieve #

The honest picture is neither the benchmark nor the skeptic’s.

Merge rates in production. Microsoft: 68.6 percent across seven repositories, against 87.1 percent for its own employees and 79.7 percent for community contributors in the same repository.¹⁰⁵ Academic study across five agent products: Cursor 71.4 percent, Codex 71.0 percent, Claude Code 61.0 percent, Copilot 58.8 percent, Devin 53.6 percent.³¹ A study of 567 pull requests in self-selected settings found 83.8 percent against 91.0 percent for matched human pull requests.¹²¹

Success varies enormously by task type and context. Microsoft’s breakdown: removal and cleanup 84.7 percent, testing 75.6 percent, refactoring 69.7 percent, bug fixes 69.4 percent, documentation 68.1 percent, features 64.5 percent, performance work 54.5 percent.¹⁰⁵ Greenfield beats brownfield measurably—77.3 percent on a new repository against 67.9 percent on a mature one—and one independent evaluation of an autonomous agent scored zero for four on modifying existing projects.^105,122 Language matters more than tool choice: one enterprise test-generation system reports viable-test rates of roughly 80 percent for Python, 40 percent for Go, and 20 percent for Java, on the same tool in the same monorepo.¹²³

Benchmarks systematically overstate. Agents resolving 72.8 percent of a widely used verified benchmark resolve 18.75 to 25 percent of realistic long-horizon software evolution tasks drawn from release notes.¹¹⁵ One major provider retired that benchmark entirely in February 2026, having audited 138 problems and found 59.4 percent with material test-design issues, alongside evidence of contamination.¹²⁴ A vendor audit of 731 successful agent trajectories found 63 percent of resolutions retrieved the fix rather than derived it—57 percent by locating the merged upstream fix on the public web, 9 percent by mining bundled git history for the future fix commit—with scores dropping 14 to 21 points once history and network access were restricted.¹²⁵

And the ceiling on unsupervised delegation is currently low. Developers report being able to fully delegate only 0 to 20 percent of tasks while using AI in roughly 60 percent of their work, delegating what is easily verifiable or low-stakes and retaining high-level design.¹²⁶

3.8 The Summary Position #

The picture supports a specific posture. Agents are already producing the majority of change at the organizations furthest along, and those organizations got there by building verification and absorption capacity rather than by waiting for better models. The constraint is real, it is measurable, and it is buildable. That is what the rest of this framework is for.