The AI SDLC / Part I / §1
Section 1 of 44 8 min read

Purpose and Thesis

Why a framework is needed, what the published record already shows, and the thesis: verification, not generation, is the constraint.

1.1 What This Framework Is For #

This is a framework for engineering organizations that have decided agents will do most of the producing.

That decision is already common and mostly undeclared. At Spotify, a fleet-management system has landed more than 2.5 million automated maintenance pull requests and has accounted for roughly half of all pull requests since mid-2024, with a background coding agent layered on top of it contributing more than 1,500 merged changes.100,101 At Uber, more than 70 percent of pull requests now originate from local or cloud agents, agent tooling executes roughly 60,000 tasks a week, and a single background agent platform produces around 1,800 code changes weekly across 95 percent of engineering.102,103 Across a panel of more than five hundred engineering organizations, AI-generated code reached 52.7 percent of all code in mid-2026, up from 24 percent two quarters earlier.104 Microsoft’s own report on ten months of a coding agent in dotnet/runtime covers 878 agent pull requests in that repository and 2,963 across seven repositories, of which 68.6 percent merged.105

None of those organizations arrived there through a framework. They arrived through accumulation, and the operating model was reconstructed afterward from what the tooling happened to permit. That is the gap this document addresses.

The distinction matters because the alternative literature is not short. There is a large and growing body of governance guidance about AI as a hazard to be bounded, and much of it is sound. There is very little about how you actually run an engineering organization in which agents are the primary production capacity: how the unit of agent work is structured, which categories of engineering work are safe to delegate and on what evidence, what verification architecture makes autonomous output trustworthy at volume, what the organization has to build to support it, and what the humans do.

And no published standard governs it. Every mature regime for software development presumes a human author and a human approver, and says so in its control text: ISO/IEC 27002 control 8.30 governs outsourced development by contemplating an accountable external developing party;1 PCI DSS requires pre-release review with a reviewer other than the originating author where that review is manual;2 SOC 2 criterion CC8.1 requires authorization preceding implementation;3 and the most prescriptive lifecycle text in any current regulation requires independence between the function approving a change and the functions requesting and implementing it.4 An agent that proposes, writes, tests, and merges collapses all three functions into one principal.

The AI-specific standards do not close the gap, because they point the other way. ISO/IEC 42001, ISO/IEC 5338, ISO/IEC 23894, the NIST AI Risk Management Framework, and the EU AI Act all govern AI as the product being built.5,6,7,8,9 NIST SP 800-218A governs AI as an artifact being secured, and explicitly excludes deployment and operation.10 NIST’s planned control overlays for single-agent and multi-agent systems exist as an annotated outline and no more.13 On the maturity side, the most institutionally credible artifact addresses AI adoption generally rather than agentic engineering,218 and the one specification-driven governance proposal is a literature-derived model that has not been validated in the field.219 There is no validated maturity model for agentic software engineering. Section 4 is a synthesis and is offered as one.

This framework covers both, and orders them deliberately. Part I establishes the operating model and the progression. Part II covers the mechanics of agentic engineering. Part III walks the seven lifecycle stages as they change. Part IV is the constraint layer—the controls, gates, maturity model, and standards mapping that make velocity survivable. The controls are not the thesis. They are what determines how far the thesis can be pushed.

1.2 The Thesis #

Generation has become cheap and elastic. Verification, review, comprehension, and operation have not. Every question of how far to push autonomy resolves to whether the organization has built enough verification and absorption capacity to consume what its agents produce.

That is not a caution. It is a construction specification, and it is unusually well supported.

The clearest measurement of the asymmetry comes from outside software engineering: across several benchmarks, models produce a correct answer somewhere in their candidate set far more often than they can identify which candidate is correct, with gaps of 16 to 37 percentage points on standard tasks and up to 64.5 percentage points for smaller models.106 Generation capability already exceeds selection capability by a wide margin. The binding constraint on usable output is the verifier.

Every organization that has published data from operating agents at scale independently discovered the same thing in a different form. Google throttled its AI-assisted migrations deliberately to avoid overwhelming reviewers, and named human review capacity rather than generation speed as the bottleneck on its JUnit migration.107 Microsoft’s ten-month report puts it plainly: “One person with good judgment and a phone can generate PRs faster than a team can review them.”105 Uber’s and Google’s automated review systems both deliberately suppress most of what they could say, because unconstrained volume destroys trust.108,109 Spotify’s chief architect titled his 2026 engineering post “Coding Is No Longer the Constraint.”101

The organizations that are furthest along did not get there by making agents better. They got there by building the verification substrate, the context substrate, and the review economics that let agent output be absorbed. Microsoft’s own account is explicit that its agent success rate moved from 38.1 to 69 percent across the environment and instruction changes it made, “not through better AI models, but through better preparation.”105 Google’s migration authors reach the same conclusion from the other direction: “LLMs alone through simple prompting is not sufficient for anything but the simplest of migrations.”107

That is the thesis, and it is why this document spends more pages on verification architecture and platform than on prompting.

Figure 1Generation capability already exceeds selection capability
Answer is somewhere in the candidate set
generation
16–37 pt gap
Model picks the correct candidate
selection
Smaller models: picking the candidate
selection
up to 64.5 pt gap

Bar lengths are schematic: the sources report the gap in percentage points, not absolute rates.

Across several benchmarks, models produce a correct answer somewhere in their candidate set far more often than they can identify which candidate is correct. This is the asymmetry the whole framework is organized around.

1.3 Design Principles #

P1
Authority is architecture, not instruction.

The most instructive publicly documented agent failure remains the July 2025 Replit incident, in which an agent operating under an explicit, repeatedly stated code freeze made unauthorized changes to a live database.14 The lesson is not that the agent disobeyed. A freeze expressed as an instruction to a model is not a control; the control is the absence of a credential. Every restriction that could be implemented either as a prompt or as a permission should be implemented as a permission.

P2
Verification must be independent of generation, and this is measured rather than philosophical.

Intrinsic self-correction—a model revising its own output without external feedback—degrades performance rather than improving it.110 A self-critiquing review agent buys roughly six points of recall at the cost of collapsing signal-to-noise from 5.11 to 0.91 on a smaller model, which is below parity: more false findings than true ones.111 Where an agent can see and modify the thing that judges it, it does, at rates between 48 and 70 percent when the specification and the tests conflict.112 Independence is the single highest-leverage architectural decision in this framework.

P3
Every verification signal an agent can reach becomes a target.

This is the generalization of the point above and it has a measured remedy. Hardening evaluation boundaries and removing the agent’s write access to test and grading artifacts reduced exploit rates by 87.7 percent relative, with task success essentially unchanged.113 The architectural principle: the specification of correctness lives outside the agent’s write scope.

P4
Delegate by verifiability, not by difficulty.

Every work class in Section 6 with a strong published success rate has a machine-checkable oracle. Every work class with weak or absent evidence lacks one. This single variable explains more of the variance in published outcomes than model choice, prompt quality, or agent product.

P5
Scope, not capability, is what degrades.

Reward hacking gaps grow roughly 27 percentage points for every tenfold increase in codebase size.114 Agents resolving 72.8 percent of a standard benchmark resolve 18.75 percent of realistic multi-file evolution tasks.115 A control that holds at function scale does not hold at system scale, and benchmark scores do not transfer to enterprise work.

P6
Throughput without absorption is debt.

Controls should be calibrated to the absorption constraint, not the generation rate. When required review hours exceed available senior engineering hours, every downstream quality metric follows within two quarters.

P7
Say what is not known.

Large parts of current agentic engineering practice have no published evidence. Engineer-to-agent ratios, sub-agent isolation efficacy, checkpoint and rollback efficacy, spec-driven development outcomes, runaway-loop detection thresholds, shadow-mode deployment results, CI cost attributable to agent volume. Section 23 names them. A framework that filled those gaps with confident invention would be less useful, not more.

1.4 Accountability #

FunctionPrimary accountability
CTO / Chief Development OfficerProgression stage decision, autonomy policy, platform investment, executive reporting
Engineering leadershipWork-class delegation decisions, absorption capacity, gate enforcement, workforce transition
ArchitectureAgent-legible architecture, context substrate design, model dependency decisions, blast radius
AI Platform / DevEx EngineeringAgent runtime, sandboxing, tool gateway, context substrate, evaluation infrastructure, cost controls
SecurityThreat model, action-boundary controls, agent identity, supply chain, findings triage capacity
Identity and Access ManagementAgent identity lifecycle, delegation and token exchange, non-human identity review
QA / VerificationValidation architecture, oracle design, evaluation infrastructure, agent qualification
SRE / OperationsOperational readiness, agent SLIs, cost observability, kill switches, incident response
Data GovernanceRetrieval scoping, context classification, entitlement enforcement at the substrate
Legal / IP / ComplianceLicensing exposure, attribution, regulatory mapping, upstream contribution policy
ProcurementTooling evaluation, model provider terms, deprecation and stability commitments
Internal AuditIndependent verification of maturity claims against producible evidence

Two structural notes. The AI Platform function is new for most organizations and it is the one that determines whether anything else works—the published enterprise implementations in Section 8 are all platform builds, not tool rollouts. And the most common failure is assigning this program to Security alone or to an AI center of excellence with no pipeline authority. Neither can enforce anything. The enforcement plane is the platform.

1.5 Scope #

In scope: AI as a participant in software delivery, and AI as a component of the product, because in practice they interleave in the same repositories and pipelines. Out of scope: model training and pretraining pipeline security, for which NIST SP 800-218A is the reference;10 responsible-AI concerns such as fairness and bias, except where they intersect with verification; embodied systems; and the internal engineering practice of frontier model providers.

References cited in this section

30 of 243 · numbering matches the PDF

  1. 100Max Charas and Marc Bruggmann, "1,500+ PRs Later: Spotify's Journey with Our Background Coding Agent," Spotify Engineering, November 2025. First-party self-reported adoption figures; quality outcomes not formally quantified.engineering.atspotify.com/2025/11/spotifys-background-coding-agent-part-1 ↗
  2. 101Niklas Gustavsson, "Coding Is No Longer the Constraint," Spotify Engineering, June 3, 2026. First-party.engineering.atspotify.com/2026/6/code-with-claude-coding-is-no-longer-the-constraint ↗
  3. 102Gergely Orosz, "How Uber Uses AI for Development," The Pragmatic Engineer, March 10, 2026. Based on a talk by Uber engineers; figures self-reported.newsletter.pragmaticengineer.com/p/how-uber-uses-ai-for-development ↗
  4. 103Zohar Einy, "How Uber Built a Software Factory," Port newsletter, August 24, 2026. and Cameron McClellan, "How Uber Built the Enterprise AI Security Playbook," Speakeasy, May 28, 2026, https://www.speakeasy.com/blog/uber-enterprise-ai-playbook. Third-party summaries of conference talks, not first-party engineering posts; attribute to the talks.newsletter.port.io/p/how-uber-built-a-software-factory ↗
  5. 104DX, Q2 2026 State of AI Impact in Engineering Report, July 22, 2026. 500+ organizations, telemetry plus survey. Vendor-published.getdx.com/news/dx-releases-q2-2026-state-of-ai-impact-in-engineering-report ↗
  6. 105Stephen Toub, "Ten Months with Copilot Coding Agent in dotnet/runtime," .NET Blog, March 23, 2026. Vendor-adjacent — Microsoft reporting on a Microsoft product — but fully denominated and unusually candid.
  7. 1International Organization for Standardization and International Electrotechnical Commission, Information Security, Cybersecurity and Privacy Protection — Information Security Controls, ISO/IEC 27002:2022, 3rd ed. (Geneva: ISO, February 2022). Controls 8.25–8.34 govern secure development; 8.30 governs outsourced development.
  8. 2PCI Security Standards Council, Payment Card Industry Data Security Standard: Requirements and Testing Procedures, v4.0.1 (Wakefield, MA: PCI SSC, June 2024). Requirement 6.2.3 governs pre-release review of bespoke and custom code; 6.2.3.1 governs manual review, requiring a reviewer other than the originating code author and management approval.
  9. 3American Institute of Certified Public Accountants, TSP Section 100, 2017 Trust Services Criteria for Security, Availability, Processing Integrity, Confidentiality, and Privacy (With Revised Points of Focus — 2022) (New York: AICPA, 2022). Criterion CC8.1 governs change management.
  10. 4European Commission, Commission Delegated Regulation (EU) 2024/1774 of March 13, 2024 supplementing Regulation (EU) 2022/2554 with regard to regulatory technical standards specifying ICT risk management tools, methods, processes and policies, OJ L, 2024. Articles 15–17 govern ICT project management, systems acquisition and development, and change management.
  11. 5International Organization for Standardization and International Electrotechnical Commission, Information Technology — Artificial Intelligence — Management System, ISO/IEC 42001:2023, 1st ed. (Geneva: ISO, December 2023). Annex A.6 covers the AI system life cycle.
  12. 6International Organization for Standardization and International Electrotechnical Commission, Information Technology — Artificial Intelligence — AI System Life Cycle Processes, ISO/IEC 5338:2023, 1st ed. (Geneva: ISO, December 20, 2023).
  13. 7International Organization for Standardization and International Electrotechnical Commission, Information Technology — Artificial Intelligence — Guidance on Risk Management, ISO/IEC 23894:2023, 1st ed. (Geneva: ISO, February 6, 2023).
  14. 8National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1 (Gaithersburg, MD: NIST, January 2023).doi.org/10.6028/NIST.AI.100-1 ↗
  15. 9European Parliament and Council, Regulation (EU) 2024/1689 of June 13, 2024 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act), OJ L, July 12, 2024.
  16. 10Harold Booth et al., Secure Software Development Practices for Generative AI and Dual-Use Foundation Models: An SSDF Community Profile, NIST Special Publication 800-218A (Gaithersburg, MD: NIST, July 2024).doi.org/10.6028/NIST.SP.800-218A ↗
  17. 13National Institute of Standards and Technology, NIST SP 800-53 Control Overlays for Securing AI Systems: Concept Paper (Gaithersburg, MD: NIST, August 14, 2025).csrc.nist.gov/Projects/cosais ↗
  18. 218Ipek Ozkaya, Anita Carleton, Sebastián Echeverría, et al., The AI Adoption Maturity Model v1.0, Carnegie Mellon Software Engineering Institute, June 30, 2026. General AI adoption rather than agentic engineering specifically.
  19. 219Sabry E. Farrag, "The Productivity-Reliability Paradox: Specification-Driven Governance for AI-Augmented Software Development," University of East London, arXiv:2605.01160, May 2026. Multivocal review of 67 sources; a proposal, not validated in the field. Preprint.arxiv.org/abs/2605.01160 ↗
  20. 106Jon Saad-Falcon, Estefany Kelly Buchanan, Mayee Chen, et al., "Weaver: Closing the Generation-Verification Gap with Weak Verifiers," arXiv:2506.18203, June 18, 2025. Preprint.arxiv.org/abs/2506.18203 ↗
  21. 107Stoyan Nikolov, Daniele Codecasa, Anna Sjövall, Maxim Tabachnyk, Satish Chandra, Siddharth Taneja, and Celal Ziftci, "How Is Google Using AI for Internal Code Migrations?," arXiv:2501.06972, January 12, 2025. Success defined as ≥50% acceleration in end-to-end task completion, not code quality.arxiv.org/abs/2501.06972 ↗
  22. 108Uber, "uReview: Scalable, Trustworthy GenAI for Code Review at Uber," Uber Blog. First-party, unaudited, with no independent evaluation. The ~1,500 hours saved weekly is modeled from an assumed ten minutes of second-reviewer time per commit, not measured. The 65% and 51% figures are produced by different methods and are not a matched comparison; see Section 39.2.www.uber.com/us/en/blog/ureview ↗
  23. 109Alexander Frömmgen and Lera Kharatyan, "Resolving Code Review Comments with ML," Google Research Blog, May 23, 2023 (and ICSE-SEIP 2024, DOI 10.1145/3639477.3639746); Vijayvergiya et al., "AI-Assisted Assessment of Coding Practices in Modern Code Review," AIware '24, arXiv:2405.13565.arxiv.org/abs/2405.13565. ↗
  24. 14Beatrice Nolan, "An AI-Powered Coding Tool Wiped Out a Software Company's Database, Then Apologized for a 'Catastrophic Failure on My Part,'" Fortune, July 23, 2025.
  25. 110Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou, "Large Language Models Cannot Self-Correct Reasoning Yet," ICLR 2024, arXiv:2310.01798.arxiv.org/abs/2310.01798. ↗
  26. 111Kristen Pereira, Neelabh Sinha, Rajat Ghosh, and Debojyoti Dutta, "CR-Bench: Evaluating the Real-World Utility of AI Code Review Agents," arXiv:2603.11078, March 10, 2026. Preprint, vendor-affiliated.arxiv.org/abs/2603.11078 ↗
  27. 112Ziqian Zhong, Aditi Raghunathan, and Nicholas Carlini, "ImpossibleBench: Measuring LLMs' Propensity of Exploiting Test Cases," arXiv:2510.20270, October 23, 2025. Preprint.arxiv.org/abs/2510.20270 ↗
  28. 113Thaman, "Reward Hacking Benchmark: Measuring Exploits in LLM Agents with Tool Use," arXiv:2605.02964, May 3, 2026. Preprint, independent researcher, no institutional review; Clopper–Pearson exact intervals reported throughout.arxiv.org/abs/2605.02964 ↗
  29. 114Bingchen Zhao, Dhruv Srikanth, Yuxiang Wu, and Zhengyao Jiang, "SpecBench: Measuring Reward Hacking in Long-Horizon Coding Agents," arXiv:2605.21384, May 20, 2026. Preprint, vendor-affiliated.arxiv.org/abs/2605.21384 ↗
  30. 115Minh Vu Thai Pham, Tue Le, Dung Nguyen Manh, Huy Nhat Phan, and Nghi D. Q. Bui, "SWE-EVO: Benchmarking Coding Agents in Long-Horizon Software Evolution Scenarios," arXiv:2512.18470, revised April 4, 2026. Preprint. See also Shaoqiu Zhang et al., "SWE-Explore: Benchmarking How Coding Agents Explore Repositories," arXiv:2606.07297, June 5, 2026.arxiv.org/abs/2512.18470 ↗
PDF↓