Stage Three — Design
How the system satisfies its requirements, and how it gets built, verified, and operated when agents write much of it.
Determine how the system will satisfy its requirements—and, distinctly, how the system will be constructed, verified, and operated in an environment where a substantial fraction of its code is generated.
12.1 What Changes at This Stage #
Design acquires a second audience. Architecture has always been written for the engineers who will implement and maintain the system. It is now also read, at scale and without the ability to ask questions, by the agents that will implement it. A design that relies on tacit team knowledge, on conventions that live in people’s heads, or on the implicit understanding that “we don’t do it that way here,” will not survive contact with a generative process that has no access to any of that.
This produces a design obligation that has no precedent: architecture must be legible to a non-participant. The mechanisms are not exotic—explicit module boundaries, dependency rules stated rather than implied, naming conventions written down, a documented rationale for the patterns in use, machine-readable constraints where the toolchain supports them. What is new is that the cost of illegibility is now paid continuously and automatically rather than occasionally at onboarding.
Second, the maintainability data in Section 3.6 should be read as a design finding, not merely a quality finding. Refactoring collapsing from 21 percent to 3.8 percent of changes, cross-file function calls down 35 percent, and duplication up 81 percent describe a codebase drifting toward local, duplicated, low-reuse structure.38 Generation is locally optimal and globally indifferent: a model asked to implement a function will implement it, not notice that something similar exists three modules over. Reuse, abstraction, and consolidation are now architectural decisions that must be designed for and enforced, because they will not emerge from the generation process.
Third, design is where the blast radius of every downstream agent action is set. An agent can only do damage in proportion to what the architecture lets any single component reach. Module boundaries, credential scoping, environment separation, and the blast radius of a bad deployment are agent-safety controls, whatever else they were designed to be.
Fourth, where the system embeds model-backed capability, design must address the model as a live dependency: which provider, which pinned snapshot, what happens on deprecation, what happens on silent behavioral change, whether a fallback path exists, and whether the system degrades gracefully when the model is wrong rather than merely when it is unavailable.
12.2 Elements to Adopt #
Agent-legible architecture. Write the constraints down. Module boundaries, allowed dependency directions, layering rules, naming conventions, error handling patterns, logging expectations, and the reasoning behind each. Place them where the toolchain will surface them to whatever is generating code—repository-level guidance files have become the de facto mechanism, and their significant property is that they are source-controlled, reviewable artifacts rather than platform configuration.52 Treat them as design documents subject to design review, because that is what they are.
Enforced architectural constraints. Documentation states intent; automated architectural checks (dependency rule enforcement, layer violation detection, cyclic dependency detection, module boundary tests) establish it. Under generation at volume, the check is what holds the line. Include duplication thresholds, because duplication is now the measured failure mode.
Blast radius as a design parameter. For each component, state explicitly what an autonomous change to it could affect: which data, which systems, which customers, which credentials. Design so that the components where an agent will operate most are the components with the smallest reachable radius. This is ordinary good design; what is new is the reason to insist on it.
Environment separation that is architectural, not procedural. The single clearest lesson from documented agent incidents is that separation enforced by process fails and separation enforced by topology holds. Development environments must not hold production credentials. This is not a new principle; it is a principle whose violation is now exploited automatically rather than occasionally.
Model dependency design. For every model-backed capability, specify the pinned snapshot rather than a floating alias, the fallback behavior on unavailability, the fallback behavior on degraded quality, the evaluation gate that governs a version change, and the owner. Treat the model as a dependency with a version, an owner, and an expiry date—but understand that pinning bounds the risk rather than eliminating it, since providers can change API surface and parameter behavior independently of model retirement.48
Design for verification. Where correctness is statistical, the system must be architected so that correctness can be measured in production: capture points for inputs and outputs subject to data classification policy, correlation identifiers, the ability to replay against a different model version, and instrumented decision points. Retrofitting observability into a probabilistic system is materially harder than into a deterministic one, because the thing you need to observe is a distribution.
Tool-protocol surfaces as designed interfaces. Where agents will reach internal systems through a tool protocol, those servers are privileged integrations and belong in the architecture, with an authorization model, an owner, and a review cadence. The current specification is explicit that servers “MUST only accept tokens that are valid for use with their own resources” and “MUST NOT accept or transit any other tokens,” and that possession of a state handle must not be treated as authentication.53 Those requirements describe a confused-deputy failure mode and a session-fixation failure mode respectively, and both are design problems rather than implementation details. The protocol’s governance moved to a Linux Foundation directed fund in December 2025, which is a relevant durability signal for architecture decisions that depend on it.54
Threat modeling that includes the toolchain. Extend threat modeling to cover the development environment itself. The relevant taxonomies are published and usable: the OWASP Top 10 for Agentic Applications, released December 2025, enumerates agent goal hijack, tool misuse and exploitation, identity and privilege abuse, agentic supply chain vulnerabilities, unexpected code execution, memory and context poisoning, insecure inter-agent communication, cascading failures, human-agent trust exploitation, and rogue agents.55 The 2026 revision of the OWASP LLM Top 10 is the first edition to weight real incident data alongside expert voting, drawing on 7,714 collected incidents of which 6,639 were classifiable, weighted at 25 percent against 75 percent community voting, and moved excessive agency from sixth to third place on that basis.56 MITRE ATLAS provides the adversary technique vocabulary and now includes a case study specifically covering supply chain attacks against AI coding assistants via configuration files.57
12.3 Controls #
| ID | Control | Minimum bar (L2) | Enforced state (L3) |
|---|---|---|---|
| DSN-1 | Architectural constraints documented in a machine-surfaced, source-controlled artifact | Present and reviewed | Constraint file changes require architecture review; drift from enforced checks alerts |
| DSN-2 | Architectural rules enforced by automated check | Dependency and layering checks in CI | Checks blocking at merge, including duplication and reuse thresholds |
| DSN-3 | Blast radius documented per component | Documented for components in agent scope | Blast radius bounded by enforced credential and network scope; verified in the running environment |
| DSN-4 | Environment separation enforced topologically | No production credentials in development environments | Separation verified by automated test; violation blocks deployment |
| DSN-5 | Model dependencies pinned to dated snapshots with declared fallbacks | Snapshot pinned, owner named | Version change gated on evaluation; alias use blocked at build |
| DSN-6 | Design for observability of probabilistic behavior | Capture and correlation points specified | Replay capability tested; evaluation instrumentation present at design review |
| DSN-7 | Tool-protocol integrations reviewed as privileged interfaces | Inventoried with named owners and authorization model | Version-pinned, signed where supported, capability drift detected automatically |
| DSN-8 | Threat model covers the development toolchain and agent surfaces | Toolchain threat model documented | Reviewed on tooling change; mapped to a published agentic risk taxonomy |
12.4 Evidence to Request #
- The architectural constraints artifact, with its review history—a file that has never been reviewed is configuration, not architecture.
- The output of architectural enforcement checks over the last quarter, including violations and how they were resolved.
- A component’s documented blast radius, verified against the credentials actually present in its runtime.
- The model dependency register with pinned snapshots, owners, and provider-published retirement floors.
- The toolchain threat model, with the date it was last revised against the current tool inventory.
12.5 Failure Modes #
- Architecture that lives in people’s heads. Previously a hiring and onboarding cost. Now a per-commit cost, paid automatically, forever.
- Constraints documented but unenforced. A guidance file with a rule that no check verifies is a suggestion. Under generation at volume, suggestions lose.
- Reuse assumed rather than designed. Nothing in a generative workflow rewards finding the existing implementation. Absent an enforced duplication threshold and a discoverable component catalog, duplication is the default outcome, and the data says it is the observed one.38
- Tool-protocol servers as convenience integrations. Stood up by an engineer to make an agent useful, holding broad credentials, never reviewed, invisible to the architecture. This is the most common way an agent acquires reach that nobody designed.
- Designing for model availability rather than model correctness. Fallback paths handle the case where the provider is down. They rarely handle the case where the model returns a confident, well-formed, wrong answer—which is the far more common failure.
References cited in this section
8 of 243 · numbering matches the PDF
- 38GitClear, The Maintainability Gap: AI Code Quality in 2026, January 2026. 623 million analyzed changes, 2023–2026. Correlational; commits are not labeled by AI authorship; vendor-published.www.gitclear.com/the_ai_code_quality_maintainability_gap ↗
- 52GitHub, "Introducing Agent HQ: Any Agent, Any Way You Work," The GitHub Blog, October 28, 2025. Announced and preview capabilities; feature availability is not evidence of adopted practice.
- 48Anthropic, "Model Deprecations," Claude platform documentation.platform.claude.com/docs/en/about-claude/model-deprecations ↗
- 53Model Context Protocol, "Authorization" and "Security Best Practices," specification version 2026-07-28.modelcontextprotocol.io ↗
- 54Anthropic, "Donating the Model Context Protocol and Establishing the Agentic AI Foundation," December 9, 2025; Linux Foundation, "Linux Foundation Announces the Formation of the Agentic AI Foundation," December 2025.
- 55OWASP GenAI Security Project, OWASP Top 10 for Agentic Applications for 2026, December 9, 2025.genai.owasp.org/resource/owasp-top-10-for-agentic-applications-for-2026 ↗
- 56OWASP GenAI Security Project, OWASP GenAI LLM Top 10 2026, v1.0, August 3, 2026. First edition to weight documented incident data alongside expert consensus.genai.owasp.org/resource/owasp-genai-llm-top-10-2026 ↗
- 57MITRE, "MITRE ATLAS,". Case study AML.CS0041, "Rules File Backdoor: Supply Chain Attack on AI Coding Assistants." See also Center for Threat-Informed Defense, "Secure AI v2 Release," April 2026.atlas.mitre.org ↗