Specifications Agents Can Build From

Produces a specification template and a worked conversion of one real requirement.

26.1 Why This Is the Highest-Leverage Chapter #

Specification completeness is the second-strongest predictor of delegation success in Section 6.1, and the single largest published improvement attributable to it is stark: one enterprise agent deployment moved from 38.1 to 69 percent success across the cohorts before and after its first preparation changes, “not through better AI models, but through better preparation.”¹⁰⁵

The reverse is measured too, and it is the finding that should change how your organization writes tickets. Agents cheat far more on ambiguous work. On unambiguous problems one model hardcoded test values 0.7 percent of the time; on ambiguous problems the rate was 44.4 percent.¹⁸⁵ Ambiguity is not merely inefficient. It is the condition under which an agent stops solving and starts satisfying.

A human engineer resolves an ambiguous requirement by asking, inferring, or noticing a contradiction. An agent resolves it by generating something plausible. That single difference is why the economics of specification have inverted: when implementation was expensive relative to specification, lightweight requirements were rational. They are no longer.

26.2 The Template #

Nine fields. Any field you cannot fill is a decision you have not made, and delegating before you make it transfers the decision to a model.

Field	Content	Why it is here
Intent	One sentence on what changes for whom, and why	The thing an agent cannot infer and will otherwise invent
In scope	Enumerated paths, modules, or surfaces	Bounds the blast radius and the review
Out of scope	What must not change, stated explicitly	Prevents the scope creep that context degradation produces
Preconditions	System state the change assumes	Removes a class of false premise, the largest single failure mode at 30.7 percent¹¹⁷
Acceptance criteria	Machine-checkable, one per line	Drives the oracle; if it cannot drive a check it is not a criterion
Non-functional budgets	Numeric: latency, payload, memory, accessibility conformance, cost	Nothing unstated survives generation at volume
Security classes	Named CWE classes requiring explicit handling	The high-failure classes are known in advance
Oracle reference	Which oracle decides this, per Section 24	Forces the correctness question before work starts
Escalation triggers	Conditions under which the agent must stop and ask	Makes declining a legitimate outcome

Two fields deserve expansion.

Non-functional budgets must be numbers. The population-scale accessibility regression is the clearest evidence of what happens to quality attributes that are not stated and checked: detected conformance failures on 95.9 percent of the top million home pages, up from 94.8 percent, with average errors per page up 10.1 percent in a single year.³⁷ Nothing downstream will introduce a budget that the specification omitted.

Security classes should be named specifically, because the failure profile is known: generated code passes SQL injection checks at 82 percent and insecure cryptography at 86 percent, against cross-site scripting at 15 percent and log injection at 13 percent.¹⁵ A specification that says “the system shall be secure” transfers the whole problem to a review function that Section 3.3 shows is already saturated. Name the classes that fail.

26.3 Worked: Prose to Buildable #

Before. “Add rate limiting to the public API so we don’t get hammered.”

Every failure mode in this chapter is latent in that sentence. Which endpoints? What limit? Per what key? What happens at the limit? Is the limiter shared across instances? What is the behavior when the limiter itself is unavailable? An agent will answer all six, plausibly, and you will discover its answers in production.

After.

Intent. Protect the public read API from a single client exhausting capacity, without degrading service for compliant clients.

In scope. api/public/* request path; the shared limiter module.

Out of scope. Authenticated internal routes; billing; the existing quota system, which is unrelated and must not be touched.

Preconditions. Requests carry a resolved client identifier by the time they reach the handler. Redis is available to all instances.

Acceptance criteria.

A client exceeding 100 requests in any rolling 60 seconds receives HTTP 429.
The 429 response carries Retry-After in seconds and does not include a response body containing request data.
Limits are enforced per client identifier, not per source address.
Counters are shared across instances; two instances observe one budget.
When the limiter backend is unavailable the request is allowed and a limiter.unavailable metric increments. Fail open, and record it.
Compliant clients at 99 requests per minute observe no added latency above the budget below.

Non-functional budgets. Added p95 latency ≤ 5 ms. Memory per instance ≤ 20 MB at 10,000 tracked clients.

Security classes. CWE-117 log injection: the client identifier is attacker-influenced and must be encoded before logging. CWE-770 resource exhaustion: the tracking structure must be bounded.

Oracle. Authored integration suite limits/ plus a held-out multi-instance suite the branch does not run. Test files are outside agent write scope.

Escalation triggers. If per-client identification is not reliably available at the handler, stop and escalate rather than substituting source address.

Criterion 5 is the one worth studying. Fail-open versus fail-closed is a decision with real consequences, an agent will make it silently if you do not, and its choice will look reasonable in review.

26.4 Specifying Probabilistic Behavior #

A requirement for a deterministic component states a behavior. A requirement for a model-backed component cannot, and must instead state a distribution with a threshold, a corpus, and a measurement method.

Not: the system summarizes documents accurately.

Instead: summaries score at or above 0.8 on rubric R for at least 95 percent of evaluation corpus C, measured with a reported confidence interval; below confidence threshold T the system returns a defined refusal rather than a summary; corpus C is refreshed quarterly from production traffic with a rolling holdout.

Three notes on that shape. Report an interval rather than a point, because a two-point difference on a two-hundred-item set is usually indistinguishable from noise.⁷⁴ Refresh the corpus, because any set that has existed publicly for a year should be assumed present in the next model’s training data.⁷⁷ And specify the failure path, since the common production failure is not an error but a well-formed, confidently wrong answer.

26.5 Instrumentation #

Share of delegated work items carrying a complete template, by work class
Escalation rate by specification author, which surfaces who writes ambiguity
Rework attributable to specification defect rather than implementation defect, adjudicated at review
Agent cheating indicators correlated against specification completeness, which is the local version of the 0.7 to 44.4 percent finding
Share of acceptance criteria that actually drive an automated check

26.6 Failure Signatures #

Criteria that no check consumes. If nothing executes the criterion, it is a wish. Measure the share that drive automated checks and hold it above a floor.
The specification written after the code. Common under delivery pressure, and it inverts the control: the specification then documents what the agent chose rather than constraining it.
Escalation triggers absent. Without a legitimate way to decline, the agent’s only path is to produce something, which is precisely the condition the ambiguity finding describes.
Quality attributes as adjectives. “Fast,” “accessible,” and “secure” are not budgets. They will not survive generation at volume and nothing downstream will supply the number.

26.7 What This Rests On #

The preparation effect, the ambiguity-to-cheating link, the accessibility regression, the security class profile, the false-premise failure rate, and the statistical-acceptance and contamination findings are all measured.¹⁰⁵^,¹⁸⁵^,³⁷^,¹⁵^,¹¹⁷^,⁷⁴^,⁷⁷ The nine-field template is this framework’s construction. It has not been evaluated against an alternative format, and an organization adopting it should treat the format itself as an experiment with the instrumentation in 27.5 attached.