The Agentic Engineering Loop

Most organizations govern the pull request and leave the loop entirely unexamined. That is backward. By the time a change reaches review, the decisive error has usually already happened, the budget has usually already been spent, and the evidence of how the work was actually done has usually already been compacted away.

5.1 The Anatomy of a Turn #

A turn is one full cycle: the model receives a prompt with its system instructions, tool definitions and history; it responds with text, tool calls, or both; tools execute and return results; the cycle repeats until the model produces a response with no tool calls.¹³⁰

Three properties of this cycle are load-bearing, and each is routinely overlooked.

Termination is typed, and the types are a telemetry schema. Published termination taxonomies distinguish success, maximum turns exceeded, budget exceeded, execution error, and structured-output retry exhaustion.¹³⁰ An organization that records only “the agent finished” is discarding the single most useful diagnostic signal it has. Instrument why the loop stopped.

Iteration limits are two-dimensional and frequently default to unlimited. Maximum turns and maximum spend are separate controls, and the vendor documentation itself notes that without limits “the loop runs until Claude finishes on its own, which is fine for well-scoped tasks but can run long on open-ended prompts.”¹³⁰ Every A2-and-above deployment should carry both.

The third bears on correctness rather than cost: compaction silently rewrites the agent’s memory of its instructions. As the context window fills, older messages are replaced with a summary, and the documentation warns explicitly that “specific instructions from early in the conversation may not be preserved.”¹³⁰ Anything that must hold for the whole task belongs in a re-injected file, not in the opening prompt. This is a common and expensive misunderstanding: organizations put their governance rules in a system prompt and assume they persist.

One further point belongs here, though it is a procurement decision rather than a design one. Harness design is a controllable variable independent of model choice. The founding result in this area demonstrated that the same model scores materially differently under different agent-computer interfaces, and that constrained, purpose-built tools with structured feedback outperform raw shell access.¹²⁷ The counter-demonstration is equally important: a roughly hundred-line harness with no tools but bash, a completely linear history, no context management, no sub-agents, and no memory reports scores within a few points of elaborate scaffolds.¹²⁸ Anyone selling harness complexity should be asked to beat that baseline. The published platform abstraction worth borrowing from a third architecture is the action/observation event stream as the canonical agent state—the closest thing the open literature has to a provenance-friendly, auditable model of agent work.¹²⁹

5.2 Context Assembly #

This is the most contested area in the entire agentic engineering literature, and the honest synthesis is not what most vendors say.

Repository instruction files improve efficiency and probably not success. A controlled study across SWE-bench tasks and developer-committed issues, using both generated and human-written context files across multiple models and agents, found that “providing context files does not generally improve task success rates, while increasing inference cost by over 20% on average”—with the null result holding across models, agents, and file provenance.¹³¹ A separate paired within-task study of 124 pull requests found median wall-clock time down 28.6 percent and median output tokens down 16.6 percent with an instruction file present, but explicitly did not evaluate task success.¹³² A field study of 15,549 agentic pull requests across 148 projects found near-symmetric outcomes: 27.7 percent of projects raised merge rate by at least 20 percent after introducing an instruction file, and 26.4 percent lowered it by at least 20 percent.¹³³

The refinement that survives all three: instruction files earn their cost for non-standard practices the agent could not infer, and file quality decides the outcome. Projects that improved had files with a median of 976 words against 569 for projects that declined.¹³³ Repository overviews, the thing vendors most commonly recommend generating, are the part with the least support. The format itself is now stewarded by the Agentic AI Foundation under the Linux Foundation, which is the durable signal for an organization standardizing on it.¹³⁵

And these files are the least secure artifact in the estate. A study of 2,303 context files across 1,925 repositories found testing content in 75 percent, implementation detail in 69.9 percent, and architecture in 67.7 percent—with security and performance each appearing in only 14.5 percent.¹³⁴ The artifact steering every agent almost never mentions the two quality attributes that degrade fastest. Published guidance exists on what security content belongs there.⁹⁹

Indexing earns its keep as repository size grows. Two major vendors hold publicly opposite architectural positions, just-in-time agentic exploration against a maintained semantic index, and each publishes evidence favoring its own.^136,137 The only defensible synthesis from published data is scale-dependence: the vendor advocating indexing reports code retention improving 0.3 percent overall but 2.6 percent on codebases over a thousand files.¹³⁷ Independent benchmark work is more sobering: across 427 samples from 25 repositories, interactive agents never accessed the gold files on 27 to 35 percent of samples despite exploration, and the median relevant evidence occupies only 4.7 percent of the file containing it.²⁰³ Context acquisition is a distinct, measurable failure surface independent of patch generation.

Long contexts degrade non-uniformly. Performance falls as input length grows, with distractors biting harder at length and a substantial gap between focused and full-context input across model families.¹³⁸ The publisher sells vector databases and has an interest in the conclusion, which should be stated—but the finding is the empirical basis for compaction, sub-agent isolation, and aggressive curation.

5.3 Self-Verification Within the Loop #

One distinction governs everything here: verification against an external oracle works; verification against the model’s own judgment does not.

The negative baseline is peer-reviewed and unambiguous. Models “struggle to self-correct their responses without external feedback, and at times, their performance even degrades after self-correction,” and prior positive results depended on oracle labels telling the model when to stop.¹¹⁰ This is the citation to use against any vendor claim that an agent reviews its own work.

The positive case is equally clear and points at execution. A fixed three-phase pipeline of localize, repair, and validate with regression tests plus generated reproduction tests, with no autonomous tool selection at all, resolved 32 percent of a benchmark at roughly $0.70 per issue, beating the agentic systems of its moment on both accuracy and cost.¹⁴⁰ The verification step, not the autonomy, carried the value.

Selection is a separate and under-invested engineering problem. Work combining serial iteration with parallel candidate generation, where each trajectory generates a test script alongside its draft edit, reached 57.4 percent at roughly $4.60 per instance—and selecting across edits drawn from top existing submissions reached 66.2 percent, outperforming the best individual member of that ensemble.¹⁴¹ The candidate pool frequently contains a correct patch that the selector fails to pick.

Critic sub-agents are measured, and the numbers are sobering. Across 584 instances of real pull request defects with full repository context, a single-shot reviewer achieved 27.0 percent recall at 3.6 percent precision, and adding iterative self-critique raised recall to 32.8 percent while collapsing signal-to-noise from 5.11 to 1.95 on the large model and from 2.89 to 0.91 on the small one.¹¹¹ Below parity means more false findings than true ones. Self-critique trades signal for recall, and smaller models degrade under it.

5.4 Planning, Decomposition, and Spec-Driven Work #

Test-driven agentic development is measured and it works. A peer-reviewed framework supplying tests and then remediation loops moved one model from 69.7 percent to 82.5 percent to 87.7 percent on a Python benchmark, and from 78.7 to 87.8 to 93.3 percent on another.¹⁵⁷ The gains shrink sharply as scope grows from function to file, reaching only 23.0 to 26.1 to 30.3 percent at file level, and the study names three limits worth carrying: more tests can make things worse through lost-in-the-middle effects, solutions sometimes satisfy only the supplied tests and fail private suites, and gains plateau after roughly three tests.

Spec-driven agentic development is a different story and it should be told honestly. Tooling exists and is well specified, defining a workflow from governing principles through specification, plan, tasks, implementation, and convergence assessment.¹⁵⁶ It presents no empirical evidence, no benchmark, and no user study showing that spec-driven development improves quality, speed, or any measurable outcome, and its own goals are labeled experimental. No independent evaluation exists. The nearest supporting evidence is indirect and points the other way: agents given specifications averaging 2,391 words resolve only 18.75 to 25 percent of realistic evolution tasks.¹¹⁵

That is not an argument against specification, which is the highest-leverage input in this framework for reasons established in Section 6.1 and Section 11. It is an argument against claiming measured efficacy for a methodology that has none yet, and for treating your own adoption of it as an experiment with instrumentation attached.

What is actually documented about long-horizon structure is that state lives in files rather than in the context window. The clearest published account externalizes specification, plan, implementation notes, and a running decision log into repository markdown, with verification commands and repair after each milestone, in a loop of plan, edit, run tools, observe, repair, update documentation, repeat.¹⁵¹ That account describes a single roughly twenty-five-hour run consuming around 13 million tokens. It is an anecdote rather than a measurement, but it is the clearest one available, and the architectural pattern is sound for a reason Section 5.6 explains.

5.5 Iteration Economics #

That last figure is the one that breaks naive capacity planning: token consumption is stochastic, so per-task budgeting must be distributional rather than a point estimate.

Two further findings from the same work destroy the intuitive planning approach. Accuracy often peaks at intermediate cost and saturates above it—spending more does not monotonically buy correctness. And neither the model nor a human expert can forecast what a task will cost: models predict their own token consumption with correlations up to 0.39 and systematically underestimate, while expert difficulty ratings correlate only weakly with actual spend.¹⁵⁴

Cost per task, triangulated across independent sources, spans roughly $0.13 to $4.60 depending on approach, with a fixed-pipeline method at $0.70 and heavy parallel sampling at $4.60.^140,141,160 The instructive comparison is between two entries on the same public leaderboard: one model-scaffold pair cost $366.81 for a 27.2 percent resolve rate while another cost $67.09 for 38.0 percent.¹⁶⁰ Cost and accuracy are not correlated across scaffold-model pairs, and paying more is not a strategy. The methodological critique underlying this is worth reading in full: accuracy-only evaluation, conflation of model-developer and downstream-developer needs, inadequate holdout sets producing shortcut-taking, and a lack of standardization—with the finding that state-of-the-art agents are “needlessly complex and costly.”¹⁵⁵

Context accumulation makes the marginal turn the expensive one. Vendor modeling of a fifty-turn session puts input at roughly 5,000 tokens per turn for turns one through ten, 20,000 for turns eleven through thirty, and 35,000 for turns thirty-one through fifty, with input outnumbering output twenty to twenty-five times.¹⁵⁹ Cost per turn is therefore roughly linear in turn index. That is an economic argument for compaction and sub-agent isolation independent of any quality argument—and it means an agent that is going to fail is cheapest to stop early.

Runaway loop detection is an engineering necessity with no evidence base. Mechanisms exist (turn caps, budget caps, concurrency and spawn-depth limits, execution time ceilings) but there is no published measurement of how often production coding agents enter non-terminating loops, no published detection heuristic with a measured false-positive rate, and no published cost-anomaly threshold. Build it, instrument it, and do not claim a benchmark you do not have.

5.6 Long-Horizon Work #

Capability on long tasks is improving fast and is routinely overstated. The anchor measurement fits success probability against human expert completion time and reports the fifty-percent time horizon roughly doubling every seven months.¹⁴⁹ The caveats matter more than the trend line and are routinely dropped: it measures task difficulty expressed in human time rather than how long an agent works autonomously; the task distribution is primarily software engineering, machine learning and cybersecurity; horizons reflect what a low-context person could accomplish; measurements above sixteen hours are unreliable with the current task suite; and one published per-model figure carries a 95 percent confidence interval spanning from under two hours to over twenty. Quote the interval or do not quote the number.

Why long horizons fail has a specific and actionable mechanism. Peer-reviewed work identifies self-conditioning: “models become more likely to make mistakes when the context contains their errors from prior turns,” and this is not a long-context artifact—injecting artificial error histories reproduces it, and larger models are more susceptible despite better long-context handling.¹⁵² The same work finds that small single-step accuracy gains compound into exponential gains in executable horizon, and that reasoning-trained models eliminate self-conditioning entirely.

That mechanism converges with two others. Multi-turn conversational degradation of roughly 39 percent, decomposed into a minor aptitude loss and a large increase in unreliability, is driven by models making early assumptions and over-relying on them.¹⁵³ And the dominant failure category in real trajectories is “false premise” at 30.7 percent—the agent forms a wrong belief early and never revisits it.¹¹⁷

• Restart beats steer. Incremental mid-task correction degrades outcomes because the failed attempt contaminates context. Checkpoint-and-restart from a clean state is mechanistically justified; mid-task nudging is not. Note honestly that the multi-turn study tested conversational generation rather than agentic coding trajectories, so the transfer is plausible and unproven.
• Early abort beats late detection. Eighty-two percent of failures show no termination after the decisive error, and observable signals surface roughly ten steps after it.¹¹⁷ A long-running trajectory is a negative signal, not a sign of diligence.
• Error recovery, not error avoidance, distinguishes success. Seventy-one percent of successful trajectories recover from at least one error.¹¹⁷ Design for recovery.

Asynchronous agents make these constraints concrete. Published platform limits include a fifty-nine-minute maximum execution time, one branch and one pull request per session, and no cross-repository changes—with the documented workaround for incompatible branch protection being to add the agent as a bypass actor, which is in direct tension with the merge-boundary control in Section 13.¹⁵⁰

5.7 Multi-Agent Orchestration #

The pattern vocabulary is settled: prompt chaining, routing, parallelization by sectioning or voting, orchestrator-workers, and evaluator-optimizer, with workflows distinguished from agents by whether control flow is predefined.¹⁴² The guidance from the same source runs against complexity: start simple and add structure “when it demonstrably improves outcomes.”

The evidence for multi-agent in software engineering is weak, and the strongest pro-multi-agent result excludes coding explicitly. The most-cited positive finding reports a 90.2 percent improvement over a single agent on an internal research evaluation, and in the same publication notes that multi-agent systems use about fifteen times more tokens than chat, that token usage alone explained 80 percent of performance variance on the relevant benchmark, and that “most coding tasks involve fewer truly parallelizable tasks than research.”¹⁴³ The gain is substantially a token-spend gain, and the vendor with the strongest published result carves coding out of it.

The counterevidence is stronger than the case.

• A peer-reviewed study of over 1,600 annotated execution traces across seven multi-agent frameworks produced a fourteen-mode failure taxonomy in three categories (system design, inter-agent misalignment, and task verification) with inter-annotator agreement at κ = 0.88, and found that obvious interventions such as better prompts and added structure produce partial gains only.¹⁴⁶ Multi-agent failure is architectural rather than promptable.
• Under matched thinking-token budgets, single-agent systems were the best performer or statistically indistinguishable from it at every budget except the lowest, across five multi-agent variants and three models; sequential multi-agent became competitive only under 70 percent artificial context corruption, which is the regime good context engineering exists to prevent.¹⁴⁷ That study covers multi-hop question answering rather than software engineering, which should be stated.
• A single agent running multi-turn conversation with cache reuse matched homogeneous multi-agent workflows at lower cost across eight benchmarks, because it pays for incremental tokens rather than repeated full prefixes.¹⁴⁸
• The strongest practitioner argument is that subagents cannot see each other’s implicit decisions, and that conflicting decisions produce bad results—with a preference for single-threaded architecture and a dedicated compression model over parallel decomposition.¹⁴⁴ That position has since been publicly softened by its author, which should be noted rather than ignored.
• The useful discriminator from a third vendor: read actions parallelize and write actions do not.¹⁴⁵ Multi-agent fits breadth-first read-heavy exploration and misfits shared context and coordinated writes.

And there is a measured cost at repository scale that no orchestration framework addresses: cross-product concurrency roughly doubles the merge conflict rate.³²

Sub-agent context isolation is a partial exception worth understanding on its mechanics. Documented behavior: a non-fork sub-agent starts with a fresh context window, intermediate tool calls stay inside it, only the final message returns to the parent, and a tool omitted from its definition is not in its session at all.¹³⁹ Governance controls are real, covering spawn depth, concurrency, budget caps that terminate background sub-agents, and instruction-shaped-pattern scanning of sub-agent output before the parent reads it, which is a prompt-injection control at the agent-to-agent boundary. No published measurement exists of whether sub-agent isolation improves task success on software engineering work. The mechanism is documented; the benefit is asserted.

5.8 How Loops Fail #

The best published failure taxonomy for coding agents covers 1,794 complete trajectories and more than 63,000 execution steps across seven models and three scaffolds.¹¹⁷

Read that table with Section 5.6’s process findings and the design implications are unambiguous. The largest failure mode is a wrong belief formed early. The second largest is ignoring the specification. Neither is fixed by a better model, and both are addressed by better inputs and earlier abort.

A separate finding kills a common intuition: fault localization is not the bottleneck. Agents identify the correct file 72 to 81 percent of the time even in failed attempts, and success depends more on achieving approximate rather than exact modifications.¹⁵⁸ Investment in localization tooling is misdirected; investment in verification and specification is not.

5.9 Controls #

5.10 Evidence to Request #

• The distribution of termination reasons for the last thousand sessions—a population that is 100 percent “success” means the taxonomy is not instrumented.
• Cost per task as a distribution with its ninety-fifth percentile, not a mean.
• A demonstration that a governance constraint survives compaction.
• The result of an adversarial retrieval test against the context substrate.
• The trajectory length distribution for successful and failed sessions, which should differ.
• Evidence that an early abort actually fired and what it saved.

5.11 Failure Modes #

• Governing the pull request and ignoring the loop. Everything expensive and everything diagnostic happens before the pull request exists.
• Rules in the system prompt. They do not survive compaction, and the failure is silent.
• Unlimited loops by default. Both turn and budget ceilings default to unlimited in at least one major SDK, and most organizations never change it.
• Self-critique treated as verification. It measurably degrades signal, and it is the most commonly marketed agent feature.
• Steering a failing agent. Mid-task correction is intuitive, feels responsible, and makes outcomes worse. Restart instead.
• Multi-agent as a default architecture. Fifteen times the tokens, an architectural failure taxonomy, no software-engineering evidence, and double the merge conflict rate.