Fleet Operations

31.1 Cost Is a Reliability Signal #

Runaway consumption is the earliest observable indicator of an agent in a loop, an agent being manipulated, or an agent operating outside its intended scope. Treating cost as a finance concern rather than an operational one loses that signal.

The unit is cost per task, not cost per token. Agentic workloads consume roughly a thousand times more tokens than conversational use, so falling unit prices coexist with rising bills.¹⁵⁴ Per-token price declines have been rapid, at roughly fortyfold per year for a fixed capability threshold, but with a spread of ninefold to nine-hundredfold depending on task, and that analysis dates from early 2025.⁴⁵

Attribute below the account. Team, repository, work class, agent deployment, and task. Observed total cost per engineer runs $200 to $600 per month including tokens against seat list prices of roughly $19 to $60, a four- to ten-fold multiple on the procurement line item, with governance infrastructure a further $50,000 to $250,000 annually.⁴⁰

Ceilings that halt. One organization caps every employee at $1,500 in monthly token spend per AI coding tool; at two tools that is roughly $36,000 per engineer per year.¹⁹⁶ Another alerts on individual daily spend above a threshold.²⁰⁴ Both are simple, and both work because they are enforced rather than reported.

Route everything through a gateway. One published implementation runs roughly 100 million model requests per day through a governance layer holding a sub-100-millisecond latency budget.¹⁰³ Centralization is what makes attribution, quota, and anomaly detection possible at all.

31.2 Where the Savings Actually Are #

Model routing research is strong in benchmark settings, reporting up to 84 percent cost savings with competitive accuracy and 97 percent of a frontier model’s quality at 24 percent of cost under time constraints.¹⁹⁹ No enterprise has published measured savings from production model routing or caching.

The one large published fleet-level saving came from somewhere else entirely: response shaping at the tool gateway, reported as a greater than 40 percent reduction in token usage across the agent fleet.¹⁰³ Compact tool responses, trimmed schemas, and on-the-fly result summarization at the gateway are a different and better-evidenced lever than model selection, and almost nobody discusses them.

Start with response shaping. Treat routing as an experiment.

31.3 Kill Switches That Reach #

Three scopes, each tested, each with a measured time-to-effect: one agent, one agent class, the whole estate.

The gap to test for is reach. Kill switches exist in the orchestrator and typically do not reach agents embedded in SaaS platforms, agents in partner environments, or agents running on a developer’s machine against corporate credentials. Run the drill against one agent in a third-party platform specifically, because that is the case that fails.

Pair the switch with credential revocation, and measure that separately. Halting the process while its tokens remain valid is a partial control.

31.4 Telemetry You Actually Read #

Emit in a versioned schema and expect to re-instrument. All generative-AI semantic convention attributes, spans, metrics, and events currently carry development status with none stable, and the conventions moved to a dedicated repository in June 2026 with no tagged release.⁹⁰ Pin the version, treat attribute names as a contract with your own pipeline, and do it anyway.

Instrument the generated-to-analyzed ratio and report it. Recording is not detection, and the ratio degrades silently as volume rises. This single metric is the difference between an evidentiary asset and a control.

Two published instrumentation choices are worth borrowing. Provider failures as a first-class signal, which has no analogue in conventional monitoring.²⁰² And session transcripts flowing back into the service catalog after each session, which is architecturally distinct from span-based tracing and gives organization-wide visibility into what agents did and for whom.¹⁹⁵

No enterprise has published an agent-fleet observability architecture — no data on trace volume, retention, storage cost, sampling strategy, or how session traces join to code outcomes. You are designing this from first principles.

31.5 Concurrency #

Bound concurrent agent work streams per repository and partition scope by path. The measured cost of not doing so is a textual conflict rate of 19.8 percent within one agent product, rising to 41.7 percent when pull requests from different agent products overlap, across 33,596 agent-authored pull requests of which 79.4 percent temporally overlapped with others.³²

The multi-vendor implication is a procurement input no vendor comparison will surface: running two agent products against one repository roughly doubles the conflict rate. If the organization wants product diversity, partition by repository rather than mixing within one.

31.6 Instrumentation #

• Cost per task at p50 and p95, and cost per accepted change, by work class and team
• Spend against quota by team, with anomaly alerts and enforced halts
• Kill-switch time-to-effect by scope, from live test, including one third-party platform
• Generated-to-analyzed telemetry ratio
• Conflict rate, with cross-product concurrency broken out
• Share of tool access flowing through the governed gateway rather than point integrations

31.7 Failure Signatures #

• Cost visible only at the account level. Cannot detect a looping agent, support a chargeback conversation, or inform a tier decision.
• A kill switch that has never been fired. Untested is a documentation artifact.
• Telemetry volume as a success metric. Volume is cost. The analyzed fraction is the control.
• Routing adopted for savings on benchmark evidence. No production measurement exists. Response shaping does have one.
• Point integrations proliferating. Every tool server stood up by a team to make an agent useful is an ungoverned credentialed path into the estate.

31.8 What This Rests On #

The token-consumption profile, the price-decline range, the per-developer cost figures, the spend cap, the gateway throughput, the response-shaping saving, the routing benchmark results, the telemetry convention status, and the conflict rates are all published, though several are vendor-reported or third-party summaries of conference talks.^{154,45,40,196,204,103,199,90,202,195,32} The kill-switch drill design and the concurrency partitioning rule are this framework’s construction.