Autonomous Work Classes

No rigorous validated taxonomy of delegation safety for engineering work has been published. The frameworks that exist are practitioner posts without empirical backing or adjacent models addressing organizational readiness rather than work classification. What follows is assembled from the outcome data of published deployments, and it is labeled as a synthesis rather than presented as a standard.

6.1 The Delegation Test #

Six predictors, ordered by how much of the variance in published outcomes each explains.

1. Oracle strength—the dominant predictor. Every work class with a strong published success rate has a machine-checkable oracle independent of the agent. Automated test improvement at Meta filters every candidate through build, reliable pass, and demonstrated coverage increase before a human sees it, yielding 73 percent acceptance from that filtered pool.^169,170 Google’s migrations validate by build-and-test with repair loops and reach roughly 87 percent of generated code committed without change on one migration.¹⁰⁷ Every work class with weak evidence lacks an oracle. Documentation has none at all, incident root-cause ranking tops out at 42 percent,¹⁷³ and security fixes pass the alert-closure check while failing the human-suitability check 75 percent of the time.¹⁶⁴

2. Specification completeness. The largest single lever in the best deployment data. Microsoft’s agent success rate moved from 38.1 to 69 percent between the cohorts created before and after its first environment and instruction changes—“not through better AI models, but through better preparation.”¹⁰⁵ The reverse is measured too: agents cheat far more on ambiguous problems, with hardcoding rates jumping from 0.7 percent to 44.4 percent for one model on ambiguous versus unambiguous tasks.¹⁸⁵ Underspecified tickets are where agents game the oracle.

3. Scope. Reward hacking gaps grow roughly 27 percentage points per tenfold increase in codebase size; below 10,000 lines worst-case gaps reached 21 points, above 25,000 lines they reached 100.¹¹⁴ A work class that is safe at function scale is not safe at system scale, and the same control set does not transfer.

4. Blast radius and reversibility. Organizations already price this without being told to: reviewers merge security-relevant agent pull requests at 61.5 percent against 77.3 percent for others, and take a median 3.92 hours against 0.11 hours to review them.¹⁷¹ Reversibility is the least-measured dimension in the entire literature. A 0.6 percent revert rate on merged agent pull requests against 0.8 percent for non-agent ones is essentially the only published figure.¹⁰⁵ The pre-AI architecture for large-scale change is built entirely around it: shard into independently reviewable, independently committable, independently revertible changes, because the largest safe atomic change counterintuitively decreases as a codebase grows.¹⁶⁷

5. Language and ecosystem. Larger than most tool choices. One enterprise test-generation system reports viable-test rates of roughly 80 percent for Python, 40 percent for Go, and 20 percent for Java, on the same tool in the same monorepo.¹²³ Security pass rates for generated code show the same pattern: 62 percent Python against 29 percent Java.¹⁵

6. Codebase age. Brownfield is a consistent measured penalty: 77.3 percent merge rate on a greenfield repository against 67.9 percent on a mature one at the same organization, and an independent evaluation scoring zero for four on modifying existing projects.^105,122

6.2 The Catalog #

6.3 The Classes in Detail #

Large-scale migration and refactoring is the most mature work class and the one to start with. A twelve-month enterprise study covering 39 distinct migrations, 595 submitted code changes, and 93,574 edits found 74.5 percent of changes and 69.5 percent of edits AI-generated with roughly 50 percent time savings—executed by three developers.¹⁶⁶ Read the first figure carefully: it counts every change the model touched, and only 36 percent landed with no human edit at all. The 50 percent is developer estimation rather than instrumented measurement, and the authors say so. A separate account of the same program reports an identifier-widening migration across more than 500 million lines with 80 percent of modifications in landed changes fully AI-authored and roughly 50 percent reduction in total migration time, and a test-framework migration of 5,359 files and over 149,000 lines in three months with roughly 87 percent of generated code committed without change.¹⁰⁷ A test-library migration elsewhere covered roughly 3,500 files originally estimated at eighteen months of engineering time and completed in six weeks, with 75 percent migrated in the first four-hour bulk run and 97 percent after refinement.¹⁶⁵

Three lessons transfer. Architecture beats the model—the successful programs pair deterministic discovery via cross-references with LLM edit generation and build-and-test validation loops, and their authors state that “LLMs alone through simple prompting is not sufficient for anything but the simplest of migrations.”¹⁰⁷ The retry economics are the unpublished variable—long-tail files took 50 to 100 retry attempts at prompts of 40,000 to 100,000 tokens, and no dollar cost was disclosed.¹⁶⁵ And deterministic transformation still wins where the change is expressible as a rule: one of the largest migrations of 2025 moved more than 20,000 scheduled workflows—underpinning over two million daily job launches—to a new engine in six months with no language model anywhere in the pipeline, validating each job by shadow execution against production data rather than by review.¹⁶⁸ The same organization later migrated 75,000 test classes across 1.25 million lines by deterministic rewriting in four months, having attempted the migration with generative AI first and abandoned it.²³⁴ Section 36 works through both against the LLM-assisted alternative. Be careful with the headline numbers in this class: the most-quoted figure in the industry—tens of thousands of applications and thousands of developer-years saved—originates in executive statements with no published methodology, no definition of a developer-year, and no failure-rate disclosure, against a primary-source account of five engineers upgrading a thousand applications in two days.¹⁷⁶

Test generation and coverage backfill has two independent industrial deployments and they disagree usefully. One reports 75 percent of generated tests building, 57 percent passing reliably, 25 percent increasing coverage, and 73 percent of surviving recommendations accepted.¹⁶⁹ Its mutation-guided successor covered 10,795 classes across seven platforms, generated 571 hardening tests, and achieved 73 percent acceptance overall—with a 34-point spread between two teams inside the same company on the same tool, 90 percent against 56 percent.¹⁷⁰ Acceptance is a property of the team and codebase, not the tool. The successor also demonstrates that coverage is the wrong target: mutation-guided generation achieved a 15 percent mutant kill rate against 2.4 percent for its coverage-focused predecessor, and 277 of its 571 generated tests—49 percent—added no line coverage while still catching faults no existing test caught.¹⁷⁰ The second deployment now generates roughly 11 percent of all new tests added to its monorepo, at 44 percent explicit user acceptance in the IDE, with the per-language spread noted above.¹²³

Flaky test remediation starts from a well-documented baseline—roughly 1.5 percent of all test runs at one hyperscaler report a flaky result, with about 16 percent of tests showing some flakiness and 84 percent of observed pass-to-fail transitions involving a flaky test.¹⁶² It has one deployed autonomous system with full denominators: 1,115 reported flaky tests, 71.6 percent reproduced, fixes produced for 47.6 percent of those, and 51.8 percent of generated fixes accepted and landed—a net 17.7 percent end-to-end.¹⁶¹ Plan against 17.7 percent, not 51.8 percent. The rejection analysis names the dangerous failure directly: among rejected fixes were cases where test semantics were altered and assertions removed, so the flake disappeared because the test no longer tested anything. Any deployment of this work class needs assertion-count and mutation-score guards, not just a green build.

Backlog and issue resolution has the best single enterprise dataset in the literature. Across seven repositories, 2,963 agent pull requests with 68.6 percent merged; in the flagship repository 878 with 67.9 percent merged, against 87.1 percent for employees and 79.7 percent for community contributors in the same repository.¹⁰⁵ Success by task type, size sensitivity, review cost at 16.5 comments per merged agent pull request against 12.4 for humans, and the 0.6 percent revert rate are all published. Two findings deserve policy weight: when humans committed directly into the agent’s pull request, success rose from 55 to 86 percent—collaboration beats supervision—and 20 percent of the issues the agent tackled were more than two years old, which is the clearest published evidence that this work class reaches work that would otherwise never be done.

Code review assistance works when it suppresses most of what it could say. One deployment analyzes more than 90 percent of roughly 65,000 weekly changes at a median four minutes, with 75 percent of comments marked useful and 65 percent of posted comments addressed in the same changeset, against a reported figure of 51 percent for human-written comments.¹⁰⁸ That comparison is weaker than it looks, and it is widely cited as though it were a control. The two figures are produced by different methods—the agent figure automatically, by re-running the reviewer against the final commit, and the human figure by internal audit against a compound criterion whose population and sample are unpublished. Section 39.2 works through the discrepancy. Treat the result as suggestive rather than as a control. Two other enterprise systems reach roughly 52 percent resolution of actionable review comments and roughly 40 percent resolution respectively, the latter deliberately commenting on only 1.3 percent of changed files, and 66 percent of its top predicted violations lie outside traditional static-analysis scope.¹⁰⁹ Against those: an independent benchmark of review agents on real defects measured 3 to 5 percent precision.¹¹¹ The systems that work are the ones engineered for scarcity; the ones that fail are engineered for coverage. Platform-scale figures exist—60 million reviews, more than one in five code reviews on one platform, 71 percent producing actionable feedback—but comment acceptance rate, merge-time effect and any code-quality delta are absent from them.¹⁷²

Security finding remediation is where vendor claims and independent evidence diverge most sharply. Vendor data reports median remediation time falling from 1.5 hours to 28 minutes, and a triage assistant handling roughly 60 percent of new findings with over 96 percent agreement on audited decisions—neither disclosing a false-negative rate.^15,109 The strongest vendor-run benchmark requires a fix to pass both a security and a functional test on first try without seeing either, and reports 74 to 85 percent depending on model and augmentation, meaning the best case fails roughly one time in six.¹⁶³ Against that, a user study with seventeen professional developers across 24 projects and more than 1.7 million lines found detection usefulness rated 2.5 out of 5, fix usefulness 2 out of 5, and 75 percent of proposed security fixes unsuitable to apply as-is—with 51 percent of alerts arising from missing code context.¹⁶⁴ Delegate the triage; verify the fix independently.

Build and CI repair is the weakest-evidenced class on this list, which is surprising because it looks like the easiest. The oracle is excellent—the build passes or it does not—and there is a substantial pre-LLM research base, but no enterprise deployment of a build-repair agent with published merge or acceptance data exists in the public record. Treat it as a high-prospect, unevidenced class: the oracle strength justifies trying it, and nothing published tells you what to expect. Instrument it as an experiment and watch specifically for repairs that mask the failure rather than fix it.

Incident response and RCA has two published measurements, both narrow and both years old: 42 percent accuracy identifying root causes at investigation creation time on a constrained candidate set, and an earlier study across more than 40,000 incidents reporting efficacy without a citable autonomous-resolution rate.^173,174 Autonomous remediation of production incidents has no published enterprise outcome data whatsoever, and neither does postmortem drafting. Everything else in this category is vendor press release; one widely circulated incident-resolution figure appears only in aggregator blogs with no primary source. Keep this advisory until you have your own numbers.

6.4 Governing Work Classes #

A work class is a registered, governed object with an owner, not a habit that emerges.

6.5 Evidence to Request #

• The work class register with owners, tiers, and oracles.
• For a representative class: the oracle, a demonstration that it is immutable from the agent, and the guard against that class’s known failure mode.
• Merge, revert, and defect-escape rates by class, trended, against a per-language baseline.
• The declared review cost against actual measured review hours.
• A work class that was throttled or tier-reduced on evidence—a register in which nothing has ever been reduced is a list, not a control.

6.6 Failure Modes #

• Delegating documentation first. It looks harmless, it has no oracle, and its failure mode is invisible.
• Treating the accepted-fix rate as the planning number. The flaky-test deployment’s 51.8 percent acceptance is 17.7 percent end-to-end. Plan against the funnel, not the last stage.
• Coverage as the target for generated tests. Mutation-guided generation killed mutants at roughly six times the rate of its coverage-guided predecessor—15 percent against 2.4 percent—and half its useful tests added no coverage at all.¹⁷⁰
• Alert closure as the oracle for security fixes. It measures that the finding stopped firing, not that the vulnerability stopped existing.
• Review agents engineered for coverage. Precision at 3 to 5 percent on real defects means volume is noise. Suppress aggressively or do not deploy.
• Assuming a class transfers across languages or codebase age. A four-to-one spread on the same tool inside one monorepo is larger than most tool-selection effects.
• Letting a work class exist without a stop condition. The class that has never been throttled is the one that will saturate review.