Open Problems

A framework is more credible for naming what it cannot yet answer. These are genuine gaps in standards, practice, or evidence—not areas awaiting a vendor. Several will close on events that are already scheduled or pending; Appendix C tracks those separately, so that a reader can tell how stale this document has become without reading it twice.

• No standard exists for attesting AI authorship. Neither major SBOM format has a native construct for “this artifact was generated by an agent.” What exists is an unsigned commit-trailer convention, platform-specific attribution metadata that has demonstrated it can be wrong, and source provenance that records how a revision was created without a field for what generated its content. The custom predicate proposed in X2 is a workaround, not a solution.
• No accepted criterion governs granting an agent write access. Benchmarks measure capability on curated tasks; they measure nothing about behavior under organizational constraint, and the gap between near-saturated benchmark scores and observed merge and security rates makes them actively misleading as a qualification signal. The staged qualification protocol in Section 14 is defensible and it is invented.
• No standard governs rollback on behavioral change. Progressive delivery mechanics are mature; the trigger for a model-backed feature that degrades without erroring is not defined anywhere, and there is no independent research on it.
• No error budget model works when correctness is unobservable at request time. The observable-proxy approach in Section 16 is a workaround. The conceptual problem, that a budget cannot be debited for failures that are not detected, is unsolved.
• No published contractual practice covers behavioral stability within a pinned snapshot. Providers publish deprecation notice periods. None commits to behavioral stability for a snapshot that remains nominally available, which is the risk that actually breaks running systems.
• No independent data exists on static analysis efficacy against AI-generated code specifically. Every efficacy figure available is vendor-published, and none discloses false-negative rates, which is the number that matters for a triage automation.
• No corpus of AI incident postmortems exists. Two or three public incidents are cited repeatedly across the entire industry because they are nearly the only ones published. Conventional outage postmortems became a shared engineering resource through a culture of publication that has not yet formed here.
• No research addresses performance or load testing under AI-assisted development. The accessibility evidence is real and independent; there is no equivalent for performance, and inferring one from reuse decline would be reasoning past the data.
• Comprehension debt is a concept, not a measurement. It names something real and widely observed. There is no instrument for it, no baseline, and no longitudinal study—only a small randomized trial on skill formation and population-level maintainability trends that are correlational.
• No published data establishes an engineer-to-agent supervision ratio. Not from a vendor, not from an enterprise, not from academia. The term originates in a vendor survey that explicitly declined to supply a figure. Absorption capacity is measurable and should be measured in its place.
• Sub-agent context isolation has a documented mechanism and an asserted benefit. No published measurement exists of whether it improves task success on software engineering work. The same is true of checkpoint-and-restart efficacy, git-worktree parallelism, and plan-and-act mode separation.
• Spec-driven agentic development has tooling and no efficacy data. No benchmark, no user study, no independent evaluation. Test-driven agentic development, by contrast, is peer-reviewed and works.
• Build and CI repair has an excellent oracle and no published enterprise deployment data. It is the highest-prospect unevidenced work class on the list.
• Documentation maintenance by agents has no published enterprise outcome data at all, and no oracle. It is simultaneously the most-delegated and the least-measured work class.
• No published deployment gates agent-written tests on mutation score. It follows directly from two documented enterprise mutation-testing programs and nobody has published doing it.
• Shadow-mode deployment of agent output is unmeasured. The pattern is well documented pre-AI; no enterprise has published applying it to agent changes.
• No enterprise has published its agent-fleet observability architecture, its sandbox compute spend, or measured savings from production model routing. The one large published fleet-level token saving came from response shaping at a tool gateway, not from routing.
• No published study measures human review effectiveness on agent-authored code against a ground-truth defect set, nor rubber-stamping rates in agent pull request review. Both are load-bearing for any auto-merge policy.
• No published study measures skill formation in juniors trained primarily under agentic workflows. Strong indirect evidence, zero direct longitudinal evidence.
• No validated maturity model for agentic software engineering exists. A rigorous diagnostic, a credible general AI maturity model, and a proposed governance escalation exist; a validated staged model of agentic engineering adoption does not. This framework’s Section 4 is a synthesis, and is offered as one.
• No model exists for when a temporal control substitutes for a logical one. One organization automerges the large majority of millions of maintenance changes with no per-change human verification, relying on cohort gating and post-deployment failure correlation rather than on deciding correctness before the merge (Section 38). It publishes no failure or rollback rate, so the safety of the design is asserted rather than demonstrated—and yet it has operated for years. The conditions under which detect-and-stop is an adequate substitute for verify-then-merge, and the blast-radius threshold at which it stops being one, are unmodeled here and unstudied anywhere.
• Standards bodies have not yet arrived. NIST has an initiative, a request for information, a concept paper, an annotated outline, and a preliminary draft—no normative text on agents. The two planned control overlays for single-agent and multi-agent systems do not exist in draft form. This framework is written for that interval and should be revised when they publish.