The Context Substrate in Practice

Produces an entitlement-preserving retrieval layer, an indexed catalog, and an evaluated set of instruction files.

28.1 Build the Substrate, Not the Markdown #

The evidence here is genuinely contested and it points away from where most organizations put their effort.

Instruction files improve efficiency and probably not success. A controlled study across multiple models and agents, using both generated and developer-written context files, found that “providing context files does not generally improve task success rates, while increasing inference cost by over 20% on average.”¹³¹ A paired within-task study of 124 pull requests found median wall-clock time down 28.6 percent and median output tokens down 16.6 percent, while explicitly not evaluating task success.¹³² A field study of 15,549 agentic pull requests across 148 projects found near-symmetric outcomes: 27.7 percent of projects raised merge rate by at least 20 percent after introducing an instruction file, and 26.4 percent lowered it by at least 20 percent.¹³³

Retrieval is where the measurable failure is. Across 427 samples from 25 repositories, interactive agents never accessed the relevant files on 27 to 35 percent of samples despite exploration, and the median relevant evidence occupies only 4.7 percent of the file containing it.²⁰³ Context acquisition is a distinct failure surface from patch generation, and it is the one with room to move.

So: index the code, expose the catalog, preserve entitlements, and keep the markdown short.

28.2 Entitlement-Preserving Retrieval #

This is the single property to copy first, and one published enterprise implementation states it directly: internal tool-protocol servers front the wiki, product management system, data warehouse, CRM, and workspace, preserving existing user permissions so agents retrieve only what the invoking human can already see.²⁰⁴

The procedure is short and the verification is the part that matters.

Resolve the invoking human’s identity at the start of every session and carry it as the retrieval principal.

Enforce entitlement at the data layer, not by filtering results after retrieval. Filtering after means the content transited the model before being removed.

Scope the index itself by classification, so that regulated data requires a separate, separately authorized path.

Verify adversarially.

Instruct a test agent to retrieve something the invoking human cannot see and confirm refusal. A policy statement is not this step.

28.3 What to Index #

In descending order of measured or plausible value:

Code, semantically. The index earns its keep as repository size grows: the vendor advocating indexing reports code retention improving 0.3 percent overall but 2.6 percent on codebases over a thousand files.¹³⁷ Two major vendors hold publicly opposite positions here, each publishing evidence favoring its own architecture, so the honest reading is scale-dependence rather than a settled answer.¹³⁶^,¹³⁷

The service catalog and ownership topology. One published implementation injects component architecture, dependency graphs, ownership topology, and architectural decisions into agent sessions from its internal developer portal, with session transcripts flowing back into the same portal, making the catalog both the context source and the audit surface.¹⁹⁵

Architecture decisions. ADRs as retrievable context are the mechanism by which an agent can know why the codebase is shaped as it is. This is reasoning rather than measurement; no study has isolated the effect.

A component catalog answering “does something like this already exist.” This is the most useful counter to the duplication trend in Section 3.6, where refactoring fell from 21 percent of changed lines to 3.8 percent while copy-paste rose from 9.4 to 15.7 percent.³⁸ Nothing in a generative workflow rewards finding the existing implementation.

A knowledge graph, if you are large enough. One organization operates a context graph of roughly 40 million entries across about 150 node and edge types, explicitly built to reduce tokens, turns, and latency.¹⁰³ That is evidenced by essentially one company via a conference talk. Treat it as promising and single-sourced.

28.4 Instruction Files: Short, Specific, Evaluated #

Given the null results, the defensible posture is narrow.

Include non-obvious house rules an agent cannot infer, build and test invocation, conventions that differ from ecosystem defaults, and security content, which is almost always missing: a study of 2,303 context files found testing content in 75 percent, implementation detail in 69.9 percent, and architecture in 67.7 percent, with security and performance each appearing in only 14.5 percent.¹³⁴ Published guidance exists on what security content belongs there, including preferring well-vetted libraries, using official package managers, pinning versions, and verifying integrity by checksum or signature.⁹⁹

Exclude repository overviews, which are the part vendors most commonly recommend generating and the part with the least support.¹³¹

Evaluate rather than generate. Before rolling a file out, run the same corpus of tasks with and without it and compare success, cost, and review rework. Projects that improved had files with a median of 976 words against 569 for projects that declined, so length and structure matter, but your own measurement matters more than either number.¹³³

Review them as code, because they are an execution surface. Across the disclosed vulnerabilities in agentic development tooling, the vulnerable object is repeatedly the configuration file rather than the code.¹⁹^,²⁰^,²¹^,²²

28.5 Instrumentation #

Retrieval coverage: share of repositories indexed, and share of sessions where the agent reached the relevant file
Adversarial retrieval test result, refreshed on every substrate change
Instruction file effect: paired task success and cost, per file, before and after
Duplication and reuse trend, as the outcome measure for the component catalog
Median tokens per session, which is where substrate improvements show up first

28.6 Failure Signatures #

Retrieval scoped by prompt. “Only look at what the user can see” is an instruction. Entitlement belongs at the data layer.
Instruction files that grow. They accumulate because adding a line is easier than diagnosing a failure. Cap the length, review on change, and re-evaluate periodically.
Generated repository overviews. The specific artifact the evidence does not support, and the one most commonly auto-produced.
The catalog nobody queries. If agent sessions do not reach it, it is documentation with an API.

28.7 What This Rests On #

The context-file null results, the efficiency effect, the field outcome distribution, the retrieval failure rate, the content-composition study, and the indexing scale effect are all measured, though several are preprints and two are vendor-published with opposing conclusions.¹³¹^,¹³²^,¹³³^,²⁰³^,¹³⁴^,¹³⁶^,¹³⁷ The enterprise implementations are self-reported, and one is sourced through a conference talk rather than a first-party engineering publication.²⁰⁴^,¹⁹⁵^,¹⁰³ The indexing priority order is this framework’s judgment.