Review at Volume

39.1 Context and Mechanism #

Uber’s review system analyzes over 90 percent of roughly 65,000 weekly changes across six monorepos, processing more than 10,000 commits a week excluding configuration files, at a median latency of four minutes within continuous integration.¹⁰⁸

The architecture is four stages. Ingestion filters low-signal targets—configuration, generated code, experimental directories—and constructs prompts with surrounding code context. Generation runs three specialized assistants: one for bugs, exception handling, and logic flaws; one enforcing organization-specific conventions from a shared registry; one for application security. Post-processing applies three filters: a second model call assigns each comment a confidence score, a semantic-similarity filter merges overlapping suggestions, and a category classifier suppresses comment types with historically low developer value. Feedback collection closes the loop.

The generation model and the grading model are different providers. That is the closest thing here to independence between production and judgment, and it is a deliberate design choice rather than an accident of procurement.

The suppression philosophy is stated directly: “A high false-positive rate undermines engineers’ perception of the tool’s accuracy and usefulness—when they encounter many false-positive comments, they start to tune out and ignore them.” Categories consistently rated poorly are suppressed outright: readability nits, minor logging tweaks, low-impact performance suggestions, and stylistic issues.¹⁰⁸

39.2 The Comparison, and Why It Is Not a Control #

Engineers mark 75 percent of posted comments useful. On automated evaluation, an average of 65 percent of posted comments are addressed in the same changeset. The post then states that internal audits show only 51 percent of human-written comments are “considered as bugs by the author and addressed in the same changeset,” and concludes that the system’s “performance significantly exceeds that of human reviewers.”¹⁰⁸

The two numbers are produced by different methods against different criteria. The 65 percent is measured automatically, by re-running the review system five times against the final commit and counting a comment as resolved when none of the re-runs reproduces a semantically similar comment. The 51 percent comes from human audit and carries a compound condition—the author had to consider the comment a bug and address it in the same changeset. The population, sample size, time window, and team scope of that audit are not published.

This matters beyond pedantry. The framework has repeatedly treated this figure as the most credible published claim that agent review adds signal, on the strength of its apparent same-population human baseline. That reading was too generous. What exists is two figures measured differently, and a first-party interpretation of the gap between them. The comparison is suggestive. It is not a control.

The system’s own limits are stated candidly, which counts in its favor:

“uReview today only has access to the code, and not to other artifacts like past PRs, feature flag configurations, database schemas, technical documentation… it can’t correctly assess overall correctness and review the system design. It’s much better at catching bugs that are evident from analyzing the source code alone.”

— Uber Engineering

39.3 What Is Missing #

No recall figure, no false-negative rate, and no statement of what proportion of real defects the system misses. Uber reports building precision-recall dashboards internally and publishes neither number. No ratio of candidate comments generated to comments posted, which means the suppression rate—the thing the whole design turns on—is unquantified in public. The 1,500 hours saved weekly, and the 39 developer-years annually derived from it, come from an internal benchmark asserting that a second human reviewer would spend ten minutes per commit; it is a modeled figure, not a measurement.

39.4 What Transfers #

Engineering for scarcity rather than coverage is the transferable design principle, and it is the same principle Section 6 draws from the wider review literature. A system that comments on everything it could comment on trains its readers to ignore it.

Splitting generation and grading across different model providers is a cheap approximation of independence, and it is available to any organization with two vendor relationships.

The measurement lesson is the one to internalize most. If an organization intends to claim its review agent beats human reviewers, the two figures must be produced the same way, on the same population, over the same window. Otherwise the honest claim is narrower: comments the system posts are addressed at a high rate, and the system covers nearly everything, which is worth having on its own terms.

39.5 What This Rests On #

One first-party post, unaudited, with no independent replication. Every figure in Section 39.1 is directly reported. The critique in Section 39.2 rests on the post’s own description of its two measurement methods, which differ on its own account. The absence of recall data is stated by omission and should be treated as unknown rather than as poor.