Krentsel, Agarwal, Cemri, Zaharia and Stoica just put a paper up on arXiv called "Reality Is the Final Verifier: On Two Key Gaps in Agentic Software Engineering" (2609.12039). It's the clearest statement I've seen of why the whole "let the AI review the AI's code" pattern is structurally weak, and it's worth reading if teams are shipping more generated code than they can manually review.
Their two-gap framework: the evaluator (a test suite, a reviewer, whatever) checks an implementation against a set of requirements under a model of the deployment environment. The requirement gap is the distance between what stakeholders actually want and what the requirements say. The model gap is the distance between the assumed environment and the real one. Hallucination widens both gaps by fabricating requirements or environment assumptions. Reward hacking exploits omissions in either one.
Here's the part that matters for review: those two gaps are computed from the same requirements and the same environment model that the implementation was built against. When the same model that wrote the code turns around and reviews it, it's re-checking the implementation against the exact same approximations it just used. Its blind spots about what the requirements approximate, and about what production actually looks like, propagate straight into the review. You're measuring one model's opinion of itself, and calling it verification.
That's the reason self-review gives you such a false sense of coverage. It catches the stuff the model already knew was dodgy, and passes everything that shares its own wrong assumptions.
Two moves actually narrow the gaps, and they're the reason I keep going on about evaluation design:
Cross-model review. A second, independent model reviewing the generated code re-derives requirements and environment assumptions from scratch. Different pretraining, different priors, different blind spots. It doesn't fix the requirement gap, but it stops the shared-blindspot failure where writer and reviewer agree on the same wrong model.
Run the thing. The paper's core point is that reality is the only final verifier and any predeployment evaluation is a proxy. A test that executes in an environment close to production beats any static "does this look reasonable" pass, because it forces the evaluator to commit to observable behavior rather than to a plausible story.
The resource-allocation framing in the paper is the honest version of all this: human judgment is the scarce input for the requirement gap, and faithful, costly evaluation is the bottleneck for the model gap. Given the volume of generated code, you can't spend dense human review on all of it. So the sensible design is: cheap AI-generated diffs route to a different model for a first independent pass, execution-based checks catch the behavioral lies, and human review concentrates on the diffs that survive both. That's where human attention returns the most per hour.
I'm not saying the same-model reviewer is useless. It's fine as a linter that catches your own model's obvious mistakes. Just don't confuse it with verification. If the writer and the reviewer share the same two gaps, you've added machinery without adding signal.
