Our pull requests go through two kinds of AI review. Cursor’s Bugbot looks for bugs, and Until compares the implementation against the plan we approved before the code existed. Some of those pull requests merge without an engineer reading every line.
So it would be ridiculous for me to argue that AI code review is useless. We rely on it. I think putting AI review only on the pull request starts too late, because nobody has yet asked what the code was supposed to do.
You can see people running into this now. On Hacker News, someone asked what tools people use for human review of AI-assisted code. CodeRabbit, Copilot, and Claude pointed at a PR were “pretty good at finding bugs and style nits,” they wrote, and much worse at duplicate modules, coupling, and separation of concerns. GitHub’s review interface had become unmanageable at the size agents produce, especially once bot comments and pasted agent output joined the thread.
I recognise the problem. Producing code got fast and cheap, while review is still the human-paced job it was when people typed every line. Another AI reviewer helps with parts of that job, but it cannot recover a decision that nobody made before the code existed.
The first serious conversation happens too late
Raed Abdennadher argues that code review assumes an author who understands the change well enough to answer questions about it. Agentic PRs make that assumption unreliable. The diff can look coherent, the tests can pass and a human name can appear at the top, while nobody can explain why the agent chose a particular boundary or what it expects to happen on failure.
Raed describes the pull request as the place where understanding is expected to appear for the first time. I agree, but I think making the person explain the finished PR is still too late. By then the agent has turned every ambiguity into code. The reviewer now has to separate the decisions the human intended from hundreds of choices the agent made because it needed an answer.
I experienced this from the slightly odd position of being a product manager opening code changes. Engineers left reasonable comments on my pull requests about libraries, internal patterns and failure handling. I often had no opinion on the choice because I had not made it. The agent had, and I was discovering its reasoning at the same time as the reviewer.
Smaller diffs make that reconstruction easier. AI reviewers can remove obvious bugs and noise. Better rollback systems reduce the cost when something still gets through. These are useful improvements, sometimes substantial ones, but they leave the first conversation about intent until after implementation.
A plan is useless if the agent can ignore it
We moved that first conversation onto the plan. Someone writes down the intended behaviour, the systems affected and the implementation choices the team actually cares about. Another person can disagree while the work is still cheap to change. Only then does an agent build.
Our first attempt relied mostly on instructions. I have written before about the many ways agents found around them, but the most embarrassing hole was self-approval. The same agent could write a plan and then record the verdict approving it. We had explicitly told it not to do that. Some models still found a plausible reason when self-approval was the shortest path to finishing.
A rule the agent is responsible for policing is still basically a suggestion. We moved the review state into the product, separated the author from the reviewer and blocked implementation until the saved review policy was satisfied. When a reviewer requests changes, the plan goes back through review instead of letting the agent decide that the objection was close enough.
That enforcement is the part I underestimated when we started building Until. A Markdown plan does not change the review process by itself. The workflow has to preserve the exact version that was agreed, who agreed it and whether the agent is allowed to begin. Otherwise “review the plan first” is just another sentence in a prompt the agent can reinterpret while trying to help.
So you don’t use AI for review?
We use quite a lot of it. AI helps interrogate the plan while it is being shaped, and after implementation Until checks the PR against the exact plan that was approved. Cursor’s Bugbot reviews the code for the bugs a plan review was never going to see.
In one week of twelve merged agent-built pull requests, Bugbot commented on six of them and left sixteen comments in total. Every comment was dealt with before merge. That is a tiny sample and says nothing about the bugs Bugbot may have missed, but it was doing useful work that I would not want to give back.
The human job becomes narrower. Someone has to own the intent and, where our review policy requires it, another person has to agree that intent before building starts. AI can find contradictions, inspect the codebase and ask awkward questions. It can later report that the implementation drifted. The person approving the plan remains accountable for the decision.
The pull request still matters
Implementation creates information the plan did not contain. Tests fail. Performance changes. A migration that looked safe on paper turns out to have an ugly edge case. High-risk work may still need a human to inspect the code, and a plan check can find a difference that should change the plan rather than the implementation.
When someone tells me they have AI code review now, I have ended up asking two fairly boring questions. Who agreed what the agent should build, and what happens when the code differs from that agreement? If the answer to the first is “the prompt” and the second is “another agent fixes it,” the PR reviewer is still being asked to reconstruct intent as well as inspect the implementation.
We have released the Until plugin for Cursor and Claude Code because we wanted that sequence to hold even when the agent was keen to skip ahead. It moves the disagreement before implementation, then leaves Until and Bugbot to do the narrower review jobs they are good at. That is how I think about code review now: agree the intent while changing it is still cheap, then use the pull request to check the implementation and deal with anything the plan missed.