How to reduce AI code review noise on pull requests
By Steven Van ·
Graphify's guide covers measuring ignored comments, graph context for reviewers, and three PR reply commands that mute findings your team rejects.
Graphify has published a guide to cutting AI code review noise on pull requests. It covers three steps: measure which comments get acted on, give the reviewer code graph context, and teach it what to skip through feedback on the pull request.
Measuring noise
The guide suggests tracking how often a comment leads to a code change. It cites ByteDance's BitsAI-CR paper, which uses an "Outdated Rate", the share of flagged lines that developers later changed. That two-stage system, a rule checker followed by a filter, reached 75.0% precision in review comment generation, with a 26.7% Outdated Rate for Go across more than 12,000 weekly active users. Without that pipeline, the same idea works on a sample of pull requests: count comments that were fixed, discussed or ignored. A high ignored share is the noise.
Cutting it down
- Style to linters: hand formatting and naming nits to tools such as ESLint or Ruff.
- Graph context: a reviewer that sees only the diff cannot tell whether a renamed parameter breaks a caller in another package. With a code graph it checks the callers instead of guessing. Hosted Graphify reviews pull requests this way.
- Filter low-confidence findings: let the reviewer propose candidates, then keep only findings backed by evidence in the code.
- Feedback: mute what the team keeps rejecting.
- Verification: separate opinion from proof.
Teaching the reviewer with PR replies
Since 20 August, Graphify learns from three commands written as replies on the pull request:
- @graphify wrong finding dismisses a finding that does not apply.
- @graphify good finding confirms one that was useful.
- @graphify suppress rule mutes a rule, optionally for one path.
Because the replies sit on the pull request, teammates see them too. The Learn page in the console lists muted findings, muted patterns and recent feedback, with who gave it and where. Learning only changes which advisory findings appear. It never changes a formal verification verdict.
Verification for behavior changes
For supported functions, formal verification either proves the new version matches the old within checked bounds or finds an input that differs. Graphify runs it on GitHub pull requests, so reviewers can skip "this might change behavior" comments.
Getting started
Connect a repository at app.graphify.com, open a pull request and reply @graphify wrong to the first finding you disagree with. The open-source CLI installs with uv tool install graphifyy and is licensed under Apache 2.0. Graphify's original guide also notes that muting a rule can hide real bugs, so it recommends scoping suppressions to a path where possible and checking the Learn page now and then.
