Same code reviewer, three models, three different reviewers

Quickchat ran its QuickReview AI code reviewer on three different models without changing anything else. It used the same passes, prompts, and posting rules on GPT-6.1 Sol, DeepSeek V4.1 Flash, and GLM-5.3 Flash. QuickReview splits each pull request into about 13 narrow passes, and a validator re-checks every candidate before one review is posted.

The test was 185 real PRs from the public AACR-Bench benchmark. A judge model scored each review against 520 bugs that AI agents had verified. By Quickchat's numbers, GPT-6.1 Sol found 89%, DeepSeek V4.1 Flash 87%, and GLM-5.3 Flash 84%. The models didn't find the same bugs, though. All three found the same 392 and missed the same 15, and they split on the other 113. On an Elasticsearch PR, GPT-6.1 Sol caught a bug in a test and missed one in the product code. The two open models did the reverse.

Style differed too. When all three found the same bug, the open models used about twice as many words to report it. On a Bitcoin PR, GPT-6.1 Sol's comment ran 79 words and DeepSeek V4.1 Flash's ran 263.

Quickchat lists some caveats. The open models ran a build from about ten days later and a later judge graded them. Each model ran once, and no human has checked the answer key or the judges. The review image also left the answer key readable, and one model used it. Quickchat gives scores with and without the affected PRs in the full write-up. The reviews, verdicts, and answer key are public as a Hugging Face dataset (DOI 10.57967/hf/11598).