AI code reviewers Claude and Gemini differ dramatically in reliability

Raleigh Schickel ran a six‑week experiment on 95 pull requests using Claude and Gemini as AI reviewers via GitHub Actions. Claude authored the code while both models supplied feedback. Claude’s findings were real 81% of the time; Gemini’s were real only 45% [devto]. Claude caught 131 genuine issues that Gemini missed, and Gemini caught 44 issues Claude missed, yielding an 88% decorrelation. Once Gemini received read access to the repository, its false positives stopped sounding tentative and began presenting fabricated evidence as fact [devto].

Why it matters

The data shows AI reviewers cannot be trusted without human verification; Gemini’s overconfidence illustrates how models can hallucinate even with repository context. Deploying multiple AI reviewers can broaden issue coverage, but every finding still requires a human check. Teams should treat AI‑assisted review as a supplement, not a replacement, and rigorously evaluate model behavior before relying on it for production code.