secprbench

SecPRBench SecPRBench evaluates whether a code reviewer finds the security flaw in a real change. Each case is the upstream change that introduced a vulnerability in one of nine C, C++, and Rust projects. The reviewer receives the diff, the commit message, and a source snapshot without usable Git...

Can Agentic Code Review Catch Vulnerabilities Before They Are Merged?

LLM agents have been increasingly adopted for security-focused code review, but existing benchmarks mostly derive the reviewed changes from their later fixes instead of the real-world bug-introducing commits. In this paper, we present SecPRBench, the first comprehensive benchmark for evaluating...