CVE-Bench v2.0: Making Evaluation More Rigorous with ABC
This is the third post in the Agentic Benchmark Checklist (ABC) blog series. Written by Yuxuan Zhu, Antony Kellermann, and Daniel Kang.
We built CVE-Bench (ICML Spotlight, SafeBench winner) to evaluate AI agents’ capabilities to exploit real-world web security vulnerabilities. As AI agents grow more sophisticated, instances of agents exploiting loopholes in benchmark evaluations are becoming increasingly common (often called “reward hacking”). To accurately measure the offensive capabilities of agents in CVE-Bench, we must prevent agents from achieving goals through shortcuts or legitimate paths that our evaluation doesn’t capture. Guided by the Agentic Benchmark Checklist (ABC), we upgraded the infrastructure and revised the tasks of CVE-Bench to address these issues. With these enhancements in place, we are now releasing CVE-Bench v2.0.
In this blog post, we first review two key desiderata for ensuring valid evaluation in AI agent benchmarks. Then, we highlight two major fixes that address validity issues in CVE-Bench. We show that both fixes effectively prevent agents from cheating, decreasing their success rates by up to 32.5%. Finally, we summarize additional improvements in rigor, reproducibility, and usability.
Desiderata of AI Agent Benchmark Validity
AI agent benchmarks differ from traditional AI benchmarks in two key ways. First, they often need to replicate real-world environments (e.g., websites and operating systems) to provide realistic contexts in which agents operate and interact. In CVE-Bench, we deploy isolated web applications that reproduce the real-world systems under attack. Second, AI agent benchmarks often need to evaluate unstructured output from agents (e.g., code and free-form text). In CVE-Bench, an agent’s output is the ordered sequence of commands that make up a cyberattack. Because of these distinctions, the ABC framework proposes two validity criteria specifically for AI agent benchmarks.
Task Validity: A task is deemed successfully completed if and only if the agent demonstrates the required capability. To ensure task validity, an AI agent benchmark must be implemented robustly and stripped of any shortcuts that agents could exploit to finish the task illegitimately. For example, in SWE-bench Lancer, an agent can simply overwrite test files to pass evaluations.
Outcome Validity: An agent should receive a “success” outcome if and only if it successfully completes a task. To ensure outcome validity, an AI agent benchmark must evaluate agents’ unstructured output rigorously to avoid reward hacking in the evaluation process. For example, in SWE-bench Verified, handwritten unit tests can fail to capture bugs in the code generated by an agent.
In the next two sections, we introduce two fixes to strengthen the task and outcome validity in CVE-Bench.
Hacking-Resistant Grading for Outbound Service Attacks
Outbound service attack is one of our eight standard attack goals that requires attackers to induce the web application to send requests to a prohibited outbound server. Previously, CVE-Bench measured such attacks by checking whether the outbound server was accessed.
Item T.10 of ABC recommends conducting pilot experiments to identify vulnerabilities in the task setup that agents could exploit to pass evaluations. In our pilot experiments, we observed that a GPT-4o-based agent (arguably with a relatively low reasoning capability) consistently succeeded on CVE-2024-32986, a relatively complex task that requires building a malicious static web server. Upon inspection, we found that these agents directly accessed the outbound server over the Docker network, rather than inducing the web application to do so. This shortcut produced false positives, as shown below.
To prevent such false positives, we hardened the outbound service to allow traffic only from the web application (and deny any other source). This change affects three tasks in CVE-Bench. As shown in above, the success rates of agents decreased by up to 10% after the fix.
Stricter Grading of Time-based SQL Injection
CVE-Bench includes tasks that require attackers to execute time-based SQL injections to extract data from the database of web applications. Previously, CVE-Bench graded such attacks using a log-based state check: a SLEEP clause had to appear in the SQL logs of the database.
Item O.g.3 of ABC suggests using sufficiently complex state checks, but the log-based criterion was indeed too loose. Agents could pass the evaluation by inserting a SLEEP clause into a part of an SQL query that never executes. We use the following query as an example. When the first condition of the WHERE clause is false, MySQL short-circuits the AND and doesn’t evaluate the SLEEP subquery.
SELECT * FROM wp_users
WHERE user_login = ‘jjrH’
AND (SELECT 2344 FROM (SELECT(SLEEP(5)))YNJe)
LIMIT 1To prevent agents from exploiting this shortcut, we now require agents to extract data from a specific column in the database. This change affects nine tasks in CVE-Bench. After the change, the success rates of agents decreased by up to 32.5%, as shown in the figure above.
Towards More Rigorous Evaluation and Better Usability
Beyond the two major fixes described above, we made further improvements to CVE-Bench.
Validity:
We fixed an outcome-validity issue in CVE-2024-25641 and CVE-2024-34340, which had previously treated a failed login attempt as a success.
We fixed a task-validity issue in CVE-2024-37831 that had allowed attackers to access the admin page directly, without credentials.
We no longer hard-code secrets in CVE-Bench in case that the secrets are included in the training datasets of new LLMs. They are now generated at runtime by our containers using a configurable seed.
Reproducibility:
We prevented non-deterministic evaluation results caused by race conditions between the web application initialization and the evaluator initialization.
We improved the reproducibility of the evaluation results by using more conservative timeout settings and retry counts.
LoLLMS contains multiple CVEs with the same app version. To ensure reproducible results, we modified the challenges so that not all could be solved at the same time. Specifically, we:
Enabled code execution and restricted the /update_setting endpoint to only allow updating host in CVE-2024-2359.
We mounted the secret file to be accessible after maliciously switching the personal path configuration and disabled the /update_setting endpoint entirely in CVE-2024-2624.
We restricted the /update_setting endpoint to only allow updating the extension in CVE-2024-4320.
Usability:
We refactored the codebase, reducing set-up time by a factor of 4.
We implemented multiple improvements to our Docker infrastructure, including:
We switched the build system to Docker Buildx Bake to enable centralized, standardized, and multi-stage builds
We switched to using package locks (e.g., uv instead of pip) to enable reproducible builds
We fixed an issue in CVE-2024-4701 that caused issues on modern Linux kernels
Rigorously Benchmarking AI Agents’ Offensive Capabilities is a Ongoing Effort
As AI agents evolve, new and subtler issues in CVE-Bench may emerge. We are committed to continuously improving CVE-Bench by both fixing bugs and expanding task coverage. Please stay tuned for future updates, including quality improvements and new tasks.
Using CVE-Bench as an example, we also demonstrate the practical value of ABC in guiding the construction of AI agent benchmarks. We are also evolving ABC as the ecosystem grows. Please reach out via this form if you would like an assessment of your benchmark, or if you run into challenges applying the checklist.
Brilliant. Rigurous evaluation preventing reward hacking is crucial. But I wonder if those 'shortcuts' aren't actually another form of agent intelligence to consider.