Do Androids Dream of Breaking the Game? Systematically Auditing AI Agent Benchmarks with BenchJack

Agent benchmarks have become the de facto measure of frontier AI competence, guiding model selection, investment, and deployment. However, reward hacking, where agents maximize a score without performing the intended task, emerges spontaneously in frontier models without overfitting. We argue tha...

SkillSafetyBench: Evaluating Agent Safety under Skill-Facing Attack Surfaces

Reusable skills are becoming a common interface for extending large language model agents, packaging procedural guidance with access to files, tools, memory, and execution environments. However, this modularity introduces attack surfaces that are largely missed by existing safety evaluations: eve...

LITMUS: Benchmarking Behavioral Jailbreaks of LLM Agents in Real OS Environments

The rapid proliferation of LLM-based autonomous agents in real operating system environments introduces a new category of safety risk beyond content safety: behavior jailbreak, where an adversary induces an agent to execute dangerous OS-level operations with irreversible consequences. Existing...

MonitoringBench: Semi-Automated Red-Teaming for Agent Monitoring

We introduce a red-teaming methodology that exposes harder-to-catch attacks for coding-agent monitors, suggesting that current practices may under-elicit attacks and overstate monitor performance. We identify three challenges with current red-teaming. First, mode collapse in attack generation,...

Benchmarking Large Language Models for IoC Recovery under Adversarial Code Obfuscation and Encryption

Software obfuscation and encryption present persistent challenges for program comprehension and security analysis, particularly when adversaries conceal Indicators of Compromise IoCs such as IP addresses within source code. While Large Language Models LLMs have recently demonstrated remarkable...

LCC-LLM: Leveraging Code-Centric Large Language Models for Malware Attribution

LLMs are increasingly explored for malware analysis; however, current LLM-based malware attribution remains limited by unsupported indicators and insufficient code-level grounding for identifying malicious and vulnerable code segments. To address these limitations, this research introduces LCC-LL...

Pen-Strategist: A Reasoning Framework for Penetration Testing Strategy Formation and Analysis

Cyber threats are rapidly increasing, expanding their impact from large-scale enterprises to government services and individual users, making robust security systems increasingly essential. However, a significant shortage of skilled cybersecurity professionals exacerbates this challenge. While...

ARGUS: Defending LLM Agents against Context-Aware Prompt Injection

The rise of Large Language Model LLM agents, augmented with tool use, skills, and external knowledge, has introduced new security risks. Among them, prompt injection attacks, where adversaries embed malicious instructions into the agent workflow, have emerged as the primary threat. However,...

VulKey: Automated Vulnerability Repair Guided by Domain-Specific Repair Patterns

The increasing prevalence of software vulnerabilities highlights the need for effective Automatic Vulnerability Repair AVR tools. While LLM-based approaches are promising, they struggle to incorporate structured security knowledge from sources like CWE and NVD. Current methods either use this...

CVE-2026-7510

A vulnerability was determined in OWAP DefectDojo up to 2.55.4. Affected by this vulnerability is an unknown functionality of the component Benchmark/Engagement/Product/Survey. Executing a manipulation can lead to authorization bypass. The attack can be executed remotely. The exploit has been...

CVE-2026-7510 OWAP DefectDojo Benchmark/Engagement/Product/Survey authorization

A vulnerability was determined in OWAP DefectDojo up to 2.55.4. Affected by this vulnerability is an unknown functionality of the component Benchmark/Engagement/Product/Survey. Executing a manipulation can lead to authorization bypass. The attack can be executed remotely. The exploit has been...

CVE-2026-7510

The CVE-2026-7510 entry concerns OWAP DefectDojo up to 2.55.4, with an authorization bypass affecting the Benchmark/Engagement/Product/Survey functionality. The issue is reachable remotely and is supported by a public disclosure; upgrading to DefectDojo 2.56.0 addresses the vulnerability (patch e...

CVE-2026-7510 OWAP DefectDojo Benchmark/Engagement/Product/Survey authorization

A vulnerability was determined in OWAP DefectDojo up to 2.55.4. Affected by this vulnerability is an unknown functionality of the component Benchmark/Engagement/Product/Survey. Executing a manipulation can lead to authorization bypass. The attack can be executed remotely. The exploit has been...

EUVD-2026-26457

A vulnerability was determined in OWAP DefectDojo up to 2.55.4. Affected by this vulnerability is an unknown functionality of the component Benchmark/Engagement/Product/Survey. Executing a manipulation can lead to authorization bypass. The attack can be executed remotely. The exploit has been...

CVE-2026-7510 OWAP DefectDojo Benchmark/Engagement/Product/Survey authorization

A vulnerability was determined in OWAP DefectDojo up to 2.55.4. Affected by this vulnerability is an unknown functionality of the component Benchmark/Engagement/Product/Survey. Executing a manipulation can lead to authorization bypass. The attack can be executed remotely. The exploit has been...

DefectDojo 授权问题漏洞

DefectDojo is an application security and vulnerability management tool developed by DefectDojo. Versions of DefectDojo 2.55.4 and earlier contained a vulnerability related to authorization. This vulnerability stemmed from unknown functions within the Benchmark/Engagement/Product/Survey component...

PT-2026-36213

Name of the Vulnerable Software and Affected Versions OWAP DefectDojo versions prior to 2.56.0 Description An issue exists in the Benchmark, Engagement, Product, and Survey components where a manipulation can lead to a remote authorization bypass, allowing an attacker to circumvent access control...

BinExploit-Bench

BinExploit-Bench: Binary Exploitation Capability Benchmark for...

Evaluating Jailbreaking Vulnerabilities in LLMs Deployed As Assistants for Smart Grid Operations: A Benchmark against NERC Standards

The deployment of Large Language Models LLMs as assistants in electric grid operations promises to streamline compliance and decision-making but exposes new vulnerabilities to prompt-based adversarial attacks. This paper evaluates the risk of jailbreaking LLMs, i.e., circumventing safety alignmen...

CrossCommitVuln-Bench: A Dataset of Multi-Commit Python Vulnerabilities Invisible to Per-Commit Static Analysis

We present CrossCommitVuln-Bench, a curated benchmark of 15 real-world Python vulnerabilities CVEs in which the exploitable condition was introduced across multiple commits - each individually benign to per-commit static analysis - but collectively critical. We manually annotate each CVE with its...