Do Agents Dream of Root Shells? Partial-Credit Evaluation of LLM Agents in Capture the Flag Challenges

Large Language Model LLM agents are increasingly proposed for autonomous cybersecurity tasks, but their capabilities in realistic offensive settings remain poorly understood. We present DeepRed, an open-source benchmark for evaluating LLM-based agents on realistic Capture The Flag CTF challenges ...

Towards Personalizing Secure Programming Education with LLM-Injected Vulnerabilities

According to constructivist theory, students learn software security more effectively when examples are grounded in their own code. Generic examples often fail to connect with students' prior work, limiting engagement and understanding. Advances in LLMs are now making it possible to automatically...

SIR-Bench: Evaluating Investigation Depth in Security Incident Response Agents

We present SIR-Bench, a benchmark of 794 test cases for evaluating autonomous security incident response agents that distinguishes genuine forensic investigation from alert parroting. Derived from 129 anonymized incident patterns with expert-validated ground truth, SIR-Bench measures not only...

T-MAP: Red-Teaming LLM Agents with Trajectory-Aware Evolutionary Search

While prior red-teaming efforts have focused on eliciting harmful text outputs from large language models LLMs, such approaches fail to capture agent-specific vulnerabilities that emerge through multi-step tool execution, particularly in rapidly growing ecosystems such as the Model Context Protoc...

Pensar Apex AI-Powered Penetration Testing

Pensar Apex is an AI-powered penetration testing using autonomous agents - directly in your terminal. Run blackbox and whitebox pentests that explore, reason, and surface real vulnerabilities...

Uncovering Security Threats and Architecting Defenses in Autonomous Agents: A Case Study of OpenClaw

The rapid evolution of Large Language Models LLMs into autonomous, tool-calling agents has fundamentally altered the cybersecurity landscape. Frameworks like OpenClaw grant AI systems operating-system-level permissions and the autonomy to execute complex workflows. This level of access creates...

Highly Autonomous Cyber-Capable Agents: Anticipating Capabilities, Tactics, and Strategic Implications

This report introduces the concept of "Highly Autonomous Cyber-Capable Agents" HACCAs, AI systems capable of autonomously conducting multi-stage cyber campaigns at a level comparable to today's top criminal hacking groups or state-affiliated threat actors, and analyzes the security implications o...

Agentic AI security: Why you need to know about autonomous agents now

Agentic AI is making headlines worldwide for its potential force-multiplying capabilities, and organizations are understandably intrigued by how it can improve throughput and capabilities. However, as with any technological revolution, unforeseen issues are inevitable, and agentic AI is no...

LLMs Generate Predictable Passwords

LLMs are bad at generating passwords: There are strong noticeable patterns among these 50 passwords that can be seen easily: All of the passwords start with a letter, usually uppercase G, almost always followed by the digit 7. Character choices are highly uneven ­ for example, L , 9, m, 2, $ and...

Understanding Human-AI Collaboration in Cybersecurity Competitions

Capture-the-Flag CTF competitions are increasingly becoming a testbed for evaluating AI capabilities at solving security tasks, due to the controlled environments and objective success criteria. Existing evaluations have focused on how successful AI is at solving CTF challenges in isolation from...

Agents of Chaos

We report an exploratory red-teaming study of autonomous language-model-powered agents deployed in a live laboratory environment with persistent memory, email accounts, Discord access, file systems, and shell execution. Over a two-week period, twenty AI researchers interacted with the agents unde...

QRS: A Rule-Synthesizing Neuro-Symbolic Triad for Autonomous Vulnerability Discovery

Static Application Security Testing SAST tools are integral to modern DevSecOps pipelines, yet tools like CodeQL, Semgrep, and SonarQube remain fundamentally constrained: they require expert-crafted queries, generate excessive false positives, and detect only predefined vulnerability patterns...

CyberExplorer: Benchmarking LLM Offensive Security Capabilities in a Real-World Attacking Simulation Environment

Real-world offensive security operations are inherently open-ended: attackers explore unknown attack surfaces, revise hypotheses under uncertainty, and operate without guaranteed success. Existing LLM-based offensive agent evaluations rely on closed-world settings with predefined goals and binary...

Introducing the Generative Application Firewall (GAF)

This paper introduces the Generative Application Firewall GAF, a new architectural layer for securing LLM applications. Existing defenses -- prompt filters, guardrails, and data-masking -- remain fragmented; GAF unifies them into a single enforcement point, much like a WAF coordinates defenses fo...

How AI made scams more convincing in 2025

This blog is part of a series where we highlight new or fast-evolving threats in consumer security. This one focuses on howAI is being used to design more realistic campaigns, accelerate social engineering, and how AI agents can be used to target individuals. Most cybercriminals stick with what...

Agentic AI for Cyber Resilience: A New Security Paradigm and Its System-Theoretic Foundations

Cybersecurity is being fundamentally reshaped by foundation-model-based artificial intelligence. Large language models now enable autonomous planning, tool orchestration, and strategic adaptation at scale, challenging security architectures built on static rules, perimeter defenses, and...

Analyzing Code Injection Attacks on LLM-Based Multi-Agent Systems in Software Development

Agentic AI and Multi-Agent Systems are poised to dominate industry and society imminently. Powered by goal-driven autonomy, they represent a powerful form of generative AI, marking a transition from reactive content generation into proactive multitasking capabilities. As an exemplar, we propose a...

2026 API and AI Security Predictions: What Experts Expect in the Year Ahead

This is a predictions blog. We know, we know; everyone does them, and they can get a bit same-y. Chances are, you’re already bored with reading them. So, we’ve decided to do things a little bit differently this year. Instead of bombarding you with just our own predictions, we’ve decided to cast t...

Prompt injection is a problem that may never be fixed, warns NCSC

Prompt injection is shaping up to be one of the most stubborn problems in AI security, and the UK’s National Cyber Security Centre NCSC has warned that it may never be “fixed” in the way SQL injection was. Two years ago, the NCSC said prompt injection might turn out to be the “SQL injection of th...

Charting the future of SOC: Human and AI collaboration for better security

Security operations centers are under pressure from unprecedented scale and complexity. Speed, precision, and consistency matter more than ever, and AI is everywhere—but hype alone doesn’t solve the challenge. This blog shares our journey and insights from building autonomous AI agents for MDR...