Jailbreak Mimicry: Automated Discovery of Narrative-Based Jailbreaks for Large Language Models

Large language models LLMs remain vulnerable to sophisticated prompt engineering attacks that exploit contextual framing to bypass safety mechanisms, posing significant risks in cybersecurity applications. We introduce Jailbreak Mimicry, a systematic methodology for training compact attacker mode...

Ask What Your Country Can Do for You: Towards a Public Red Teaming Model

AI systems have the potential to produce both benefits and harms, but without rigorous and ongoing adversarial evaluation, AI actors will struggle to assess the breadth and magnitude of the AI risk surface. Researchers from the field of systems design have developed several effective sociotechnic...

Genesis: Evolving Attack Strategies for LLM Web Agent Red-Teaming

As large language model LLM agents increasingly automate complex web tasks, they boost productivity while simultaneously introducing new security risks. However, relevant studies on web agent attacks remain limited. Existing red-teaming approaches mainly rely on manually crafted attack strategies...

CrossGuard: Safeguarding MLLMs against Joint-Modal Implicit Malicious Attacks

Multimodal Large Language Models MLLMs achieve strong reasoning and perception capabilities but are increasingly vulnerable to jailbreak attacks. While existing work focuses on explicit attacks, where malicious content resides in a single modality, recent studies reveal implicit attacks, in which...

BlueCodeAgent: A Blue Teaming Agent Enabled by Automated Red Teaming for CodeGen AI

As large language models LLMs are increasingly used for code generation, concerns over the security risks have grown substantially. Early research has primarily focused on red teaming, which aims to uncover and evaluate vulnerabilities and risks of CodeGen models. However, progress on the blue...

RedTWIZ: Diverse LLM Red Teaming Via Adaptive Attack Planning

This paper presents the vision, scientific contributions, and technical details of RedTWIZ: an adaptive and diverse multi-turn red teaming framework, to audit the robustness of Large Language Models LLMs in AI-assisted software development. Our work is driven by three major research streams: 1...

LegalSim: Multi-Agent Simulation of Legal Systems for Discovering Procedural Exploits

We present LegalSim, a modular multi-agent simulation of adversarial legal proceedings that explores how AI systems can exploit procedural weaknesses in codified rules. Plaintiff and defendant agents choose from a constrained action space for example, discovery requests, motions, meet-and-confer,...

RedCodeAgent: Automatic Red-Teaming Agent against Diverse Code Agents

Code agents have gained widespread adoption due to their strong code generation capabilities and integration with code interpreters, enabling dynamic execution, debugging, and interactive programming capabilities. While these advancements have streamlined complex workflows, they have also...

Automatic Red Teaming LLM-Based Agents with Model Context Protocol Tools

The remarkable capability of large language models LLMs has led to the wide application of LLM-based agents in various domains. To standardize interactions between LLM-based agents and their environments, model context protocol MCP tools have become the de facto standard and are now widely...

RedTeam-Tactics-and-Techniques

It is an offensive tool for Windows. The repository contains a script that appears to be a Windows shell extension, likely used for red teaming or penetration testing. The script is written in a format compatible with the Windows Taskbar and Shell. The script is designed to toggle the desktop,...

AI-Powered Villager Pen Testing Tool Hits 11,000 PyPI Downloads Amid Abuse Concerns

A new artificial intelligence AI-powered penetration testing tool linked to a China-based company has attracted nearly 11,000 downloads on the Python Package Index PyPI repository, raising concerns that it could be repurposed by cybercriminals for malicious purposes. Dubbed Villager, the framewor...

offensiveinterview

It is an offensive tool for penetration testing and red teaming. The repository contains a collection of interview questions to screen offensive red team/pentest candidates, categorized into open-ended, knowledge-based, and scenario-based questions. The questions cover various topics such as...

From Firewalls to Frontiers: AI Red-Teaming Is a Domain-Specific Evolution of Cyber Red-Teaming

A red team simulates adversary attacks to help defenders find effective strategies to defend their systems in a real-world operational setting. As more enterprise systems adopt AI, red-teaming will need to evolve to address the unique vulnerabilities and risks posed by AI systems. We take the...

Amazon Nova AI Challenge -- Trusted AI: Advancing Secure, AI-Assisted Software Development

AI systems for software development are rapidly gaining prominence, yet significant challenges remain in ensuring their safety. To address this, Amazon launched the Trusted AI track of the Amazon Nova AI Challenge, a global competition among 10 university teams to drive advances in secure AI. In...

Prompt Optimization and Evaluation for LLM Automated Red Teaming

Applications that use Large Language Models LLMs are becoming widespread, making the identification of system vulnerabilities increasingly important. Automated Red Teaming accelerates this effort by using an LLM to generate and execute attacks against target systems. Attack generators are evaluat...

Security Challenges in AI Agent Deployment: Insights from a Large Scale Public Competition

Recent advances have enabled LLM-powered AI agents to autonomously execute complex tasks by combining language model reasoning with tools, memory, and web access. But can these systems be trusted to follow deployment policies in realistic environments, especially under attack? To investigate, we...

PurpCode: Reasoning for Safer Code Generation

We introduce PurpCode, the first post-training recipe for training safe code reasoning models towards generating secure code and defending against malicious cyberactivities. PurpCode trains a reasoning model in two stages: i Rule Learning, which explicitly teaches the model to reference cybersafe...

DREAM: Scalable Red Teaming for Text-To-Image Generative Systems Via Distribution Modeling

Despite the integration of safety alignment and external filters, text-to-image T2I generative models are still susceptible to producing harmful content, such as sexual or violent imagery. This raises serious concerns about unintended exposure and potential misuse. Red teaming, which aims to...

Key Takeaways from the Take Command Summit 2025: Outpacing the Adversary – Red Teaming in a Complex Threat Landscape

The evolving complexity of modern infrastructures calls for more than traditional pen testing. In this session from the Take Command 2025 Virtual Cybersecurity Summit, red team experts shared how organizations are using continuous testing to outpace attackers — and better prepare their teams to...

PRM-Free Security Alignment of Large Models Via Red Teaming and Adversarial Training

Large Language Models LLMs have demonstrated remarkable capabilities across diverse applications, yet they pose significant security risks that threaten their safe deployment in critical domains. Current security alignment methodologies predominantly rely on Process Reward Models PRMs to evaluate...