AutoRISE: Agent-Driven Strategy Evolution for Red-Teaming Large Language Models

Automated red-teaming methods for large language models typically optimize attack prompts within a fixed, human-designed strategy, leaving the attack strategy itself unchanged. We instead optimize the strategy. We propose AutoRISE, a method that searches over executable attack programs rather tha...

Adaptive Instruction Composition for Automated LLM Red-Teaming

Many approaches to LLM red-teaming leverage an attacker LLM to discover jailbreaks against a target. Several of them task the attacker with identifying effective strategies through trial and error, resulting in a semantically limited range of successes. Another approach discovers diverse attacks ...

ARES: Adaptive Red-Teaming and End-To-End Repair of Policy-Reward System

Reinforcement Learning from Human Feedback RLHF is central to aligning Large Language Models LLMs, yet it introduces a critical vulnerability: an imperfect Reward Model RM can become a single point of failure when it fails to penalize unsafe behaviors. While existing red-teaming approaches...

SkillAttack: Automated Red Teaming of Agent Skills through Attack Path Refinement

LLM-based agent systems increasingly rely on agent skills sourced from open registries to extend their capabilities, yet the openness of such ecosystems makes skills difficult to thoroughly vet. Existing attacks rely on injecting malicious instructions into skills, making them easily detectable b...

TreeTeaming: Autonomous Red-Teaming of Vision-Language Models Via Hierarchical Strategy Exploration

The rapid advancement of Vision-Language Models VLMs has brought their safety vulnerabilities into sharp focus. However, existing red teaming methods are fundamentally constrained by an inherent linear exploration paradigm, confining them to optimizing within a predefined strategy set and...

T-MAP: Red-Teaming LLM Agents with Trajectory-Aware Evolutionary Search

While prior red-teaming efforts have focused on eliciting harmful text outputs from large language models LLMs, such approaches fail to capture agent-specific vulnerabilities that emerge through multi-step tool execution, particularly in rapidly growing ecosystems such as the Model Context Protoc...

PISmith: Reinforcement Learning-Based Red Teaming for Prompt Injection Defenses

Prompt injection poses serious security risks to real-world LLM applications, particularly autonomous agents. Although many defenses have been proposed, their robustness against adaptive attacks remains insufficiently evaluated, potentially creating a false sense of security. In this work, we...

The Face of Penetration Testing is Changing: Announcing Metasploit Pro 5.0.0

The role and demand for red-teaming capabilities are growing, as more exploitable CVEs make their way into criminal hands. Being proactive is no longer a capability that can be reserved for annual tests, but a continuous assessment to determine exposure and even through the validation of an...

Risk-Adjusted Harm Scoring for Automated Red Teaming for LLMs in Financial Services

The rapid adoption of large language models LLMs in financial services introduces new operational, regulatory, and security risks. Yet most red-teaming benchmarks remain domain-agnostic and fail to capture failure modes specific to regulated BFSI settings, where harmful behavior can be elicited...

A Systematic Review of Algorithmic Red Teaming Methodologies for Assurance and Security of AI Applications

Cybersecurity threats are becoming increasingly sophisticated, making traditional defense mechanisms and manual red teaming approaches insufficient for modern organizations. While red teaming has long been recognized as an effective method to identify vulnerabilities by simulating real-world...

Red-Teaming Claude Opus and ChatGPT-Based Security Advisors for Trusted Execution Environments

Trusted Execution Environments TEEs e.g., Intel SGX and ArmTrustZone aim to protect sensitive computation from a compromised operating system, yet real deployments remain vulnerable to microarchitectural leakage, side-channel attacks, and fault injection. In parallel, security teams increasingly...

Trojan Horses in Recruiting: A Red-Teaming Case Study on Indirect Prompt Injection in Standard Vs. Reasoning Models

As Large Language Models LLMs are increasingly integrated into automated decision-making pipelines, specifically within Human Resources HR, the security implications of Indirect Prompt Injection IPI become critical. While a prevailing hypothesis posits that "Reasoning" or "Chain-of-Thought" Model...

Aegis: Towards Governance, Integrity, and Security of AI Voice Agents

With the rapid advancement and adoption of Audio Large Language Models ALLMs, voice agents are now being deployed in high-stakes domains such as banking, customer service, and IT support. However, their vulnerabilities to adversarial misuse still remain unexplored. While prior work has examined...

Top AI Tools for Red Teaming in 2026

Red teaming has undergone a radical evolution. Modern organizations can no longer rely solely on human creativity or…...

Jailbreaking LLMs Via Calibration

Safety alignment in Large Language Models LLMs often creates a systematic discrepancy between a model's aligned output and the underlying pre-aligned data distribution. We propose a framework in which the effect of safety alignment on next-token prediction is modeled as a systematic distortion of...

Putting Privacy to the Test: Introducing Red Teaming for Research Data Anonymization

Recently, the data protection practices of researchers in human-computer interaction and elsewhere have gained attention. Initial results suggest that researchers struggle with anonymization, partly due to a lack of clear, actionable guidance. In this work, we propose simulating re-identification...

AJAR: Adaptive Jailbreak Architecture for Red-Teaming

As Large Language Models LLMs evolve from static chatbots into autonomous agents capable of tool execution, the landscape of AI safety is shifting from content moderation to action security. However, existing red-teaming frameworks remain bifurcated: they either focus on rigid, script-based text...

SecureCAI: Injection-Resilient LLM Assistants for Cybersecurity Operations

Large Language Models have emerged as transformative tools for Security Operations Centers, enabling automated log analysis, phishing triage, and malware explanation; however, deployment in adversarial cybersecurity environments exposes critical vulnerabilities to prompt injection attacks where...

RedBench: A Universal Dataset for Comprehensive Red Teaming of Large Language Models

As large language models LLMs become integral to safety-critical applications, ensuring their robustness against adversarial prompts is paramount. However, existing red teaming datasets suffer from inconsistent risk categorizations, limited domain coverage, and outdated evaluations, hindering...

Jailbreaking LLMs and VLMs: Mechanisms, Evaluation, and Unified Defense

This paper provides a systematic survey of jailbreak attacks and defenses on Large Language Models LLMs and Vision-Language Models VLMs, emphasizing that jailbreak vulnerabilities stem from structural factors such as incomplete training data, linguistic ambiguity, and generative uncertainty. It...