232 matches found
Risk-Adjusted Harm Scoring for Automated Red Teaming for LLMs in Financial Services
The rapid adoption of large language models LLMs in financial services introduces new operational, regulatory, and security risks. Yet most red-teaming benchmarks remain domain-agnostic and fail to capture failure modes specific to regulated BFSI settings, where harmful behavior can be elicited...
A Systematic Review of Algorithmic Red Teaming Methodologies for Assurance and Security of AI Applications
Cybersecurity threats are becoming increasingly sophisticated, making traditional defense mechanisms and manual red teaming approaches insufficient for modern organizations. While red teaming has long been recognized as an effective method to identify vulnerabilities by simulating real-world...
Red-Teaming Claude Opus and ChatGPT-Based Security Advisors for Trusted Execution Environments
Trusted Execution Environments TEEs e.g., Intel SGX and ArmTrustZone aim to protect sensitive computation from a compromised operating system, yet real deployments remain vulnerable to microarchitectural leakage, side-channel attacks, and fault injection. In parallel, security teams increasingly...
Trojan Horses in Recruiting: A Red-Teaming Case Study on Indirect Prompt Injection in Standard Vs. Reasoning Models
As Large Language Models LLMs are increasingly integrated into automated decision-making pipelines, specifically within Human Resources HR, the security implications of Indirect Prompt Injection IPI become critical. While a prevailing hypothesis posits that "Reasoning" or "Chain-of-Thought" Model...
Aegis: Towards Governance, Integrity, and Security of AI Voice Agents
With the rapid advancement and adoption of Audio Large Language Models ALLMs, voice agents are now being deployed in high-stakes domains such as banking, customer service, and IT support. However, their vulnerabilities to adversarial misuse still remain unexplored. While prior work has examined...
Top AI Tools for Red Teaming in 2026
Red teaming has undergone a radical evolution. Modern organizations can no longer rely solely on human creativity or…...
Jailbreaking LLMs Via Calibration
Safety alignment in Large Language Models LLMs often creates a systematic discrepancy between a model's aligned output and the underlying pre-aligned data distribution. We propose a framework in which the effect of safety alignment on next-token prediction is modeled as a systematic distortion of...
Putting Privacy to the Test: Introducing Red Teaming for Research Data Anonymization
Recently, the data protection practices of researchers in human-computer interaction and elsewhere have gained attention. Initial results suggest that researchers struggle with anonymization, partly due to a lack of clear, actionable guidance. In this work, we propose simulating re-identification...
AJAR: Adaptive Jailbreak Architecture for Red-Teaming
As Large Language Models LLMs evolve from static chatbots into autonomous agents capable of tool execution, the landscape of AI safety is shifting from content moderation to action security. However, existing red-teaming frameworks remain bifurcated: they either focus on rigid, script-based text...
SecureCAI: Injection-Resilient LLM Assistants for Cybersecurity Operations
Large Language Models have emerged as transformative tools for Security Operations Centers, enabling automated log analysis, phishing triage, and malware explanation; however, deployment in adversarial cybersecurity environments exposes critical vulnerabilities to prompt injection attacks where...
RedBench: A Universal Dataset for Comprehensive Red Teaming of Large Language Models
As large language models LLMs become integral to safety-critical applications, ensuring their robustness against adversarial prompts is paramount. However, existing red teaming datasets suffer from inconsistent risk categorizations, limited domain coverage, and outdated evaluations, hindering...
Jailbreaking LLMs and VLMs: Mechanisms, Evaluation, and Unified Defense
This paper provides a systematic survey of jailbreak attacks and defenses on Large Language Models LLMs and Vision-Language Models VLMs, emphasizing that jailbreak vulnerabilities stem from structural factors such as incomplete training data, linguistic ambiguity, and generative uncertainty. It...
OpenRT: An Open-Source Red Teaming Framework for Multimodal LLMs
The rapid integration of Multimodal Large Language Models MLLMs into critical applications is increasingly hindered by persistent safety vulnerabilities. However, existing red-teaming benchmarks are often fragmented, limited to single-turn text interactions, and lack the scalability required for...
Jailbreak-Zero: A Path to Pareto Optimal Red Teaming for Large Language Models
This paper introduces Jailbreak-Zero, a novel red teaming methodology that shifts the paradigm of Large Language Model LLM safety evaluation from a constrained example-based approach to a more expansive and effective policy-based framework. By leveraging an attack LLM to generate a high volume of...
Exploit for CVE-2025-14174
CVE-2025-14174 PoC Exploit Repository Overview This reposi...
Cisco Integrated AI Security and Safety Framework Report
Artificial intelligence AI systems are being readily and rapidly adopted, increasingly permeating critical domains: from consumer platforms and enterprise software to networked systems with embedded agents. While this has unlocked potential for human productivity gains, the attack surface has...
A Safety and Security Framework for Real-World Agentic Systems
This paper introduces a dynamic and actionable framework for securing agentic AI systems in enterprise deployment. We contend that safety and security are not merely fixed attributes of individual models but also emergent properties arising from the dynamic interactions among models, orchestrator...
Beyond Fixed and Dynamic Prompts: Embedded Jailbreak Templates for Advancing LLM Security
As the use of large language models LLMs continues to expand, ensuring their safety and robustness has become a critical challenge. In particular, jailbreak attacks that bypass built-in safety mechanisms are increasingly recognized as a tangible threat across industries, driving the need for...
From Tabletop to Turnkey: Building Cyber Resilience in Financial Services
Introduction Financial institutions are facing a new reality: cyber-resilience has passed from being a best practice to an operational necessity to a prescriptive regulatory requirement. Crisis management or Tabletop exercises, for a long time relatively rare in the context of cybersecurity, have...
SIRAJ: Diverse and Efficient Red-Teaming for LLM Agents Via Distilled Structured Reasoning
The ability of LLM agents to plan and invoke tools exposes them to new safety risks, making a comprehensive red-teaming system crucial for discovering vulnerabilities and ensuring their safe deployment. We present SIRAJ: a generic red-teaming framework for arbitrary black-box LLM agents. We emplo...