Jailbreaking LLMs Via Calibration

Safety alignment in Large Language Models LLMs often creates a systematic discrepancy between a model's aligned output and the underlying pre-aligned data distribution. We propose a framework in which the effect of safety alignment on next-token prediction is modeled as a systematic distortion of...

AJAR: Adaptive Jailbreak Architecture for Red-Teaming

As Large Language Models LLMs evolve from static chatbots into autonomous agents capable of tool execution, the landscape of AI safety is shifting from content moderation to action security. However, existing red-teaming frameworks remain bifurcated: they either focus on rigid, script-based text...

The Echo Chamber Multi-Turn LLM Jailbreak

The availability of Large Language Models LLMs has led to a new generation of powerful chatbots that can be developed at relatively low cost. As companies deploy these tools, security challenges need to be addressed to prevent financial loss and reputational damage. A key security challenge is...

Multi-Turn Jailbreaking Attack in Multi-Modal Large Language Models

In recent years, the security vulnerabilities of Multi-modal Large Language Models MLLMs have become a serious concern in the Generative Artificial Intelligence GenAI research. These highly intelligent models, capable of performing multi-modal tasks with high accuracy, are also severely susceptib...

Jailbreaking Attacks Vs. Content Safety Filters: How Far Are We in the LLM Safety Arms Race?

As large language models LLMs are increasingly deployed, ensuring their safe use is paramount. Jailbreaking, adversarial prompts that bypass model alignment to trigger harmful outputs, present significant risks, with existing studies reporting high success rates in evading common LLMs. However,...

Trust in LLM-Controlled Robotics: A Survey of Security Threats, Defenses and Challenges

The integration of Large Language Models LLMs into robotics has revolutionized their ability to interpret complex human commands and execute sophisticated tasks. However, such paradigm shift introduces critical security vulnerabilities stemming from the ''embodiment gap'', a discord between the...

Prompt injection is a problem that may never be fixed, warns NCSC

Prompt injection is shaping up to be one of the most stubborn problems in AI security, and the UK’s National Cyber Security Centre NCSC has warned that it may never be “fixed” in the way SQL injection was. Two years ago, the NCSC said prompt injection might turn out to be the “SQL injection of th...

A Practical Framework for Evaluating Medical AI Security: Reproducible Assessment of Jailbreaking and Privacy Vulnerabilities across Clinical Specialties

Medical Large Language Models LLMs are increasingly deployed for clinical decision support across diverse specialties, yet systematic evaluation of their robustness to adversarial misuse and privacy leakage remains inaccessible to most researchers. Existing security benchmarks require GPU cluster...

TeleAI-Safety: A Comprehensive LLM Jailbreaking Benchmark Towards Attacks, Defenses, and Evaluations

While the deployment of large language models LLMs in high-value industries continues to expand, the systematic assessment of their safety against jailbreak and prompt-based attacks remains insufficient. Existing safety evaluation benchmarks and frameworks are often limited by an imbalanced...

Safe2Harm: Semantic Isomorphism Attacks for Jailbreaking Large Language Models

Large Language Models LLMs have demonstrated exceptional performance across various tasks, but their security vulnerabilities can be exploited by attackers to generate harmful content, causing adverse impacts across various societal domains. Most existing jailbreak methods revolve around Prompt...

Jailbreaking Large Vision Language Models in Intelligent Transportation Systems

Large Vision Language Models LVLMs demonstrate strong capabilities in multimodal reasoning and many real-world applications, such as visual question answering. However, LVLMs are highly vulnerable to jailbreaking attacks. This paper systematically analyzes the vulnerabilities of LVLMs integrated ...

Can AI Models Be Jailbroken to Phish Elderly Victims? an End-To-End Evaluation

We present an end-to-end demonstration of how attackers can exploit AI safety failures to harm vulnerable populations: from jailbreaking LLMs to generate phishing content, to deploying those messages against real targets, to successfully compromising elderly victims. We systematically evaluated...

DrAttack

DrAttack: Prompt Decomposition and Reconstruction Makes Powerf...

JPRO: Automated Multimodal Jailbreaking Via Multi-Agent Collaboration Framework

The widespread application of large VLMs makes ensuring their secure deployment critical. While recent studies have demonstrated jailbreak attacks on VLMs, existing approaches are limited: they require either white-box access, restricting practicality, or rely on manually crafted patterns, leadin...

AutoAdv: Automated Adversarial Prompting for Multi-Turn Jailbreaking of Large Language Models

Large Language Models LLMs remain vulnerable to jailbreaking attacks where adversarial prompts elicit harmful outputs, yet most evaluations focus on single-turn interactions while real-world attacks unfold through adaptive multi-turn conversations. We present AutoAdv, a training-free framework fo...

Prevalence of Security and Privacy Risk-Inducing Usage of AI-Based Conversational Agents

Recent improvement gains in large language models LLMs have lead to everyday usage of AI-based Conversational Agents CAs. At the same time, LLMs are vulnerable to an array of threats, including jailbreaks and, for example, causing remote code execution when fed specific inputs. As a result, users...

Researchers break OpenAI guardrails

The maker of ChatGPT released a toolkit to help protect its AI from attack earlier this month. Almost immediately, someone broke it. On October 6, OpenAI ran an event called DevDay where it unveiled a raft of new tools and services for software programmers who use its products. As part of that, i...

Countermind: A Multi-Layered Security Architecture for Large Language Models

The security of Large Language Model LLM applications is fundamentally challenged by "form-first" attacks like prompt injection and jailbreaking, where malicious instructions are embedded within user inputs. Conventional defenses, which rely on post hoc output filtering, are often brittle and fai...

Pattern Enhanced Multi-Turn Jailbreaking: Exploiting Structural Vulnerabilities in Large Language Models

Large language models LLMs remain vulnerable to multi-turn jailbreaking attacks that exploit conversational context to bypass safety constraints gradually. These attacks target different harm categories like malware generation, harassment, or fraud through distinct conversational approaches...

AutoDAN-Reasoning: Enhancing Strategies Exploration Based Jailbreak Attacks with Test-Time Scaling

Recent advancements in jailbreaking large language models LLMs, such as AutoDAN-Turbo, have demonstrated the power of automated strategy discovery. AutoDAN-Turbo employs a lifelong learning agent to build a rich library of attack strategies from scratch. While highly effective, its test-time...