Probing the Robustness of Large Language Models Safety to Latent Perturbations

Safety alignment is a key requirement for building reliable Artificial General Intelligence. Despite significant advances in safety alignment, we observe that minor latent shifts can still trigger unsafe responses in aligned models. We argue that this stems from the shallow nature of existing...

LLM Jailbreak Oracle

As large language models LLMs become increasingly deployed in safety-critical applications, the lack of systematic methods to assess their vulnerability to jailbreak attacks presents a critical security gap. We introduce the jailbreak oracle problem: given a model, prompt, and decoding strategy,...

SoK: Evaluating Jailbreak Guardrails for Large Language Models

Large Language Models LLMs have achieved remarkable progress, but their deployment has exposed critical vulnerabilities, particularly to jailbreak attacks that circumvent safety mechanisms. Guardrails--external defense mechanisms that monitor and control LLM interaction--have emerged as a promisi...

Evaluation Empirique De La Sécurisation Et De L'Alignement De ChatGPT Et Gemini: Analyse Comparative Des Vulnérabilités Par Expérimentations De Jailbreaks

Large Language models LLMs are transforming digital usage, particularly in text generation, image creation, information retrieval and code development. ChatGPT, launched by OpenAI in November 2022, quickly became a reference, prompting the emergence of competitors such as Google's Gemini. However...

LLMs Caught in the Crossfire: Malware Requests and Jailbreak Challenges

The widespread adoption of Large Language Models LLMs has heightened concerns about their security, particularly their vulnerability to jailbreak attacks that leverage crafted prompts to generate malicious outputs. While prior research has been conducted on general security capabilities of LLMs,...

Why LLM Safety Guardrails Collapse after Fine-Tuning: a Similarity Analysis between Alignment and Fine-Tuning Datasets

Recent advancements in large language models LLMs have underscored their vulnerability to safety alignment jailbreaks, particularly when subjected to downstream fine-tuning. However, existing mitigation strategies primarily focus on reactively addressing jailbreak incidents after safety guardrail...

BitBypass: a New Direction in Jailbreaking Aligned Large Language Models with Bitstream Camouflage

The inherent risk of generating harmful and unsafe content by Large Language Models LLMs, has highlighted the need for their safety alignment. Various techniques like supervised fine-tuning, reinforcement learning from human feedback, and red-teaming were developed for ensuring the safety alignme...

Align Is Not Enough: Multimodal Universal Jailbreak Attack against Multimodal Large Language Models

Large Language Models LLMs have evolved into Multimodal Large Language Models MLLMs, significantly enhancing their capabilities by integrating visual information and other types, thus aligning more closely with the nature of human intelligence, which processes a variety of data forms beyond just...

Test-Time Immunization: a Universal Defense Framework against Jailbreaks for (Multimodal) Large Language Models

While multimodal large language models LLMs have attracted widespread attention due to their exceptional capabilities, they remain vulnerable to jailbreak attacks. Various defense methods are proposed to defend against jailbreak attacks, however, they are often tailored to specific types of...

Jailbreak Distillation: Renewable Safety Benchmarking

Large language models LLMs are rapidly deployed in critical applications, raising urgent needs for robust safety benchmarking. We propose Jailbreak Distillation JBDistill, a novel benchmark construction framework that "distills" jailbreak attacks into high-quality and easily-updatable safety...

GeneBreaker: Jailbreak Attacks against DNA Language Models with Pathogenicity Guidance

DNA, encoding genetic instructions for almost all living organisms, fuels groundbreaking advances in genomics and synthetic biology. Recently, DNA Foundation Models have achieved success in designing synthetic functional DNA sequences, even whole genomes, but their susceptibility to jailbreaking...

Finetuning-Activated Backdoors in LLMs

Finetuning openly accessible Large Language Models LLMs has become standard practice for achieving task-specific performance improvements. Until now, finetuning has been regarded as a controlled and secure process in which training on benign datasets led to predictable behaviors. In this paper, w...

One Model Transfer to All: on Robust Jailbreak Prompts Generation against LLMs

Safety alignment in large language models LLMs is increasingly compromised by jailbreak attacks, which can manipulate these models to generate harmful or unintended content. Investigating these attacks is crucial for uncovering model vulnerabilities. However, many existing jailbreak strategies fa...

CVE-2021-42111

An issue was discovered in the RCDevs OpenOTP app 1.4.13 and 1.4.14 for iOS. If it is installed on a jailbroken device, it is possible to retrieve the PIN code used to access the application. The IOS app version 1.4.1631262629 resolves this issue by storing a hash PIN code...

CVE-2020-3681

Authenticated and encrypted payload MMEs can be forged and remotely sent to any HPAV2 system using a jailbreak key recoverable from code...

CVE-2019-19834

Directory Traversal in ruckuscli2 in Ruckus Wireless Unleashed through 200.7.10.102.64 allows a remote attacker to jailbreak the CLI via enable-debug-script-exec with ../../../bin/sh as the parameter...

When Safety Detectors Aren'T Enough: a Stealthy and Effective Jailbreak Attack on LLMs Via Steganographic Techniques

Jailbreak attacks pose a serious threat to large language models LLMs by bypassing built-in safety mechanisms and leading to harmful outputs. Studying these attacks is crucial for identifying vulnerabilities and improving model security. This paper presents a systematic survey of jailbreak method...

SafeKey: Amplifying Aha-Moment Insights for Safety Reasoning

Large Reasoning Models LRMs introduce a new generation paradigm of explicitly reasoning before answering, leading to remarkable improvements in complex tasks. However, they pose great safety risks against harmful queries and adversarial attacks. While recent mainstream safety efforts on LRMs,...

Exploring Jailbreak Attacks on LLMs through Intent Concealment and Diversion

Although large language models LLMs have achieved remarkable advancements, their security remains a pressing concern. One major threat is jailbreak attacks, where adversarial prompts bypass model safeguards to generate harmful or objectionable content. Researchers study jailbreak attacks to...

SudoLLM : on Multi-Role Alignment of Language Models

User authorization-based access privileges are a key feature in many safety-critical systems, but have thus far been absent from the large language model LLM realm. In this work, drawing inspiration from such access control systems, we introduce sudoLLM, a novel framework that results in multi-ro...