Not All Tokens Are Created Equal: Query-Efficient Jailbreak Fuzzing for LLMs

Large Language ModelsLLMs are widely deployed, yet are vulnerable to jailbreak prompts that elicit policy-violating outputs. Although prior studies have uncovered these risks, they typically treat all tokens as equally important during prompt mutation, overlooking the varying contributions of...

Jailbreak Mimicry: Automated Discovery of Narrative-Based Jailbreaks for Large Language Models

Large language models LLMs remain vulnerable to sophisticated prompt engineering attacks that exploit contextual framing to bypass safety mechanisms, posing significant risks in cybersecurity applications. We introduce Jailbreak Mimicry, a systematic methodology for training compact attacker mode...

DecipherGuard: Understanding and Deciphering Jailbreak Prompts for a Safer Deployment of Intelligent Software Systems

Intelligent software systems powered by Large Language Models LLMs are increasingly deployed in critical sectors, raising concerns about their safety during runtime. Through an industry-academic collaboration when deploying an LLM-powered virtual customer assistant, a critical software engineerin...

LLM Jailbreak Detection for (Almost) Free!

Large language models LLMs enhance security through alignment when widely used, but remain susceptible to jailbreak attacks capable of producing inappropriate content. Jailbreak detection methods show promise in mitigating jailbreak attacks through the assistance of other models or multiple model...

Jailbreaking Commercial Black-Box LLMs with Explicitly Harmful Prompts

Evaluating jailbreak attacks is challenging when prompts are not overtly harmful or fail to induce harmful outputs. Unfortunately, many existing red-teaming datasets contain such unsuitable prompts. To evaluate attacks accurately, these datasets need to be assessed and cleaned for maliciousness...

LLMs Caught in the Crossfire: Malware Requests and Jailbreak Challenges

The widespread adoption of Large Language Models LLMs has heightened concerns about their security, particularly their vulnerability to jailbreak attacks that leverage crafted prompts to generate malicious outputs. While prior research has been conducted on general security capabilities of LLMs,...

One Model Transfer to All: on Robust Jailbreak Prompts Generation against LLMs

Safety alignment in large language models LLMs is increasingly compromised by jailbreak attacks, which can manipulate these models to generate harmful or unintended content. Investigating these attacks is crucial for uncovering model vulnerabilities. However, many existing jailbreak strategies fa...

Graph of Attacks: Improved Black-Box and Interpretable Jailbreaks for LLMs

The challenge of ensuring Large Language Models LLMs align with societal standards is of increasing interest, as these models are still prone to adversarial jailbreaks that bypass their safety mechanisms. Identifying these vulnerabilities is crucial for enhancing the robustness of LLMs against su...

GraphAttack: Exploiting Representational Blindspots in LLM Safety Mechanisms

Large Language Models LLMs have been equipped with safety mechanisms to prevent harmful outputs, but these guardrails can often be bypassed through "jailbreak" prompts. This paper introduces a novel graph-based approach to systematically generate jailbreak prompts through semantic transformations...