Jailbreaking Attacks Vs. Content Safety Filters: How Far Are We in the LLM Safety Arms Race?

As large language models LLMs are increasingly deployed, ensuring their safe use is paramount. Jailbreaking, adversarial prompts that bypass model alignment to trigger harmful outputs, present significant risks, with existing studies reporting high success rates in evading common LLMs. However,...

Trust in LLM-Controlled Robotics: A Survey of Security Threats, Defenses and Challenges

The integration of Large Language Models LLMs into robotics has revolutionized their ability to interpret complex human commands and execute sophisticated tasks. However, such paradigm shift introduces critical security vulnerabilities stemming from the ''embodiment gap'', a discord between the...

Prompt injection is a problem that may never be fixed, warns NCSC

Prompt injection is shaping up to be one of the most stubborn problems in AI security, and the UK’s National Cyber Security Centre NCSC has warned that it may never be “fixed” in the way SQL injection was. Two years ago, the NCSC said prompt injection might turn out to be the “SQL injection of th...

A Practical Framework for Evaluating Medical AI Security: Reproducible Assessment of Jailbreaking and Privacy Vulnerabilities across Clinical Specialties

Medical Large Language Models LLMs are increasingly deployed for clinical decision support across diverse specialties, yet systematic evaluation of their robustness to adversarial misuse and privacy leakage remains inaccessible to most researchers. Existing security benchmarks require GPU cluster...

TeleAI-Safety: A Comprehensive LLM Jailbreaking Benchmark Towards Attacks, Defenses, and Evaluations

While the deployment of large language models LLMs in high-value industries continues to expand, the systematic assessment of their safety against jailbreak and prompt-based attacks remains insufficient. Existing safety evaluation benchmarks and frameworks are often limited by an imbalanced...

Safe2Harm: Semantic Isomorphism Attacks for Jailbreaking Large Language Models

Large Language Models LLMs have demonstrated exceptional performance across various tasks, but their security vulnerabilities can be exploited by attackers to generate harmful content, causing adverse impacts across various societal domains. Most existing jailbreak methods revolve around Prompt...

Jailbreaking Large Vision Language Models in Intelligent Transportation Systems

Large Vision Language Models LVLMs demonstrate strong capabilities in multimodal reasoning and many real-world applications, such as visual question answering. However, LVLMs are highly vulnerable to jailbreaking attacks. This paper systematically analyzes the vulnerabilities of LVLMs integrated ...

Can AI Models Be Jailbroken to Phish Elderly Victims? an End-To-End Evaluation

We present an end-to-end demonstration of how attackers can exploit AI safety failures to harm vulnerable populations: from jailbreaking LLMs to generate phishing content, to deploying those messages against real targets, to successfully compromising elderly victims. We systematically evaluated...

DrAttack

DrAttack: Prompt Decomposition and Reconstruction Makes Powerf...

JPRO: Automated Multimodal Jailbreaking Via Multi-Agent Collaboration Framework

The widespread application of large VLMs makes ensuring their secure deployment critical. While recent studies have demonstrated jailbreak attacks on VLMs, existing approaches are limited: they require either white-box access, restricting practicality, or rely on manually crafted patterns, leadin...

AutoAdv: Automated Adversarial Prompting for Multi-Turn Jailbreaking of Large Language Models

Large Language Models LLMs remain vulnerable to jailbreaking attacks where adversarial prompts elicit harmful outputs, yet most evaluations focus on single-turn interactions while real-world attacks unfold through adaptive multi-turn conversations. We present AutoAdv, a training-free framework fo...

Prevalence of Security and Privacy Risk-Inducing Usage of AI-Based Conversational Agents

Recent improvement gains in large language models LLMs have lead to everyday usage of AI-based Conversational Agents CAs. At the same time, LLMs are vulnerable to an array of threats, including jailbreaks and, for example, causing remote code execution when fed specific inputs. As a result, users...

Researchers break OpenAI guardrails

The maker of ChatGPT released a toolkit to help protect its AI from attack earlier this month. Almost immediately, someone broke it. On October 6, OpenAI ran an event called DevDay where it unveiled a raft of new tools and services for software programmers who use its products. As part of that, i...

Countermind: A Multi-Layered Security Architecture for Large Language Models

The security of Large Language Model LLM applications is fundamentally challenged by "form-first" attacks like prompt injection and jailbreaking, where malicious instructions are embedded within user inputs. Conventional defenses, which rely on post hoc output filtering, are often brittle and fai...

Pattern Enhanced Multi-Turn Jailbreaking: Exploiting Structural Vulnerabilities in Large Language Models

Large language models LLMs remain vulnerable to multi-turn jailbreaking attacks that exploit conversational context to bypass safety constraints gradually. These attacks target different harm categories like malware generation, harassment, or fraud through distinct conversational approaches...

AutoDAN-Reasoning: Enhancing Strategies Exploration Based Jailbreak Attacks with Test-Time Scaling

Recent advancements in jailbreaking large language models LLMs, such as AutoDAN-Turbo, have demonstrated the power of automated strategy discovery. AutoDAN-Turbo employs a lifelong learning agent to build a rich library of attack strategies from scratch. While highly effective, its test-time...

PS4-4.05-Kernel-Exploit

This repository contains a fully implemented kernel exploit for the PlayStation 4 on firmware version 4.05. The exploit, known as "namedobj," allows for arbitrary code execution as kernel, enabling jailbreaking and kernel-level modifications to the system. It includes a loader that listens for...

ENJ: Optimizing Noise with Genetic Algorithms to Jailbreak LSMs

The widespread application of Large Speech Models LSMs has made their security risks increasingly prominent. Traditional speech adversarial attack methods face challenges in balancing effectiveness and stealth. This paper proposes Evolutionary Noise Jailbreak ENJ, which utilizes a genetic algorit...

Measuring the Vulnerability Disclosure Policies of AI Vendors

As AI is increasingly integrated into products and critical systems, researchers are paying greater attention to identifying related vulnerabilities. Effective remediation depends on whether vendors are willing to accept and respond to AI vulnerability reports. In this paper, we examine the...

Amazon Nova AI Challenge -- Trusted AI: Advancing Secure, AI-Assisted Software Development

AI systems for software development are rapidly gaining prominence, yet significant challenges remain in ensuring their safety. To address this, Amazon launched the Trusted AI track of the Amazon Nova AI Challenge, a global competition among 10 university teams to drive advances in secure AI. In...