System Prompt Extraction Attacks and Defenses in Large Language Models

The system prompt in Large Language Models LLMs plays a pivotal role in guiding model behavior and response generation. Often containing private configuration details, user roles, and operational instructions, the system prompt has become an emerging attack target. Recent studies have shown that...

Red-Teaming Text-To-Image Systems by Rule-Based Preference Modeling

Text-to-image T2I models raise ethical and safety concerns due to their potential to generate inappropriate or harmful images. Evaluating these models' security through red-teaming is vital, yet white-box approaches are limited by their need for internal access, complicating their use with...

Semantic-Preserving Adversarial Attacks on LLMs: an Adaptive Greedy Binary Search Approach

Large Language Models LLMs increasingly rely on automatic prompt engineering in graphical user interfaces GUIs to refine user inputs and enhance response accuracy. However, the diversity of user requirements often leads to unintended misinterpretations, where automated optimizations distort...

CVE-2024-7523

A select option could partially obscure security prompts. This could be used by a malicious site to trick a user into granting permissions. This issue only affects Android versions of Firefox. This vulnerability affects Firefox 129...

CVE-2024-6605

Firefox Android allowed immediate interaction with permission prompts. This could be used for tapjacking. This vulnerability affects Firefox 128...

CVE-2024-27855

The issue was addressed with improved checks. This issue is fixed in iOS 16.7.8 and iPadOS 16.7.8, iOS 17.5 and iPadOS 17.5, macOS Sonoma 14.5, macOS Ventura 13.6.7. A shortcut may be able to use sensitive data with certain actions without prompting the user...

CVE-2024-5389

In lunary-ai/lunary version 1.2.13, an insufficient granularity of access control vulnerability allows users to create, update, get, and delete prompt variations for datasets not owned by their organization. This issue arises due to the application not properly validating the ownership of dataset...

CVE-2024-40807

A logic issue was addressed with improved checks. This issue is fixed in macOS Monterey 12.7.6, macOS Sonoma 14.6, macOS Ventura 13.6.8. A shortcut may be able to use sensitive data with certain actions without prompting the user...

CVE-2023-22499

Deno is a runtime for JavaScript and TypeScript that uses V8 and is built in Rust. Multi-threaded programs were able to spoof interactive permission prompt by rewriting the prompt to suggest that program is waiting on user confirmation to unrelated action. A malicious program could clear the...

One Model Transfer to All: on Robust Jailbreak Prompts Generation against LLMs

Safety alignment in large language models LLMs is increasingly compromised by jailbreak attacks, which can manipulate these models to generate harmful or unintended content. Investigating these attacks is crucial for uncovering model vulnerabilities. However, many existing jailbreak strategies fa...

CVE-2025-4280

MacOS version of Poedit bundles a Python interpreter that inherits the Transparency, Consent, and Control TCC permissions granted by the user to the main application bundle. An attacker with local user access can invoke this interpreter with arbitrary commands or scripts, leveraging the...

CAIN: Hijacking LLM-Humans Conversations Via a Two-Stage Malicious System Prompt Generation and Refining Framework

Large language models LLMs have advanced many applications, but are also known to be vulnerable to adversarial attacks. In this work, we introduce a novel security threat: hijacking AI-human conversations by manipulating LLMs' system prompts to produce malicious answers only to specific targeted...

Adaptive Plan-Execute Framework for Smart Contract Security Auditing

Large Language Models LLMs have shown great promise in code analysis and auditing; however, they still struggle with hallucinations and limited context-aware reasoning. We introduce SmartAuditFlow, a novel Plan-Execute framework that enhances smart contract security analysis through dynamic audit...

Invisible Prompts, Visible Threats: Malicious Font Injection in External Resources for Large Language Models

Large Language Models LLMs are increasingly equipped with capabilities of real-time web search and integrated with protocols like Model Context Protocol MCP. This extension could introduce new security vulnerabilities. We present a systematic investigation of LLM vulnerabilities to hidden...

Are Vision-Language Models Safe in the Wild? A Meme-Based Benchmark Study

Rapid deployment of vision-language models VLMs magnifies safety risks, yet most evaluations rely on artificial images. This study asks: How safe are current VLMs when confronted with meme images that ordinary users share? To investigate this question, we introduce MemeSafetyBench, a...

Evaluating the Efficacy of LLM Safety Solutions : the Palit Benchmark Dataset

Large Language Models LLMs are increasingly integrated into critical systems in industries like healthcare and finance. Users can often submit queries to LLM-enabled chatbots, some of which can enrich responses with information retrieved from internal databases storing sensitive data. This gives...

R1dacted: Investigating Local Censorship in DeepSeek'S R1 Language Model

DeepSeek recently released R1, a high-performing large language model LLM optimized for reasoning tasks. Despite its efficient training pipeline, R1 achieves competitive performance, even surpassing leading reasoning models like OpenAI's o1 on several benchmarks. However, emerging reports suggest...

ProxyPrompt: Securing System Prompts against Prompt Extraction Attacks

The integration of large language models LLMs into a wide range of applications has highlighted the critical role of well-crafted system prompts, which require extensive testing and domain expertise. These prompts enhance task performance but may also encode sensitive information and filtering...

LARGO: Latent Adversarial Reflection through Gradient Optimization for Jailbreaking LLMs

Efficient red-teaming method to uncover vulnerabilities in Large Language Models LLMs is crucial. While recent attacks often use LLMs as optimizers, the discrete language space make gradient-based methods struggle. We introduce LARGO Latent Adversarial Reflection through Gradient Optimization, a...

git: The sideband payload is passed unfiltered to the terminal in git

A flaw was found in Git. When cloning, fetching, or pushing from a server, informational or error messages are transported from the remote Git process to the client via a sideband channel. These messages are prefixed with "remote:" and printed directly to the standard error output. Typically, thi...