CVE-2024-48141

A prompt injection vulnerability in the chatbox of Zhipu AI CodeGeeX v2.17.0 allows attackers to access and exfiltrate all previous and subsequent chat data between the user and the AI assistant via a crafted message...

CVE-2024-48140

A prompt injection vulnerability in the chatbox of Butterfly Effect Limited Monica Your AI Copilot powered by ChatGPT4 v6.3.0 allows attackers to access and exfiltrate all previous and subsequent chat data between the user and the AI assistant via a crafted message...

CVE-2024-48145

A prompt injection vulnerability in the chatbox of Netangular Technologies ChatNet AI Version v1.0 allows attackers to access and exfiltrate all previous and subsequent chat data between the user and the AI assistant via a crafted message...

CVE-2024-48142

A prompt injection vulnerability in the chatbox of Butterfly Effect Limited Monica ChatGPT AI Assistant v2.4.0 allows attackers to access and exfiltrate all previous and subsequent chat data between the user and the AI assistant via a crafted message...

CVE-2023-29374

In LangChain through 0.0.131, the LLMMathChain chain allows prompt injection attacks that can execute arbitrary code via the Python exec method...

GitLab Duo Vulnerability Enabled Attackers to Hijack AI Responses with Hidden Prompts

Cybersecurity researchers have discovered an indirect prompt injection flaw in GitLab's artificial intelligence AI assistant Duo that could have allowed attackers to steal source code and inject untrusted HTML into its responses, which could then be used to direct victims to malicious websites...

CVE-2023-32786

In Langchain through 0.0.155, prompt injection allows an attacker to force the service to retrieve data from an arbitrary URL, essentially providing SSRF and potentially injecting content into downstream tasks...

A Critical Evaluation of Defenses against Prompt Injection Attacks

Large Language Models LLMs are vulnerable to prompt injection attacks, and several defenses have recently been proposed, often claiming to mitigate these attacks successfully. However, we argue that existing studies lack a principled approach to evaluating these defenses. In this paper, we argue...

Mapping the Future of AI Security

AI security is one of the most pressing challenges facing the world today. Artificial intelligence is extraordinarily powerful, and, especially considering the advent of Agentic AI, growing more so by the day. But it is for this reason that securing it is so important. AI handles massive amounts ...

Alignment under Pressure: the Case for Informed Adversaries When Evaluating LLM Defenses

Large language models LLMs are rapidly deployed in real-world applications ranging from chatbots to agentic systems. Alignment is one of the main approaches used to defend against attacks such as prompt injection and jailbreaks. Recent defenses report near-zero Attack Success Rates ASR even again...

Is Your Prompt Safe? Investigating Prompt Injection Attacks against Open-Source LLMs

Whitepaper called Is Your Prompt Safe? Investigating Prompt Injection Attacks Against Open-Source LLMs...

The Hidden Dangers of Browsing AI Agents

Autonomous browsing agents powered by large language models LLMs are increasingly used to automate web-based tasks. However, their reliance on dynamic content, tool execution, and user-provided data exposes them to a broad attack surface. This paper presents a comprehensive security evaluation of...

WASP: Benchmarking Web Agent Security against Prompt Injection Attacks

Autonomous UI agents powered by AI have tremendous potential to boost human productivity by automating routine tasks such as filing taxes and paying bills. However, a major challenge in unlocking their full potential is security, which is exacerbated by the agent's ability to take action on their...

Analysing Safety Risks in LLMs Fine-Tuned with Pseudo-Malicious Cyber Security Data

The integration of large language models LLMs into cyber security applications presents significant opportunities, such as enhancing threat analysis and malware detection, but can also introduce critical risks and safety concerns, including personal data leakage and automated generation of new...

DataSentinel: a Game-Theoretic Detection of Prompt Injection Attacks

LLM-integrated applications and agents are vulnerable to prompt injection attacks, where an attacker injects prompts into their inputs to induce attacker-desired outputs. A detection method aims to determine whether a given input is contaminated by an injected prompt. However, existing detection...

FreeBSD : vscode -- security feature bypass vulnerability (6f10b49d-07b1-4be4-8abf-edf880b16ad2)

The version of FreeBSD installed on the remote host is prior to tested version. It is, therefore, affected by a vulnerability as referenced in the 6f10b49d-07b1-4be4-8abf-edf880b16ad2 advisory. VSCode developers report: A security feature bypass vulnerability exists in VS Code 1.100.0 and earlier...

GenAI Security: Outsmarting the Bots with a Proactive Testing Framework

The increasing sophistication and integration of Generative AI GenAI models into diverse applications introduce new security challenges that traditional methods struggle to address. This research explores the critical need for proactive security measures to mitigate the risks associated with...

Red Teaming the Mind of the Machine: a Systematic Evaluation of Prompt Injection and Jailbreak Vulnerabilities in LLMs

Large Language Models LLMs are increasingly integrated into consumer and enterprise applications. Despite their capabilities, they remain susceptible to adversarial attacks such as prompt injection and jailbreaks that override alignment safeguards. This paper provides a systematic investigation o...

vscode -- security feature bypass vulnerability

VSCode developers report: A security feature bypass vulnerability exists in VS Code 1.100.0 and earlier versions where a maliciously crafted URL could be considered trusted when it should not have due to how VS Code handled glob patterns in the trusted domains feature. When paired with the fetch...

Defending against Indirect Prompt Injection by Instruction Detection

The integration of Large Language Models LLMs with external sources is becoming increasingly common, with Retrieval-Augmented Generation RAG being a prominent example. However, this integration introduces vulnerabilities of Indirect Prompt Injection IPI attacks, where hidden instructions embedded...