1611 matches found
LlamaFirewall: an Open Source Guardrail System for Building Secure AI Agents
Large language models LLMs have evolved from simple chatbots into autonomous agents capable of performing complex tasks such as editing production code, orchestrating workflows, and taking higher-stakes actions based on untrusted inputs like webpages and emails. These capabilities introduce new...
OET: Optimization-Based Prompt Injection Evaluation Toolkit
Large Language Models LLMs have demonstrated remarkable capabilities in natural language understanding and generation, enabling their widespread adoption across various domains. However, their susceptibility to prompt injection attacks poses significant security risks, as adversarial inputs can...
Researchers Demonstrate How MCP Prompt Injection Can Be Used for Both Attack and Defense
As the field of artificial intelligence AI continues to evolve at a rapid pace, fresh research has found how techniques that render the Model Context Protocol MCP susceptible to prompt injection attacks could be used to develop security tooling or identify malicious tools, according to a new repo...
New Reports Uncover Jailbreaks, Unsafe Code, and Data Theft Risks in Leading AI Systems
Various generative artificial intelligence GenAI services have been found vulnerable to two types of jailbreak attacks that make it possible to produce illicit or dangerous content. The first of the two techniques, codenamed Inception, instructs an AI tool to imagine a fictitious scenario, which...
Akamai Firewall for AI: Get Powerful Protection for New LLM App Threats
Protect against LLM attacks such as prompt injection, exfiltration and extraction, and toxic AI outputs with Akamai Firewall for AI...
Applying Security Engineering to Prompt Injection Security
This seems like an important advance in LLM security against prompt injection: Google DeepMind has unveiled CaMeL CApabilities for MachinE Learning, a new approach to stopping prompt-injection attacks that abandons the failed strategy of having AI models police themselves. Instead, CaMeL treats...
CachePrune: Neural-Based Attribution Defense against Indirect Prompt Injection Attacks
Large Language Models LLMs are identified as being susceptible to indirect prompt injection attack, where the model undesirably deviates from user-provided instructions by executing tasks injected in the prompt context. This vulnerability stems from LLMs' inability to distinguish between data and...
Token-Efficient Prompt Injection Attack: Provoking Cessation in LLM Reasoning Via Adaptive Token Compression
While reasoning large language models LLMs demonstrate remarkable performance across various tasks, they also contain notable security vulnerabilities. Recent research has uncovered a "thinking-stopped" vulnerability in DeepSeek-R1, where model-generated reasoning tokens can forcibly interrupt th...
Robustness Via Referencing: Defending against Prompt Injection Attacks by Referencing the Executed Instruction
Large language models LLMs have demonstrated impressive performance and have come to dominate the field of natural language processing NLP across various tasks. However, due to their strong instruction-following capabilities and inability to distinguish between instructions and data content, LLMs...
CVE-2025-31363
Mattermost versions 10.4.x = 10.4.2, 10.5.x = 10.5.0, 9.11.x = 9.11.9 fail to restrict domains the LLM can request to contact upstream which allows an authenticated user to exfiltrate data from an arbitrary server accessible to the victim via performing a prompt injection in the AI plugin's Jira...
Adversarial Attacks on LLM-As-A-Judge Systems: Insights from Prompt Injections
LLM as judge systems used to assess text quality code correctness and argument strength are vulnerable to prompt injection attacks. We introduce a framework that separates content author attacks from system prompt attacks and evaluate five models Gemma 3.27B Gemma 3.4B Llama 3.2 3B GPT 4 and Clau...
Mattermost Information Disclosure Vulnerability
Mattermost is an open source collaboration platform from Mattermost, Inc. in the United States. Mattermost suffers from an information disclosure vulnerability. The vulnerability stems from an under-restricted LLM request domain. An attacker can exploit the vulnerability to perform prompt injecti...
Prompt Injection
github.com/mattermost/mattermost-server is vulnerable to prompt injection. The vulnerability is due to insufficient domain restriction to the AI plugin's Jira tool, allowing authenticated users to exfiltrate data from arbitrary servers via crafted prompts...
Mattermost Server 9.11.x < 9.11.10 / 10.4.x < 10.4.3 / 10.5.x < 10.5.1 / 10.6.0 (MMSA-2024-00401)
The version of Mattermost Server installed on the remote host is prior to 9.11.10, 10.4.3, or 10.5.1 / 10.6.0. It is, therefore, affected by a vulnerability as referenced in the MMSA-2024-00401 advisory. - Mattermost versions 10.4.x = 10.4.2, 10.5.x = 10.5.0, 9.11.x = 9.11.9 fail to restrict...
Breaking the Prompt Wall (I): a Real-World Case Study of Attacking ChatGPT Via Lightweight Prompt Injection
Whitepaper called Breaking The Prompt Wall I: A Real-World Case Study Of Attacking ChatGPT Via Lightweight Prompt Injection...
CVE-2025-3579
In versions prior to Aidex 1.7, an authenticated malicious user, taking advantage of an open registry, could execute unauthorised commands within the system. This includes executing operating system Unix commands, interacting with internal services such as PHP or MySQL, and even invoking native...
GHSA-9H6J-4FFX-CM84 Mattermost doesn't restrict domains LLM can request to contact upstream
Mattermost versions 10.4.x = 10.4.2, 10.5.x = 10.5.0, 9.11.x = 9.11.9 fail to restrict domains the LLM can request to contact upstream which allows an authenticated user to exfiltrate data from an arbitrary server accessible to the victim via performing a prompt injection in the AI plugin's Jira...
Mattermost doesn't restrict domains LLM can request to contact upstream
Mattermost versions 10.4.x = 10.4.2, 10.5.x = 10.5.0, 9.11.x = 9.11.9 fail to restrict domains the LLM can request to contact upstream which allows an authenticated user to exfiltrate data from an arbitrary server accessible to the victim via performing a prompt injection in the AI plugin's Jira...
CVE-2025-31363
Mattermost versions 10.4.x = 10.4.2, 10.5.x = 10.5.0, 9.11.x = 9.11.9 fail to restrict domains the LLM can request to contact upstream which allows an authenticated user to exfiltrate data from an arbitrary server accessible to the victim via performing a prompt injection in the AI plugin's Jira...
