AIRTBench: Measuring Autonomous AI Red Teaming Capabilities in Language Models

We introduce AIRTBench, an AI red teaming benchmark for evaluating language models' ability to autonomously discover and exploit Artificial Intelligence and Machine Learning AI/ML security vulnerabilities. The benchmark consists of 70 realistic black-box capture-the-flag CTF challenges from the...

Doppelgänger Method: Breaking Role Consistency in LLM Agent via Prompt-based Transferable Adversarial Attack

Since the advent of large language models, prompt engineering now enables the rapid, low-effort creation of diverse autonomous agents that are already in widespread use. Yet this convenience raises urgent concerns about the safety, robustness, and behavioral consistency of the underlying prompts,...

Leaky Thoughts: Large Reasoning Models Are Not Private Thinkers

We study privacy leakage in the reasoning traces of large reasoning models used as personal agents. Unlike final outputs, reasoning traces are often assumed to be internal and safe. We challenge this assumption by showing that reasoning traces frequently contain sensitive user data, which can be...

Probe Before You Talk: Towards Black-Box Defense against Backdoor Unalignment for Large Language Models

Backdoor unalignment attacks against Large Language Models LLMs enable the stealthy compromise of safety alignment using a hidden trigger while evading normal safety auditing. These attacks pose significant threats to the applications of LLMs in the real-world Large Language Model as a Service...

AgentSmith Flaw in LangSmith’s Prompt Hub Exposed User API Keys, Data

A CVSS 8.8 AgentSmith flaw in LangSmith's Prompt Hub exposed AI agents to data theft and LLM manipulation. Learn how malicious AI agents could steal API keys and hijack LLM responses. Fix deployed...

Astra Linux – Vulnerability in Firefox

By first using the AI chatbot in one tab and then activating it in another tab, the document title from the previous tab would be leaked into the chat prompt. This vulnerability was fixed in Firefox 137...

The Safety Reminder: a Soft Prompt to Reactivate Delayed Safety Awareness in Vision-Language Models

As Vision-Language Models VLMs demonstrate increasing capabilities across real-world applications such as code generation and chatbot assistance, ensuring their safety has become paramount. Unlike traditional Large Language Models LLMs, VLMs face unique vulnerabilities due to their multimodal...

CVE-2025-49150

Cursor is a code editor built for programming with AI. Prior to 0.51.0, by default, the setting json.schemaDownload.enable was set to True. This means that by writing a JSON file, an attacker can trigger an arbitrary HTTP GET request that does not require user confirmation. Since the Cursor Agent...

AgentVigil: Generic Black-Box Red-Teaming for Indirect Prompt Injection against LLM Agents

The strong planning and reasoning capabilities of Large Language Models LLMs have fostered the development of agent-based systems capable of leveraging external tools and interacting with increasingly complex environments. However, these powerful features also introduce a critical security risk:...

LLMs Cannot Reliably Judge (Yet?): a Comprehensive Assessment on the Robustness of LLM-As-A-Judge

Large Language Models LLMs have demonstrated remarkable intelligence across various tasks, which has inspired the development and widespread adoption of LLM-as-a-Judge systems for automated model testing, such as red teaming and benchmarking. However, these systems are susceptible to adversarial...

LLMail-Inject: a Dataset from a Realistic Adaptive Prompt Injection Challenge

Indirect Prompt Injection attacks exploit the inherent limitation of Large Language Models LLMs to distinguish between instructions and data in their inputs. Despite numerous defense proposals, the systematic evaluation against adaptive adversaries remains limited, even when successful attacks ca...

Prompt Attacks Reveal Superficial Knowledge Removal in Unlearning Methods

In this work, we show that some machine unlearning methods may fail when subjected to straightforward prompt attacks. We systematically evaluate eight unlearning techniques across three model families, and employ output-based, logit-based, and probe analysis to determine to what extent supposedly...

Design Patterns for Securing LLM Agents against Prompt Injections

As AI agents powered by Large Language Models LLMs become increasingly versatile and capable of addressing a broad spectrum of tasks, ensuring their security has become a critical challenge. Among the most pressing threats are prompt injection attacks, which exploit the agent's resilience on...

DAVSP: Safety Alignment for Large Vision-Language Models Via Deep Aligned Visual Safety Prompt

Large Vision-Language Models LVLMs have achieved impressive progress across various applications but remain vulnerable to malicious queries that exploit the visual modality. Existing alignment approaches typically fail to resist malicious queries while preserving utility on benign ones effectivel...

Evaluation Empirique De La Sécurisation Et De L'Alignement De ChatGPT Et Gemini: Analyse Comparative Des Vulnérabilités Par Expérimentations De Jailbreaks

Large Language models LLMs are transforming digital usage, particularly in text generation, image creation, information retrieval and code development. ChatGPT, launched by OpenAI in November 2022, quickly became a reference, prompting the emergence of competitors such as Google's Gemini. However...

Securing Generative AI Agentic Workflows: Risks, Mitigation, and a Proposed Firewall Architecture

Generative Artificial Intelligence GenAI presents significant advancements but also introduces novel security challenges, particularly within agentic workflows where AI agents operate autonomously. These risks escalate in multi-agent systems due to increased interaction complexity. This paper...

CVE-2025-49619: Improper Neutralization of Special Elements Used in a Template Engine

Skyvern through 0.1.85 is vulnerable to server-side template injection SSTI in the Prompt field of workflow blocks such as the Navigation v2 Block. Improper sanitization of Jinja2 template input allows authenticated users to inject crafted expressions that are evaluated on the server, leading to...

CVE-2025-49619

Skyvern through 0.1.85 is vulnerable to server-side template injection SSTI in the Prompt field of workflow blocks such as the Navigation v2 Block. Improper sanitization of Jinja2 template input allows authenticated users to inject crafted expressions that are evaluated on the server, leading to...

Stealix: Model Stealing Via Prompt Evolution

Model stealing poses a significant security risk in machine learning by enabling attackers to replicate a black-box model without access to its training data, thus jeopardizing intellectual property and exposing sensitive information. Recent methods that use pre-trained diffusion models for data...

Sentinel: SOTA Model to Protect against Prompt Injections

Large Language Models LLMs are increasingly powerful but remain vulnerable to prompt injection attacks, where malicious inputs cause the model to deviate from its intended instructions. This paper introduces Sentinel, a novel detection model, qualifire/prompt-injection-sentinel, based on the...