Diverging Towards Hallucination: Detection of Failures in Vision-Language Models Via Multi-Token Aggregation

Vision-language models VLMs now rival human performance on many multimodal tasks, yet they still hallucinate objects or generate unsafe text. Current hallucination detectors, e.g., single-token linear probing SLP and PTrue, typically analyze only the logit of the first generated token or just its...

MPMA: Preference Manipulation Attack against Model Context Protocol

Model Context Protocol MCP standardizes interface mapping for large language models LLMs to access external data and tools, which revolutionizes the paradigm of tool selection and facilitates the rapid expansion of the LLM agent tool ecosystem. However, as the MCP is increasingly adopted,...

ProxyPrompt: Securing System Prompts against Prompt Extraction Attacks

The integration of large language models LLMs into a wide range of applications has highlighted the critical role of well-crafted system prompts, which require extensive testing and domain expertise. These prompts enhance task performance but may also encode sensitive information and filtering...

S3C2 Summit 2024-09: Industry Secure Software Supply Chain Summit

While providing economic and software development value, software supply chains are only as strong as their weakest link. Over the past several years, there has been an exponential increase in cyberattacks, specifically targeting vulnerable links in critical software supply chains. These attacks...

On Technique Identification and Threat-Actor Attribution Using LLMs and Embedding Models

Attribution of cyber-attacks remains a complex but critical challenge for cyber defenders. Currently, manual extraction of behavioral indicators from dense forensic documentation causes significant attribution delays, especially following major incidents at the international scale. This research...

SafeTrans: LLM-Assisted Transpilation from C to Rust

Rust is a strong contender for a memory-safe alternative to C as a "systems" programming language, but porting the vast amount of existing C code to Rust is a daunting task. In this paper, we evaluate the potential of large language models LLMs to automate the transpilation of C code to idiomatic...

Cape: Context-Aware Prompt Perturbation Mechanism with Differential Privacy

Large Language Models LLMs have gained significant popularity due to their remarkable capabilities in text understanding and generation. However, despite their widespread deployment in inference services such as ChatGPT, concerns about the potential leakage of sensitive user data have arisen...

Adversarial Attack on Large Language Models Using Exponentiated Gradient Descent

As Large Language Models LLMs are widely used, understanding them systematically is key to improving their safety and realizing their full potential. Although many models are aligned using techniques such as reinforcement learning from human feedback RLHF, they are still vulnerable to jailbreakin...

Red Teaming the Mind of the Machine: a Systematic Evaluation of Prompt Injection and Jailbreak Vulnerabilities in LLMs

Large Language Models LLMs are increasingly integrated into consumer and enterprise applications. Despite their capabilities, they remain susceptible to adversarial attacks such as prompt injection and jailbreaks that override alignment safeguards. This paper provides a systematic investigation o...

LM-Scout: Analyzing the Security of Language Model Integration in Android Apps

Developers are increasingly integrating Language Models LMs into their mobile apps to provide features such as chat-based assistants. To prevent LM misuse, they impose various restrictions, including limits on the number of queries, input length, and allowed topics. However, if the LM integration...

LiteLMGuard: Seamless and Lightweight On-Device Prompt Filtering for Safeguarding Small Language Models against Quantization-Induced Risks and Vulnerabilities

The growing adoption of Large Language Models LLMs has influenced the development of their lighter counterparts-Small Language Models SLMs-to enable on-device deployment across smartphones and edge devices. These SLMs offer enhanced privacy, reduced latency, server-free functionality, and improve...

Federated Large Language Models: Feasibility, Robustness, Security and Future Directions

The integration of Large Language Models LLMs and Federated Learning FL presents a promising solution for joint training on distributed data while preserving privacy and addressing data silo issues. However, this emerging field, known as Federated Large Language Models FLLM, faces significant...

POISONCRAFT: Practical Poisoning of Retrieval-Augmented Generation for Large Language Models

Large language models LLMs have achieved remarkable success in various domains, primarily due to their strong capabilities in reasoning and generating human-like text. Despite their impressive performance, LLMs are susceptible to hallucinations, which can lead to incorrect or misleading outputs...

System Prompt Poisoning: Persistent Attacks on Large Language Models beyond User Injection

Large language models LLMs have gained widespread adoption across diverse applications due to their impressive generative capabilities. Their plug-and-play nature enables both developers and end users to interact with these models through simple prompts. However, as LLMs become more integrated in...

CVE-2025-30165

A flaw was found in vLLM's multi-node configuration, which is vulnerable to remote code execution due to unsafe deserialization using pickle over a ZeroMQ SUB socket. If the primary vLLM host is compromised, attackers can escalate privileges and execute arbitrary code on connected secondary hosts...

FedTDP: a Privacy-Preserving and Unified Framework for Trajectory Data Preparation Via Federated Learning

Trajectory data, which capture the movement patterns of people and vehicles over time and space, are crucial for applications like traffic optimization and urban planning. However, issues such as noise and incompleteness often compromise data quality, leading to inaccurate trajectory analyses and...

Safeguard-By-Development: a Privacy-Enhanced Development Paradigm for Multi-Agent Collaboration Systems

Multi-agent collaboration systems MACS, powered by large language models LLMs, solve complex problems efficiently by leveraging each agent's specialization and communication between agents. However, the inherent exchange of information between agents and their interaction with external...

NVIDIA TensorRT-LLM python executor code issue vulnerability

NVIDIA TensorRT-LLM is a high-performance inference acceleration library from NVIDIA for defining, optimizing, and executing inference in production environments for large language models LLMs. A code issue vulnerability exists in NVIDIA TensorRT-LLM that stems from insufficient data validation a...

OBLIVIATE: Robust and Practical Machine Unlearning for Large Language Models

Large language models LLMs trained over extensive corpora risk memorizing sensitive, copyrighted, or toxic content. To address this, we propose OBLIVIATE, a robust unlearning framework that removes targeted data while preserving model utility. The framework follows a structured process: extractin...

Weaponizing Language Models for Cybersecurity Offensive Operations: Automating Vulnerability Assessment Report Validation; a Review Paper

This, with the ever-increasing sophistication of cyberwar, calls for novel solutions. In this regard, Large Language Models LLMs have emerged as a highly promising tool for defensive and offensive cybersecurity-related strategies. While existing literature has focused much on the defensive use of...