A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment

The remarkable success of Large Language Models LLMs has illuminated a promising pathway toward achieving Artificial General Intelligence for both academic and industrial communities, owing to their unprecedented performance across various applications. As LLMs continue to gain prominence in both...

Efficient RL-Based Cache Vulnerability Exploration by Penalizing Useless Agent Actions

Cache-timing attacks exploit microarchitectural characteristics to leak sensitive data, posing a severe threat to modern systems. Despite its severity, analyzing the vulnerability of a given cache structure against cache-timing attacks is challenging. To this end, a method based on Reinforcement...

D2R: Dual Regularization Loss with Collaborative Adversarial Generation for Model Robustness

The robustness of Deep Neural Network models is crucial for defending models against adversarial attacks. Recent defense methods have employed collaborative learning frameworks to enhance model robustness. Two key limitations of existing methods are i insufficient guidance of the target model via...

From Threat to Tool: Leveraging Refusal-Aware Injection Attacks for Safety Alignment

Safely aligning large language models LLMs often demands extensive human-labeled preference data, a process that's both costly and time-consuming. While synthetic data offers a promising alternative, current methods frequently rely on complex iterative prompting or auxiliary models. To address...

Can In-Context Reinforcement Learning Recover from Reward Poisoning Attacks?

We study the corruption-robustness of in-context reinforcement learning ICRL, focusing on the Decision-Pretrained Transformer DPT, Lee et al., 2023. To address the challenge of reward poisoning attacks targeting the DPT, we propose a novel adversarial training framework, called Adversarially...

What Really Is a Member? Discrediting Membership Inference Via Poisoning

Membership inference tests aim to determine whether a particular data point was included in a language model's training set. However, recent works have shown that such tests often fail under the strict definition of membership based on exact matching, and have suggested relaxing this definition t...

Saffron-1: Towards an Inference Scaling Paradigm for LLM Safety Assurance

Existing safety assurance research has primarily focused on training-phase alignment to instill safe behaviors into LLMs. However, recent studies have exposed these methods' susceptibility to diverse jailbreak attacks. Concurrently, inference scaling has significantly advanced LLM reasoning...

Identifying and Understanding Cross-Class Features in Adversarial Training

Adversarial training AT has been considered one of the most effective methods for making deep neural networks robust against adversarial attacks, while the training mechanisms and dynamics of AT remain open research problems. In this paper, we present a novel perspective on studying AT through th...

VLMs Can Aggregate Scattered Training Patches

Whitepaper called VLMs Can Aggregate Scattered Training Patches...

Prediction Inconsistency Helps Achieve Generalizable Detection of Adversarial Examples

Adversarial detection protects models from adversarial attacks by refusing suspicious test samples. However, current detection methods often suffer from weak generalization: their effectiveness tends to degrade significantly when applied to adversarially trained models rather than naturally train...

FERRET: Private Deep Learning Faster and Better Than DPSGD

We revisit 1-bit gradient compression through the lens of mutual-information differential privacy MI-DP. Building on signSGD, we propose FERRET--Fast and Effective Restricted Release for Ethical Training--which transmits at most one sign bit per parameter group with Bernoulli masking. Theory: We...

Robust Anti-Backdoor Instruction Tuning in LVLMs

Large visual language models LVLMs have demonstrated excellent instruction-following capabilities, yet remain vulnerable to stealthy backdoor attacks when finetuned using contaminated data. Existing backdoor defense techniques are usually developed for single-modal visual or language models under...

Mind the Gap: a Practical Attack on GGUF Quantization

With the increasing size of frontier LLMs, post-training quantization has become the standard for memory-efficient deployment. Recent work has shown that basic rounding-based quantization schemes pose security risks, as they can be exploited to inject malicious behaviors into quantized models tha...

Which Factors Make Code LLMs More Vulnerable to Backdoor Attacks? A Systematic Study

Code LLMs are increasingly employed in software development. However, studies have shown that they are vulnerable to backdoor attacks: when a trigger a specific input pattern appears in the input, the backdoor will be activated and cause the model to generate malicious outputs. Researchers have...

Security Concerns for Large Language Models: a Survey

Large Language Models LLMs such as GPT-4 and its recent iterations, Google's Gemini, Anthropic's Claude 3 models, and xAI's Grok have caused a revolution in natural language processing, but their capabilities also introduce new security vulnerabilities. In this survey, we provide a comprehensive...

Adversarial Machine Learning for Robust Password Strength Estimation

Passwords remain one of the most common methods for securing sensitive data in the digital age. However, weak password choices continue to pose significant risks to data security and privacy. This study aims to solve the problem by focusing on developing robust password strength estimation models...

Towards Secure MLOps: Surveying Attacks, Mitigation Strategies, and Research Challenges

The rapid adoption of machine learning ML technologies has driven organizations across diverse sectors to seek efficient and reliable methods to accelerate model development-to-deployment. Machine Learning Operations MLOps has emerged as an integrative approach addressing these requirements by...

Hush! Protecting Secrets during Model Training: an Indistinguishability Approach

We consider the problem of secret protection, in which a business or organization wishes to train a model on their own data, while attempting to not leak secrets potentially contained in that data via the model. The standard method for training models to avoid memorization of secret information i...

Adversarial Threat Vectors and Risk Mitigation for Retrieval-Augmented Generation Systems

Retrieval-Augmented Generation RAG systems, which integrate Large Language Models LLMs with external knowledge sources, are vulnerable to a range of adversarial attack vectors. This paper examines the importance of RAG systems through recent industry adoption trends and identifies the prominent...

CVE-2025-5171

A vulnerability, which was classified as critical, has been found in llisoft MTA Maita Training System 4.5. This issue affects the function this.fileService.download of the file com\llisoft\controller\OpenController.java. The manipulation of the argument url leads to unrestricted upload. The atta...