SOFT: Selective Data Obfuscation for Protecting LLM Fine-Tuning against Membership Inference Attacks

Whitepaper called SOFT: Selective Data Obfuscation For Protecting LLM Fine-Tuning Against Membership Inference Attacks...

ObfusBFA: a Holistic Approach to Safeguarding DNNs from Different Types of Bit-Flip Attacks

Bit-flip attacks BFAs represent a serious threat to Deep Neural Networks DNNs, where flipping a small number of bits in the model parameters or binary code can significantly degrade the model accuracy or mislead the model prediction in a desired way. Existing defenses exclusively focus on...

Enabling Secure AI Inference: Trend Cybertron Leverages NVIDIA Universal LLM NIM Microservices

Learn how Trend's Cybertron has been harnessing the power of NVIDIA Universal LLM NIM Microservices...

ZTaint-Havoc: from Havoc Mode to Zero-Execution Fuzzing-Driven Taint Inference

Fuzzing is a widely used technique for discovering software vulnerabilities, but identifying hot bytes that influence program behavior remains challenging. Traditional taint analysis can track such bytes white-box, but suffers from scalability issue. Fuzzing-Driven Taint Inference FTI offers a...

Doxing Via the Lens: Revealing Location-Related Privacy Leakage on Multi-Modal Large Reasoning Models

Recent advances in multi-modal large reasoning models MLRMs have shown significant ability to interpret complex visual content. While these models enable impressive reasoning capabilities, they also introduce novel and underexplored privacy risks. In this paper, we identify a novel category of...

GradEscape: a Gradient-Based Evader against AI-Generated Text Detectors

In this paper, we introduce GradEscape, the first gradient-based evader designed to attack AI-generated text AIGT detectors. GradEscape overcomes the undifferentiable computation problem, caused by the discrete nature of text, by introducing a novel approach to construct weighted embeddings for t...

Saffron-1: Towards an Inference Scaling Paradigm for LLM Safety Assurance

Existing safety assurance research has primarily focused on training-phase alignment to instill safe behaviors into LLMs. However, recent studies have exposed these methods' susceptibility to diverse jailbreak attacks. Concurrently, inference scaling has significantly advanced LLM reasoning...

What Really Is a Member? Discrediting Membership Inference Via Poisoning

Membership inference tests aim to determine whether a particular data point was included in a language model's training set. However, recent works have shown that such tests often fail under the strict definition of membership based on exact matching, and have suggested relaxing this definition t...

Membership Inference Attacks for Unseen Classes

Shadow model attacks are the state-of-the-art approach for membership inference attacks on machine learning models. However, these attacks typically assume an adversary has access to a background nonmember data distribution that matches the distribution the target model was trained on. We initiat...

Breaking the Gaussian Barrier: Residual-PAC Privacy for Automatic Privatization

The Probably Approximately Correct PAC Privacy framework 1 provides a powerful instance-based methodology for certifying privacy in complex data-driven systems. However, existing PAC Privacy algorithms rely on a Gaussian mutual information upper bound. We show that this is in general too...

When Better Features Mean Greater Risks: the Performance-Privacy Trade-Off in Contrastive Learning

With the rapid advancement of deep learning technology, pre-trained encoder models have demonstrated exceptional feature extraction capabilities, playing a pivotal role in the research and application of deep learning. However, their widespread use has raised significant concerns about the risk o...

Membership Inference Attacks on Sequence Models

Sequence models, such as Large Language Models LLMs and autoregressive image generators, have a tendency to memorize and inadvertently leak sensitive information. While this tendency has critical legal implications, existing tools are insufficient to audit the resulting risks. We hypothesize that...

Watermarking Degrades Alignment in Language Models: Analysis and Mitigation

Watermarking techniques for large language models LLMs can significantly impact output quality, yet their effects on truthfulness, safety, and helpfulness remain critically underexamined. This paper presents a systematic analysis of how two popular watermarking approaches-Gumbel and KGW-affect...

Clustering and Median Aggregation Improve Differentially Private Inference

Differentially private DP language model inference is an approach for generating private synthetic text. A sensitive input example is used to prompt an off-the-shelf large language model LLM to produce a similar example. Multiple examples can be aggregated together to formally satisfy the DP...

Evaluating Apple Intelligence'S Writing Tools for Privacy against Large Language Model-Based Inference Attacks: Insights from Early Datasets

The misuse of Large Language Models LLMs to infer emotions from text for malicious purposes, known as emotion inference attacks, poses a significant threat to user privacy. In this paper, we investigate the potential of Apple Intelligence's writing tools, integrated across iPhone, iPad, and...

Keyed Chaotic Dynamics for Privacy-Preserving Neural Inference

Neural network inference typically operates on raw input data, increasing the risk of exposure during preprocessing and inference. Moreover, neural architectures lack efficient built-in mechanisms for directly authenticating input data. This work introduces a novel encryption method for ensuring...

Privacy Leaks by Adversaries: Adversarial Iterations for Membership Inference Attack

Membership inference attack MIA has become one of the most widely used and effective methods for evaluating the privacy risks of machine learning models. These attacks aim to determine whether a specific sample is part of the model's training set by analyzing the model's output. While traditional...

Denial Of Service (DoS)

vLLM is vulnerable to Denial of Service DoS. The vulnerability is due to improper input validation that accepts unexpected or malformed pattern and type fields in tool-related requests, which can crash the inference worker...

CSVAR: Enhancing Visual Privacy in Federated Learning Via Adaptive Shuffling against Overfitting

Although federated learning preserves training data within local privacy domains, the aggregated model parameters may still reveal private characteristics. This vulnerability stems from clients' limited training data, which predisposes models to overfitting. Such overfitting enables models to...

Unlearning Inversion Attacks for Graph Neural Networks

Graph unlearning methods aim to efficiently remove the impact of sensitive data from trained GNNs without full retraining, assuming that deleted information cannot be recovered. In this work, we challenge this assumption by introducing the graph unlearning inversion attack: given only black-box...