42 matches found
SOFT: Selective Data Obfuscation for Protecting LLM Fine-Tuning against Membership Inference Attacks
Whitepaper called SOFT: Selective Data Obfuscation For Protecting LLM Fine-Tuning Against Membership Inference Attacks...
What Really Is a Member? Discrediting Membership Inference Via Poisoning
Membership inference tests aim to determine whether a particular data point was included in a language model's training set. However, recent works have shown that such tests often fail under the strict definition of membership based on exact matching, and have suggested relaxing this definition t...
When Better Features Mean Greater Risks: the Performance-Privacy Trade-Off in Contrastive Learning
With the rapid advancement of deep learning technology, pre-trained encoder models have demonstrated exceptional feature extraction capabilities, playing a pivotal role in the research and application of deep learning. However, their widespread use has raised significant concerns about the risk o...
Membership Inference Attacks for Unseen Classes
Shadow model attacks are the state-of-the-art approach for membership inference attacks on machine learning models. However, these attacks typically assume an adversary has access to a background nonmember data distribution that matches the distribution the target model was trained on. We initiat...
Membership Inference Attacks on Sequence Models
Sequence models, such as Large Language Models LLMs and autoregressive image generators, have a tendency to memorize and inadvertently leak sensitive information. While this tendency has critical legal implications, existing tools are insufficient to audit the resulting risks. We hypothesize that...
Privacy Leaks by Adversaries: Adversarial Iterations for Membership Inference Attack
Membership inference attack MIA has become one of the most widely used and effective methods for evaluating the privacy risks of machine learning models. These attacks aim to determine whether a specific sample is part of the model's training set by analyzing the model's output. While traditional...
CSVAR: Enhancing Visual Privacy in Federated Learning Via Adaptive Shuffling against Overfitting
Although federated learning preserves training data within local privacy domains, the aggregated model parameters may still reveal private characteristics. This vulnerability stems from clients' limited training data, which predisposes models to overfitting. Such overfitting enables models to...
Unlearning Inversion Attacks for Graph Neural Networks
Graph unlearning methods aim to efficiently remove the impact of sensitive data from trained GNNs without full retraining, assuming that deleted information cannot be recovered. In this work, we challenge this assumption by introducing the graph unlearning inversion attack: given only black-box...
Bayesian Perspective on Memorization and Reconstruction
We introduce a new Bayesian perspective on the concept of data reconstruction, and leverage this viewpoint to propose a new security definition that, in certain settings, provably prevents reconstruction attacks. We use our paradigm to shed new light on one of the most notorious attacks in the...
Practical Bayes-Optimal Membership Inference Attacks
We develop practical and theoretically grounded membership inference attacks MIAs against both independent and identically distributed i.i.d. data and graph-structured data. Building on the Bayesian decision-theoretic framework of Sablayrolles et al., we derive the Bayes-optimal membership...
TensorShield: Safeguarding On-Device Inference by Shielding Critical DNN Tensors with TEE
To safeguard user data privacy, on-device inference has emerged as a prominent paradigm on mobile and Internet of Things IoT devices. This paradigm involves deploying a model provided by a third party on local devices to perform inference tasks. However, it exposes the private model to two primar...
Unveiling Impact of Frequency Components on Membership Inference Attacks for Diffusion Models
Diffusion models have achieved tremendous success in image generation, but they also raise significant concerns regarding privacy and copyright issues. Membership Inference Attacks MIAs are designed to ascertain whether specific data were utilized during a model's training phase. As current MIAs...
Strong Membership Inference Attacks on Massive Datasets and (Moderately) Large Language Models
State-of-the-art membership inference attacks MIAs typically require training many reference models, making it difficult to scale these attacks to large pre-trained language models LLMs. As a result, prior research has either relied on weaker attacks that avoid training reference models e.g.,...
On Membership Inference Attacks in Knowledge Distillation
Nowadays, Large Language Models LLMs are trained on huge datasets, some including sensitive information. This poses a serious privacy concern because privacy attacks such as Membership Inference Attacks MIAs may detect this sensitive information. While knowledge distillation compresses LLMs into...
A Survey on Privacy Risks and Protection in Large Language Models
Although Large Language Models LLMs have become increasingly integral to diverse applications, their capabilities raise significant privacy concerns. This survey offers a comprehensive overview of privacy risks associated with LLMs and examines current solutions to mitigate these challenges. Firs...
The DCR Delusion: Measuring the Privacy Risk of Synthetic Data
Synthetic data has become an increasingly popular way to share data without revealing sensitive information. Though Membership Inference Attacks MIAs are widely considered the gold standard for empirically assessing the privacy of a synthetic dataset, practitioners and researchers often rely on...
Can Differentially Private Fine-Tuning LLMs Protect against Privacy Attacks?
Fine-tuning large language models LLMs has become an essential strategy for adapting them to specialized tasks; however, this process introduces significant privacy challenges, as sensitive training data may be inadvertently memorized and exposed. Although differential privacy DP offers strong...
Revisiting Data Auditing in Large Vision-Language Models
With the surge of large language models LLMs, Large Vision-Language Models VLMs--which integrate vision encoders with LLMs for accurate visual grounding--have shown great potential in tasks like generalist agents and robotic control. However, VLMs are typically trained on massive web-scraped...
DeSIA: Attribute Inference Attacks against Limited Fixed Aggregate Statistics
Empirical inference attacks are a popular approach for evaluating the privacy risk of data release mechanisms in practice. While an active attack literature exists to evaluate machine learning models or synthetic data release, we currently lack comparable methods for fixed aggregate statistics, i...
