Urania: Differentially Private Insights into AI Use

We introduce $Urania$, a novel framework for generating insights about LLM chatbot interactions with rigorous differential privacy DP guarantees. The framework employs a private clustering mechanism and innovative keyword extraction methods, including frequency-based, TF-IDF-based, and LLM-guided...

Big Bird: Privacy Budget Management for W3C'S Privacy-Preserving Attribution API

Privacy-preserving advertising APIs like Privacy-Preserving Attribution PPA are designed to enhance web privacy while enabling effective ad measurement. PPA offers an alternative to cross-site tracking with encrypted reports governed by differential privacy DP, but current designs lack a principl...

Membership Inference Attacks on Sequence Models

Sequence models, such as Large Language Models LLMs and autoregressive image generators, have a tendency to memorize and inadvertently leak sensitive information. While this tendency has critical legal implications, existing tools are insufficient to audit the resulting risks. We hypothesize that...

QA-HFL: Quality-Aware Hierarchical Federated Learning for Resource-Constrained Mobile Devices with Heterogeneous Image Quality

This paper introduces QA-HFL, a quality-aware hierarchical federated learning framework that efficiently handles heterogeneous image quality across resource-constrained mobile devices. Our approach trains specialized local models for different image quality levels and aggregates their features...

Learning to Diagnose Privately: DP-Powered LLMs for Radiology Report Classification

Purpose: This study proposes a framework for fine-tuning large language models LLMs with differential privacy DP to perform multi-abnormality classification on radiology report text. By injecting calibrated noise during fine-tuning, the framework seeks to mitigate the privacy risks associated wit...

PCEvolve: Private Contrastive Evolution for Synthetic Dataset Generation Via Few-Shot Private Data and Generative APIs

The rise of generative APIs has fueled interest in privacy-preserving synthetic data generation. While the Private Evolution PE algorithm generates Differential Privacy DP synthetic images using diffusion model APIs, it struggles with few-shot private data due to the limitations of its DP-protect...

Dropout-Robust Mechanisms for Differentially Private and Fully Decentralized Mean Estimation

Achieving differentially private computations in decentralized settings poses significant challenges, particularly regarding accuracy, communication cost, and robustness against information leakage. While cryptographic solutions offer promise, they often suffer from high communication overhead or...

Privacy and Security Threat for OpenAI GPTs

Large language models LLMs demonstrate powerful information handling capabilities and are widely integrated into chatbot applications. OpenAI provides a platform for developers to construct custom GPTs, extending ChatGPT's functions and integrating external services. Since its release in November...

Clustering and Median Aggregation Improve Differentially Private Inference

Differentially private DP language model inference is an approach for generating private synthetic text. A sensitive input example is used to prompt an off-the-shelf large language model LLM to produce a similar example. Multiple examples can be aggregated together to formally satisfy the DP...

Gradient Inversion Attacks on Parameter-Efficient Fine-Tuning

Federated learning FL allows multiple data-owners to collaboratively train machine learning models by exchanging local gradients, while keeping their private data on-device. To simultaneously enhance privacy and training efficiency, recently parameter-efficient fine-tuning PEFT of large-scale...

FERRET: Private Deep Learning Faster and Better Than DPSGD

We revisit 1-bit gradient compression through the lens of mutual-information differential privacy MI-DP. Building on signSGD, we propose FERRET--Fast and Effective Restricted Release for Ethical Training--which transmits at most one sign bit per parameter group with Bernoulli masking. Theory: We...

Client-Side Zero-Shot LLM Inference for Comprehensive In-Browser URL Analysis

Malicious websites and phishing URLs pose an ever-increasing cybersecurity risk, with phishing attacks growing by 40% in a single year. Traditional detection approaches rely on machine learning classifiers or rule-based scanners operating in the cloud, but these face significant challenges in...

Towards Trustworthy Federated Learning with Untrusted Participants

Resilience against malicious participants and data privacy are essential for trustworthy federated learning, yet achieving both with good utility typically requires the strong assumption of a trusted central server. This paper shows that a significantly weaker assumption suffices: each pair of...

Evaluating Apple Intelligence'S Writing Tools for Privacy against Large Language Model-Based Inference Attacks: Insights from Early Datasets

The misuse of Large Language Models LLMs to infer emotions from text for malicious purposes, known as emotion inference attacks, poses a significant threat to user privacy. In this paper, we investigate the potential of Apple Intelligence's writing tools, integrated across iPhone, iPad, and...

Improper Access Control

github.com/mattermost/mattermost-server is vulnerable to improper access control. The vulnerability is due to insufficient permission checks when changing team privacy settings, allowing unauthorized team administrators to access and modify team invite IDs via the /api/v4/teams/:teamId/privacy...

An Algorithmic Pipeline for GDPR-Compliant Healthcare Data Anonymisation: Moving toward Standardisation

High-quality real-world data RWD is essential for healthcare but must be transformed to comply with the General Data Protection Regulation GDPR. GDPRs broad definitions of quasi-identifiers QIDs and sensitive attributes SAs complicate implementation. We aim to standardise RWD anonymisation for GD...

Secure and Private Federated Learning: Achieving Adversarial Resilience through Robust Aggregation

Federated Learning FL enables collaborative machine learning across decentralized data sources without sharing raw data. It offers a promising approach to privacy-preserving AI. However, FL remains vulnerable to adversarial threats from malicious participants, referred to as Byzantine clients, wh...

Keyed Chaotic Dynamics for Privacy-Preserving Neural Inference

Neural network inference typically operates on raw input data, increasing the risk of exposure during preprocessing and inference. Moreover, neural architectures lack efficient built-in mechanisms for directly authenticating input data. This work introduces a novel encryption method for ensuring...

Privacy Leaks by Adversaries: Adversarial Iterations for Membership Inference Attack

Membership inference attack MIA has become one of the most widely used and effective methods for evaluating the privacy risks of machine learning models. These attacks aim to determine whether a specific sample is part of the model's training set by analyzing the model's output. While traditional...

Differentially Private Distribution Release of Gaussian Mixture Models Via KL-Divergence Minimization

Gaussian Mixture Models GMMs are widely used statistical models for representing multi-modal data distributions, with numerous applications in data mining, pattern recognition, data simulation, and machine learning. However, recent research has shown that releasing GMM parameters poses significan...