Model Inversion Attacks Meet Cryptographic Fuzzy Extractors

Model inversion attacks pose an open challenge to privacy-sensitive applications that use machine learning ML models. For example, face authentication systems use modern ML models to compute embedding vectors from face images of the enrolled users and store them. If leaked, inversion attacks can...

Split Happens: Combating Advanced Threats with Split Learning and Function Secret Sharing

Split Learning SL -- splits a model into two distinct parts to help protect client data while enhancing Machine Learning ML processes. Though promising, SL has proven vulnerable to different attacks, thus raising concerns about how effective it may be in terms of data privacy. Recent works have...

Model Inversion Attacks on Llama 3: Extracting PII from Large Language Models

Large language models LLMs have transformed natural language processing, but their ability to memorize training data poses significant privacy risks. This paper investigates model inversion attacks on the Llama 3.2 model, a multilingual LLM developed by Meta. By querying the model with carefully...

Diffusion-Based Task-Oriented Semantic Communications with Model Inversion Attack

Semantic communication has emerged as a promising neural network-based system design for 6G networks. Task-oriented semantic communication is a novel paradigm whose core goal is to efficiently complete specific tasks by transmitting semantic information, optimizing communication efficiency and ta...

AIRTBench: Measuring Autonomous AI Red Teaming Capabilities in Language Models

We introduce AIRTBench, an AI red teaming benchmark for evaluating language models' ability to autonomously discover and exploit Artificial Intelligence and Machine Learning AI/ML security vulnerabilities. The benchmark consists of 70 realistic black-box capture-the-flag CTF challenges from the...

DiffUMI: Training-Free Universal Model Inversion Via Unconditional Diffusion for Face Recognition

Face recognition technology presents serious privacy risks due to its reliance on sensitive and immutable biometric data. To address these concerns, such systems typically convert raw facial images into embeddings, which are traditionally viewed as privacy-preserving. However, model inversion...

A Survey on Privacy Risks and Protection in Large Language Models

Although Large Language Models LLMs have become increasingly integral to diverse applications, their capabilities raise significant privacy concerns. This survey offers a comprehensive overview of privacy risks associated with LLMs and examines current solutions to mitigate these challenges. Firs...

Being Naughty to See Who Was Nice: Machine Learning Attacks on Santa’s List

Editor’s note: We had planned to publish our Hacky Holidays blog series throughout December 2021 – but then Log4Shell happened, and we dropped everything to focus on this major vulnerability that impacted the entire cybersecurity community worldwide. Now that it’s 2022, we’re feeling in need of...

New Framework Released to Protect Machine Learning Systems From Adversarial Attacks

Microsoft, in collaboration with MITRE, IBM, NVIDIA, and Bosch, has released a new open framework that aims to help security analysts detect, respond to, and remediate adversarial attacks against machine learning ML systems. Called the Adversarial ML Threat Matrix, the initiative is an attempt to...