30 matches found
ai-kill-chain
Extended Cyber Kill Chain for AI-Era Threats An update to the Lockheed Martin Cyber Kill Chain for defenders working against LLM and agentic AI attacks. Adds a pre-attack stage for model supply chain compromise. Adds AI-specific sub-techniques to each of the original seven stages. Splits the...
MEA-Bench
MEA-Bench: A Benchmark for Model Extraction Attacks This repository provides a unified benchmark for model extraction attacks, defenses, adaptive attacks, and evaluation. The public interface is organized around a small number of portable commands. Method-specific Python modules and legacy run...
offensive-ai-compilation
Offensive AI Compilation A curated list of useful resources that cover Offensive AI. 📁 Contents 📁 🚫 Abuse 🚫 🧠 Adversarial Machine Learning 🧠 ⚡ Attacks ⚡ 🔒 Extraction 🔒 ⚠️ Limitations ⚠️ 🛡️ Defensive actions 🛡️ 🔗 Useful links 🔗 ⬅️ Inversion or inference ⬅️ 🛡️ Defensive actions 🛡️ 🔗 Useful links 🔗 💉...
GraphIP-Bench
GraphIP-Bench How Hard Is It to Steal a Graph Neural Network, and Can We Stop It? A unified benchmark and library for evaluating model-extraction attacks and ownership defenses on graph neural networks under a single reproducible black-box protocol. Why GraphIP-Bench · Highlights · Quick Start ·...
ai-security-battle
🤖 AI Security Battle System Red Team vs Blue Team: Adversarial ML Combat Simulation A comprehensive adversarial machine learning security simulation where Red Team attackers battle against Blue Team defenders in a 30-minute AI/ML security warfare scenario. 🎯 Project Overview This system simulates...
CVE-2024-45436
Ollama CVE-2024-45436 Exploit A clean and efficient exploit implementation for the Ollama ZIP traversal vulnerability CVE-2024-45436. 中文文档:READMECN.md Vulnerability Description CVE-2024-45436 is a path traversal vulnerability also known as "Zip Slip" in Ollama versions prior to 0.1.47. The...
PrivacyRaven
Note: This project is on hiatus. PrivacyRaven is a privacy testing library for deep learning systems. You can use it to determine the susceptibility of a model to different privacy attacks; evaluate privacy preserving machine learning techniques; develop novel privacy metrics and attacks; and...
Trustworthy Distributed Machine Learning in Cloud-Edge-IoT Systems: A Heterogeneity-Aware Survey of Security, Privacy, and Robustness
Distributed machine learning DML, including conventional distributed training and federated learning FL, enables collaborative model development across cloud-edge-IoT systems without fully centralizing raw data. However, distributing data, computation, updates, and orchestration increases the...
The Limits of Query Monitoring against Distributed Model Extraction
Model providers now face distillation campaigns run through thousands of accounts, each of which looks like an ordinary user. When network and payment metadata cannot link those accounts, the only evidence left is what they ask. We study what a provider can and cannot learn from those queries...
Hidden Risks of Jev: An Empirical Study of Security, Privacy, and Dual Use
Jev turns natural-language questions into typed answers and probabilities with low latency and cost, enabling applications to route requests and select tools. While this interface allows Jev to integrate naturally into application workflows as a decision layer, the security and privacy implicatio...
Do Defenses against LLM Extraction Work across Attacks? A Lifecycle Benchmark of Black-Box Model Extraction
Large language models LLMs deployed through text-only APIs face model extraction risks, as adversaries can collect their responses to train surrogates that reproduce their capabilities. While prior work has developed diverse attacks and defenses, evaluations remain fragmented across access...
AI Model Extraction Attacks: Bypassing Single-Client Assumptions in Defenses
Ensuring the protection of Artificial Intelligence AI models deployed in military Command and Control C2 systems and critical infrastructure is essential for maintaining information superiority. Model Extraction Attacks MEAs pose a significant threat, as they enable adversaries to replicate...
MAI-2025-0016
Large Language Models LLMs integrated into hate speech detection systems are susceptible to adversarial attacks and model extraction vulnerabilities. Adversarial attacks involve the strategic alteration of hate speech text to evade detection mechanisms, while model extraction facilitates the...
Anthropic Claims Chinese AI Firms ‘Distilled’ Claude to Train Their Models
Anthropic claims Chinese AI firms distilled Claude to train rival AI models, raising concerns about model extraction, security risks, and AI distillation abuse...
Breaking SafetyCore: Exploring the Risks of On-Device AI Deployment
Due to hardware and software improvements, an increasing number of AI models are deployed on-device. This shift enhances privacy and reduces latency, but also introduces security risks distinct from traditional software. In this article, we examine these risks through the real-world case study of...
Measuring the Vulnerability Disclosure Policies of AI Vendors
As AI is increasingly integrated into products and critical systems, researchers are paying greater attention to identifying related vulnerabilities. Effective remediation depends on whether vendors are willing to accept and respond to AI vulnerability reports. In this paper, we examine the...
LLM4MEA: Data-Free Model Extraction Attacks on Sequential Recommenders Via Large Language Models
Recent studies have demonstrated the vulnerability of sequential recommender systems to Model Extraction Attacks MEAs. MEAs collect responses from recommender systems to replicate their functionality, enabling unauthorized deployments and posing critical privacy and security risks. Black-box...
Entangled Threats: a Unified Kill Chain Model for Quantum Machine Learning Security
Quantum Machine Learning QML systems inherit vulnerabilities from classical machine learning while introducing new attack surfaces rooted in the physical and algorithmic layers of quantum computing. Despite a growing body of research on individual attack vectors - ranging from adversarial poisoni...
BarkBeetle: Stealing Decision Tree Models with Fault Injection
Machine learning models, particularly decision trees DTs, are widely adopted across various domains due to their interpretability and efficiency. However, as ML models become increasingly integrated into privacy-sensitive applications, concerns about their confidentiality have grown, particularly...
CEGA: a Cost-Effective Approach for Graph-Based Model Extraction and Acquisition
Graph Neural Networks GNNs have demonstrated remarkable utility across diverse applications, and their growing complexity has made Machine Learning as a Service MLaaS a viable platform for scalable deployment. However, this accessibility also exposes GNN to serious security threats, most notably...