offensive-ai-compilation

Offensive AI Compilation A curated list of useful resources that cover Offensive AI. 📁 Contents 📁 🚫 Abuse 🚫 🧠 Adversarial Machine Learning 🧠 ⚡ Attacks ⚡ 🔒 Extraction 🔒 ⚠️ Limitations ⚠️ 🛡️ Defensive actions 🛡️ 🔗 Useful links 🔗 ⬅️ Inversion or inference ⬅️ 🛡️ Defensive actions 🛡️ 🔗 Useful links 🔗 💉...

CVE-2024-45436

Ollama CVE-2024-45436 Exploit A clean and efficient exploit implementation for the Ollama ZIP traversal vulnerability CVE-2024-45436. 中文文档：READMECN.md Vulnerability Description CVE-2024-45436 is a path traversal vulnerability also known as "Zip Slip" in Ollama versions prior to 0.1.47. The...

PrivacyRaven

Note: This project is on hiatus. PrivacyRaven is a privacy testing library for deep learning systems. You can use it to determine the susceptibility of a model to different privacy attacks; evaluate privacy preserving machine learning techniques; develop novel privacy metrics and attacks; and...

ai-kill-chain

Extended Cyber Kill Chain for AI-Era Threats An update to the Lockheed Martin Cyber Kill Chain for defenders working against LLM and agentic AI attacks. Adds a pre-attack stage for model supply chain compromise. Adds AI-specific sub-techniques to each of the original seven stages. Splits the...

ai-security-battle

🤖 AI Security Battle System Red Team vs Blue Team: Adversarial ML Combat Simulation A comprehensive adversarial machine learning security simulation where Red Team attackers battle against Blue Team defenders in a 30-minute AI/ML security warfare scenario. 🎯 Project Overview This system simulates...

MEA-Bench

MEA-Bench: A Benchmark for Model Extraction Attacks This repository provides a unified benchmark for model extraction attacks, defenses, adaptive attacks, and evaluation. The public interface is organized around a small number of portable commands. Method-specific Python modules and legacy run...

Hidden Risks of Jev: An Empirical Study of Security, Privacy, and Dual Use

Jev turns natural-language questions into typed answers and probabilities with low latency and cost, enabling applications to route requests and select tools. While this interface allows Jev to integrate naturally into application workflows as a decision layer, the security and privacy implicatio...

Do Defenses against LLM Extraction Work across Attacks? A Lifecycle Benchmark of Black-Box Model Extraction

Large language models LLMs deployed through text-only APIs face model extraction risks, as adversaries can collect their responses to train surrogates that reproduce their capabilities. While prior work has developed diverse attacks and defenses, evaluations remain fragmented across access...

AI Model Extraction Attacks: Bypassing Single-Client Assumptions in Defenses

Ensuring the protection of Artificial Intelligence AI models deployed in military Command and Control C2 systems and critical infrastructure is essential for maintaining information superiority. Model Extraction Attacks MEAs pose a significant threat, as they enable adversaries to replicate...

Anthropic Claims Chinese AI Firms ‘Distilled’ Claude to Train Their Models

Anthropic claims Chinese AI firms distilled Claude to train rival AI models, raising concerns about model extraction, security risks, and AI distillation abuse...

Breaking SafetyCore: Exploring the Risks of On-Device AI Deployment

Due to hardware and software improvements, an increasing number of AI models are deployed on-device. This shift enhances privacy and reduces latency, but also introduces security risks distinct from traditional software. In this article, we examine these risks through the real-world case study of...

Measuring the Vulnerability Disclosure Policies of AI Vendors

As AI is increasingly integrated into products and critical systems, researchers are paying greater attention to identifying related vulnerabilities. Effective remediation depends on whether vendors are willing to accept and respond to AI vulnerability reports. In this paper, we examine the...

LLM4MEA: Data-Free Model Extraction Attacks on Sequential Recommenders Via Large Language Models

Recent studies have demonstrated the vulnerability of sequential recommender systems to Model Extraction Attacks MEAs. MEAs collect responses from recommender systems to replicate their functionality, enabling unauthorized deployments and posing critical privacy and security risks. Black-box...

Entangled Threats: a Unified Kill Chain Model for Quantum Machine Learning Security

Quantum Machine Learning QML systems inherit vulnerabilities from classical machine learning while introducing new attack surfaces rooted in the physical and algorithmic layers of quantum computing. Despite a growing body of research on individual attack vectors - ranging from adversarial poisoni...

BarkBeetle: Stealing Decision Tree Models with Fault Injection

Machine learning models, particularly decision trees DTs, are widely adopted across various domains due to their interpretability and efficiency. However, as ML models become increasingly integrated into privacy-sensitive applications, concerns about their confidentiality have grown, particularly...

CEGA: a Cost-Effective Approach for Graph-Based Model Extraction and Acquisition

Graph Neural Networks GNNs have demonstrated remarkable utility across diverse applications, and their growing complexity has made Machine Learning as a Service MLaaS a viable platform for scalable deployment. However, this accessibility also exposes GNN to serious security threats, most notably...

GradEscape: a Gradient-Based Evader against AI-Generated Text Detectors

In this paper, we introduce GradEscape, the first gradient-based evader designed to attack AI-generated text AIGT detectors. GradEscape overcomes the undifferentiable computation problem, caused by the discrete nature of text, by introducing a novel approach to construct weighted embeddings for t...

MISLEADER: Defending against Model Extraction with Ensembles of Distilled Models

Model extraction attacks aim to replicate the functionality of a black-box model through query access, threatening the intellectual property IP of machine-learning-as-a-service MLaaS providers. Defending against such attacks is challenging, as it must balance efficiency, robustness, and utility...

RADEP: a Resilient Adaptive Defense Framework against Model Extraction Attacks

Machine Learning as a Service MLaaS enables users to leverage powerful machine learning models through cloud-based APIs, offering scalability and ease of deployment. However, these services are vulnerable to model extraction attacks, where adversaries repeatedly query the application programming...

Evaluating Query Efficiency and Accuracy of Transfer Learning-Based Model Extraction Attack in Federated Learning

Federated Learning FL is a collaborative learning framework designed to protect client data, yet it remains highly vulnerable to Intellectual Property IP threats. Model extraction ME attacks pose a significant risk to Machine Learning as a Service MLaaS platforms, enabling attackers to replicate...