Awesome-MoAI-Security

移动端设备端AI安全资源精选 SoK：移动设备端AI系统的攻防全景 移动设备端AI系统通过 LiteRT/TFLite 、Core ML 、ExecuTorch 、ONNX 等机器学习框架以及硬件加速器在本地执行AI模型。本仓库追踪理解和保护此类系统所需的安全研究，因为设备端模型的本地存储引入了新的安全风险。 移动设备端AI系统概览 目录 阅读路线图 分类体系速览 跨支柱安全分析 针对MoAI系统的攻击 对抗性攻击 模型相似性利用 梯度重建 预处理操纵 后门攻击 载荷注入 模型量化 图像隐写 对抗性权重攻击 模型窃取攻击 静态分析 动态分析 侧信道 能耗-延迟攻击 针对MoAI系统的防御...

foolbox

home: true heroImage: /logo.png heroText: Foolbox tagline: "Foolbox: ataques adversarios rápidos para medir la robustez de modelos de aprendizaje automático en PyTorch, TensorFlow y JAX" actionText: Comenzar → actionLink: /guide/ features: title: Rendimiento nativo details: Foolbox 3 está...

augustus

Augustus - LLM vulnerability scanner for prompt injection, jailbreak, and adversarial attack testing Augustus - LLM Vulnerability Scanner Test large language models against 210+ adversarial attacks covering prompt injection, jailbreaks, encoding exploits, and data extraction. Augustus is a Go-bas...

OpenAttack

文档 • 功能与用途 • 使用示例 • 攻击模型 • 工具包设计 OpenAttack 是一个基于 Python 的开源文本对抗攻击工具包，它处理文本对抗攻击的完整流程，包括文本预处理、访问受害者模型、生成对抗样本和评估。 功能与用途 OpenAttack 具有以下特性： ⭐️ 支持所有攻击类型 。OpenAttack 支持所有类型的攻击，包括句子级/单词级/字符级扰动以及基于梯度/得分/决策/盲攻击模型； ⭐️ 多语言支持 。OpenAttack 目前支持中文和英文。其可扩展的设计能够快速支持更多语言； ⭐️ 并行处理 。OpenAttack...

Johnny Still Receives Spam SMS: Assessing the Robustness of SMS Spam Detection

SMS spam detection systems often achieve high accuracy in controlled environments but struggle against adversarial attacks and increasingly sophisticated spam tactics in real-world deployments. In this paper, we evaluate the robustness of SMS anti-spam systems that end users actually rely on,...

Categorical Robustness Assessment for Machine Learning Based Network Intrusion Detection Systems

Network Intrusion Detection Systems NIDS heavily utlize Machine Learning ML but ML models can be manipulated via adversarial attacks. These attacks add carefully crafted perturbations to network traffic data that leads to misclassifications. While prior work has demonstrated adversarial...

Context-Based Adversarial Attacks on AI Code Generators: Vulnerability Analysis and Implications

AI-powered code generation systems have transformed software development but introduce critical inference-time security vulnerabilities. This research presents a systematic investigation of context-based adversarial attacks, where strategically crafted contextual inputs, including comments,...

Protecting On-Device AI Inference: A Systematic Review of Attacks and Defence Mechanisms

The need for secure and private Artificial Intelligence AI and Machine Learning ML on edge and mobile devices has increased the necessity of protecting the architecture of these systems from threats to both security and privacy. With an ever-increasing number of pre-trained AI models being used o...

Beyond Content Safety: Real-Time Monitoring for Reasoning Vulnerabilities in Large Language Models

Large language models LLMs increasingly rely on explicit chain-of-thought CoT reasoning to solve complex tasks, yet the safety of the reasoning process itself remains largely unaddressed. Existing work on LLM safety focuses on content safety--detecting harmful, biased, or factually incorrect...

Towards Secure Retrieval-Augmented Generation: A Comprehensive Review of Threats, Defenses and Benchmarks

Retrieval-Augmented Generation RAG significantly mitigates the hallucinations and domain knowledge deficiency in large language models by incorporating external knowledge bases. However, the multi-module architecture of RAG introduces complex system-level security vulnerabilities. Guided by the R...

Enhancing Network Intrusion Detection Systems: A Multi-Layer Ensemble Approach to Mitigate Adversarial Attacks

Adversarial examples can represent a serious threat to machine learning ML algorithms. If used to manipulate the behaviour of ML-based Network Intrusion Detection Systems NIDS, they can jeopardize network security. In this work, we aim to mitigate such risks by increasing the robustness of NIDS...

Optimal Transport-Guided Adversarial Attacks on Graph Neural Network-Based Bot Detection

The rise of bot accounts on social media poses significant risks to public discourse. To address this threat, modern bot detectors increasingly rely on Graph Neural Networks GNNs. However, the effectiveness of these GNN-based detectors in real-world settings remains poorly understood. In practice...

AI-Driven Cybersecurity Threats: A Survey of Emerging Risks and Defensive Strategies

Artificial Intelligence's dual-use nature is revolutionizing the cybersecurity landscape, introducing new threats across four main categories: deepfakes and synthetic media, adversarial AI attacks, automated malware, and AI-powered social engineering. This paper aims to analyze emerging risks,...

Enhancing Adversarial Robustness of IoT Intrusion Detection Via SHAP-Based Attribution Fingerprinting

The rapid proliferation of Internet of Things IoT devices has transformed numerous industries by enabling seamless connectivity and data-driven automation. However, this expansion has also exposed IoT networks to increasingly sophisticated security threats, including adversarial attacks targeting...

Quantifying the Risk of Transferred Black Box Attacks

Neural networks have become pervasive across various applications, including security-related products. However, their widespread adoption has heightened concerns regarding vulnerability to adversarial attacks. With emerging regulations and standards emphasizing security, organizations must...

Secure Control of Connected and Autonomous Electrified Vehicles under Adversarial Cyber-Attacks

Connected and Autonomous Electrified Vehicles CAEV is the solution to the future smart mobility having benefits of efficient traffic flow and cleaner environmental impact. Although CAEV has advantages they are still susceptible to adversarial cyber attacks due to their autonomous electric operati...

Breaking Agent Backbones: Evaluating the Security of Backbone LLMs in AI Agents

AI agents powered by large language models LLMs are being deployed at scale, yet we lack a systematic understanding of how the choice of backbone LLM affects agent security. The non-deterministic sequential nature of AI agents complicates security modeling, while the integration of traditional...

Enhancing Security in Deep Reinforcement Learning: A Comprehensive Survey on Adversarial Attacks and Defenses

With the wide application of deep reinforcement learning DRL techniques in complex fields such as autonomous driving, intelligent manufacturing, and smart healthcare, how to improve its security and robustness in dynamic and changeable environments has become a core issue in current research...

Beyond Text: Multimodal Jailbreaking of Vision-Language and Audio Models through Perceptually Simple Transformations

Multimodal large language models MLLMs have achieved remarkable progress, yet remain critically vulnerable to adversarial attacks that exploit weaknesses in cross-modal processing. We present a systematic study of multimodal jailbreaks targeting both vision-language and audio-language models,...

Exploring the Effect of DNN Depth on Adversarial Attacks in Network Intrusion Detection Systems

Adversarial attacks pose significant challenges to Machine Learning ML systems and especially Deep Neural Networks DNNs by subtly manipulating inputs to induce incorrect predictions. This paper investigates whether increasing the layer depth of deep neural networks affects their robustness agains...