OpenAttack

Documentación • Características y Usos • Ejemplos de Uso • Modelos de Ataque • Diseño del Toolkit OpenAttack es un kit de herramientas de ataque adversarial textual de código abierto basado en Python, que maneja todo el proceso de ataque adversarial textual, incluyendo preprocesamiento de texto,...

augustus

Augustus - LLM vulnerability scanner for prompt injection, jailbreak, and adversarial attack testing Augustus - LLM Vulnerability Scanner Test large language models against 210+ adversarial attacks covering prompt injection, jailbreaks, encoding exploits, and data extraction. Augustus is a Go-bas...

Awesome-MoAI-Security

Awesome Mobile On-Device AI Security Seguridad de IA en Dispositivos Móviles SoK: Landscape de Ataques y Defensas de los Sistemas de IA en Dispositivos Móviles Los sistemas de IA en dispositivos móviles ejecutan modelos de IA localmente a través de frameworks de ML como LiteRT/TFLite , Core ML ,...

foolbox

home: true heroImage: /logo.png heroText: Foolbox tagline: "Foolbox: ataques adversarios rápidos para medir la robustez de modelos de aprendizaje automático en PyTorch, TensorFlow y JAX" actionText: Comenzar → actionLink: /guide/ features: title: Rendimiento nativo details: Foolbox 3 está...

Johnny Still Receives Spam SMS: Assessing the Robustness of SMS Spam Detection

SMS spam detection systems often achieve high accuracy in controlled environments but struggle against adversarial attacks and increasingly sophisticated spam tactics in real-world deployments. In this paper, we evaluate the robustness of SMS anti-spam systems that end users actually rely on,...

Categorical Robustness Assessment for Machine Learning Based Network Intrusion Detection Systems

Network Intrusion Detection Systems NIDS heavily utlize Machine Learning ML but ML models can be manipulated via adversarial attacks. These attacks add carefully crafted perturbations to network traffic data that leads to misclassifications. While prior work has demonstrated adversarial...

Context-Based Adversarial Attacks on AI Code Generators: Vulnerability Analysis and Implications

AI-powered code generation systems have transformed software development but introduce critical inference-time security vulnerabilities. This research presents a systematic investigation of context-based adversarial attacks, where strategically crafted contextual inputs, including comments,...

Protecting On-Device AI Inference: A Systematic Review of Attacks and Defence Mechanisms

The need for secure and private Artificial Intelligence AI and Machine Learning ML on edge and mobile devices has increased the necessity of protecting the architecture of these systems from threats to both security and privacy. With an ever-increasing number of pre-trained AI models being used o...

Beyond Content Safety: Real-Time Monitoring for Reasoning Vulnerabilities in Large Language Models

Large language models LLMs increasingly rely on explicit chain-of-thought CoT reasoning to solve complex tasks, yet the safety of the reasoning process itself remains largely unaddressed. Existing work on LLM safety focuses on content safety--detecting harmful, biased, or factually incorrect...

Towards Secure Retrieval-Augmented Generation: A Comprehensive Review of Threats, Defenses and Benchmarks

Retrieval-Augmented Generation RAG significantly mitigates the hallucinations and domain knowledge deficiency in large language models by incorporating external knowledge bases. However, the multi-module architecture of RAG introduces complex system-level security vulnerabilities. Guided by the R...

Enhancing Network Intrusion Detection Systems: A Multi-Layer Ensemble Approach to Mitigate Adversarial Attacks

Adversarial examples can represent a serious threat to machine learning ML algorithms. If used to manipulate the behaviour of ML-based Network Intrusion Detection Systems NIDS, they can jeopardize network security. In this work, we aim to mitigate such risks by increasing the robustness of NIDS...

Optimal Transport-Guided Adversarial Attacks on Graph Neural Network-Based Bot Detection

The rise of bot accounts on social media poses significant risks to public discourse. To address this threat, modern bot detectors increasingly rely on Graph Neural Networks GNNs. However, the effectiveness of these GNN-based detectors in real-world settings remains poorly understood. In practice...

AI-Driven Cybersecurity Threats: A Survey of Emerging Risks and Defensive Strategies

Artificial Intelligence's dual-use nature is revolutionizing the cybersecurity landscape, introducing new threats across four main categories: deepfakes and synthetic media, adversarial AI attacks, automated malware, and AI-powered social engineering. This paper aims to analyze emerging risks,...

Enhancing Adversarial Robustness of IoT Intrusion Detection Via SHAP-Based Attribution Fingerprinting

The rapid proliferation of Internet of Things IoT devices has transformed numerous industries by enabling seamless connectivity and data-driven automation. However, this expansion has also exposed IoT networks to increasingly sophisticated security threats, including adversarial attacks targeting...

Quantifying the Risk of Transferred Black Box Attacks

Neural networks have become pervasive across various applications, including security-related products. However, their widespread adoption has heightened concerns regarding vulnerability to adversarial attacks. With emerging regulations and standards emphasizing security, organizations must...

Secure Control of Connected and Autonomous Electrified Vehicles under Adversarial Cyber-Attacks

Connected and Autonomous Electrified Vehicles CAEV is the solution to the future smart mobility having benefits of efficient traffic flow and cleaner environmental impact. Although CAEV has advantages they are still susceptible to adversarial cyber attacks due to their autonomous electric operati...

Breaking Agent Backbones: Evaluating the Security of Backbone LLMs in AI Agents

AI agents powered by large language models LLMs are being deployed at scale, yet we lack a systematic understanding of how the choice of backbone LLM affects agent security. The non-deterministic sequential nature of AI agents complicates security modeling, while the integration of traditional...

Beyond Text: Multimodal Jailbreaking of Vision-Language and Audio Models through Perceptually Simple Transformations

Multimodal large language models MLLMs have achieved remarkable progress, yet remain critically vulnerable to adversarial attacks that exploit weaknesses in cross-modal processing. We present a systematic study of multimodal jailbreaks targeting both vision-language and audio-language models,...

Enhancing Security in Deep Reinforcement Learning: A Comprehensive Survey on Adversarial Attacks and Defenses

With the wide application of deep reinforcement learning DRL techniques in complex fields such as autonomous driving, intelligent manufacturing, and smart healthcare, how to improve its security and robustness in dynamic and changeable environments has become a core issue in current research...

Exploring the Effect of DNN Depth on Adversarial Attacks in Network Intrusion Detection Systems

Adversarial attacks pose significant challenges to Machine Learning ML systems and especially Deep Neural Networks DNNs by subtly manipulating inputs to induce incorrect predictions. This paper investigates whether increasing the layer depth of deep neural networks affects their robustness agains...