Busting the Paper Ballot: Voting Meets Adversarial Machine Learning

We show the security risk associated with using machine learning classifiers in United States election tabulators. The central classification task in election tabulation is deciding whether a mark does or does not appear on a bubble associated to an alternative in a contest on the ballot. Barrett...

LLMs Cannot Reliably Judge (Yet?): a Comprehensive Assessment on the Robustness of LLM-As-A-Judge

Large Language Models LLMs have demonstrated remarkable intelligence across various tasks, which has inspired the development and widespread adoption of LLM-as-a-Judge systems for automated model testing, such as red teaming and benchmarking. However, these systems are susceptible to adversarial...

SDN-Based False Data Detection with Its Mitigation and Machine Learning Robustness for In-Vehicle Networks

As the development of autonomous and connected vehicles advances, the complexity of modern vehicles increases, with numerous Electronic Control Units ECUs integrated into the system. In an in-vehicle network, these ECUs communicate with one another using an standard protocol called Controller Are...

SATversary: Adversarial Attacks on Satellite Fingerprinting

As satellite systems become increasingly vulnerable to physical layer attacks via SDRs, novel countermeasures are being developed to protect critical systems, particularly those lacking cryptographic protection, or those which cannot be upgraded to support modern cryptography. Among these is...

Explainer-Guided Targeted Adversarial Attacks against Binary Code Similarity Detection Models

Binary code similarity detection BCSD serves as a fundamental technique for various software engineering tasks, e.g., vulnerability detection and classification. Attacks against such models have therefore drawn extensive attention, aiming at misleading the models to generate erroneous predictions...

Practical Adversarial Attacks on Stochastic Bandits Via Fake Data Injection

Adversarial attacks on stochastic bandits have traditionally relied on some unrealistic assumptions, such as per-round reward manipulation and unbounded perturbations, limiting their relevance to real-world systems. We propose a more practical threat model, Fake Data Injection, which reflects...

Security Concerns for Large Language Models: a Survey

Large Language Models LLMs such as GPT-4 and its recent iterations, Google's Gemini, Anthropic's Claude 3 models, and xAI's Grok have caused a revolution in natural language processing, but their capabilities also introduce new security vulnerabilities. In this survey, we provide a comprehensive...

Safety Alignment Can Be Not Superficial with Explicit Safety Signals

Recent studies on the safety alignment of large language models LLMs have revealed that existing approaches often operate superficially, leaving models vulnerable to various adversarial attacks. Despite their significance, these studies generally fail to offer actionable solutions beyond data...

Preventing Adversarial AI Attacks against Autonomous Situational Awareness: a Maritime Case Study

Adversarial artificial intelligence AI attacks pose a significant threat to autonomous transportation, such as maritime vessels, that rely on AI components. Malicious actors can exploit these systems to deceive and manipulate AI-driven operations. This paper addresses three critical research...

Scalable Defense against In-The-Wild Jailbreaking Attacks with Safety Context Retrieval

Large Language Models LLMs are known to be vulnerable to jailbreaking attacks, wherein adversaries exploit carefully engineered prompts to induce harmful or unethical responses. Such threats have raised critical concerns about the safety and reliability of LLMs in real-world deployment. While...

GenAI Security: Outsmarting the Bots with a Proactive Testing Framework

The increasing sophistication and integration of Generative AI GenAI models into diverse applications introduce new security challenges that traditional methods struggle to address. This research explores the critical need for proactive security measures to mitigate the risks associated with...

Quantum Support Vector Regression for Robust Anomaly Detection

Anomaly Detection AD is critical in data analysis, particularly within the domain of IT security. In recent years, Machine Learning ML algorithms have emerged as a powerful tool for AD in large-scale data. In this study, we explore the potential of quantum ML approaches, specifically quantum kern...

Deploying AI Agents? Learn to Secure Them Before Hackers Strike Your Business

AI agents are changing the way businesses work. They can answer questions, automate tasks, and create better user experiences. But with this power comes new risks — like data leaks, identity theft, and malicious misuse. If your company is exploring or already using AI agents, you need to ask: Are...

A Comprehensive Analysis of Adversarial Attacks against Spam Filters

Deep learning has revolutionized email filtering, which is critical to protect users from cyber threats such as spam, malware, and phishing. However, the increasing sophistication of adversarial attacks poses a significant challenge to the effectiveness of these filters. This study investigates t...

Constrained Network Adversarial Attacks: Validity, Robustness, and Transferability

While machine learning has significantly advanced Network Intrusion Detection Systems NIDS, particularly within IoT environments where devices generate large volumes of data and are increasingly susceptible to cyber threats, these models remain vulnerable to adversarial attacks. Our research...

Analysis of the Vulnerability of Machine Learning Regression Models to Adversarial Attacks Using Data from 5G Wireless Networks

This article describes the process of creating a script and conducting an analytical study of a dataset using the DeepMIMO emulator. An advertorial attack was carried out using the FGSM method to maximize the gradient. A comparison is made of the effectiveness of binary classifiers in the task of...

Mitigating the Structural Bias in Graph Adversarial Defenses

In recent years, graph neural networks GNNs have shown great potential in addressing various graph structure-related downstream tasks. However, recent studies have found that current GNNs are susceptible to malicious adversarial attacks. Given the inevitable presence of adversarial attacks in the...

The Dark Side of Digital Twins: Adversarial Attacks on AI-Driven Water Forecasting

Digital twins DTs are improving water distribution systems by using real-time data, analytics, and prediction models to optimize operations. This paper presents a DT platform designed for a Spanish water supply network, utilizing Long Short-Term Memory LSTM networks to predict water consumption...

Adversarial Attacks on LLM-As-A-Judge Systems: Insights from Prompt Injections

LLM as judge systems used to assess text quality code correctness and argument strength are vulnerable to prompt injection attacks. We introduce a framework that separates content author attacks from system prompt attacks and evaluate five models Gemma 3.27B Gemma 3.4B Llama 3.2 3B GPT 4 and Clau...

Property-Preserving Hashing for $\Ell_1$-Distance Predicates: Applications to Countering Adversarial Input Attacks

Perceptual hashing is used to detect whether an input image is similar to a reference image with a variety of security applications. Recently, they have been shown to succumb to adversarial input attacks which make small imperceptible changes to the input image yet the hashing algorithm does not...