AdRo-FL: Informed and Secure Client Selection for Federated Learning in the Presence of Adversarial Aggregator

Whitepaper called AdRo-FL: Informed And Secure Client Selection For Federated Learning In The Presence Of Adversarial Aggregator...

Analyzing PDFs like Binaries: Adversarially Robust PDF Malware Analysis Via Intermediate Representation and Language Model

Malicious PDF files have emerged as a persistent threat and become a popular attack vector in web-based attacks. While machine learning-based PDF malware classifiers have shown promise, these classifiers are often susceptible to adversarial attacks, undermining their reliability. To address this...

Probing the Robustness of Large Language Models Safety to Latent Perturbations

Safety alignment is a key requirement for building reliable Artificial General Intelligence. Despite significant advances in safety alignment, we observe that minor latent shifts can still trigger unsafe responses in aligned models. We argue that this stems from the shallow nature of existing...

Safety Interventions against Adversarial Patches in an Open-Source Driver Assistance System

Drivers are becoming increasingly reliant on advanced driver assistance systems ADAS as autonomous driving technology becomes more popular and developed with advanced safety features to enhance road safety. However, the increasing complexity of the ADAS makes autonomous vehicles AVs more exposed ...

Chain-Of-Code Collapse: Reasoning Failures in LLMs Via Adversarial Prompting in Code Generation

Large Language Models LLMs have achieved remarkable success in tasks requiring complex reasoning, such as code generation, mathematical problem solving, and algorithmic synthesis -- especially when aided by reasoning tokens and Chain-of-Thought prompting. Yet, a core question remains: do these...

Bias Amplification in RAG: Poisoning Knowledge Retrieval to Steer LLMs

In Large Language Models, Retrieval-Augmented Generation RAG systems can significantly enhance the performance of large language models by integrating external knowledge. However, RAG also introduces new security risks. Existing research focuses mainly on how poisoning attacks in RAG systems affe...

LLMs Cannot Reliably Judge (Yet?): a Comprehensive Assessment on the Robustness of LLM-As-A-Judge

Large Language Models LLMs have demonstrated remarkable intelligence across various tasks, which has inspired the development and widespread adoption of LLM-as-a-Judge systems for automated model testing, such as red teaming and benchmarking. However, these systems are susceptible to adversarial...

TRIDENT -- a Three-Tier Privacy-Preserving Propaganda Detection Model in Mobile Networks Using Transformers, Adversarial Learning, and Differential Privacy

The proliferation of propaganda on mobile platforms raises critical concerns around detection accuracy and user privacy. To address this, we propose TRIDENT - a three-tier propaganda detection model implementing transformers, adversarial learning, and differential privacy which integrates syntact...

GenBreak: Red Teaming Text-To-Image Generators Using Large Language Models

Text-to-image T2I models such as Stable Diffusion have advanced rapidly and are now widely used in content creation. However, these models can be misused to generate harmful content, including nudity or violence, posing significant safety risks. While most platforms employ content moderation...

Adversarial Text Generation with Dynamic Contextual Perturbation

Adversarial attacks on Natural Language Processing NLP models expose vulnerabilities by introducing subtle perturbations to input text, often leading to misclassification while maintaining human readability. Existing methods typically focus on word-level or local text segment alterations,...

Attacking Attention of Foundation Models Disrupts Downstream Tasks

Foundation models represent the most prominent and recent paradigm shift in artificial intelligence. Foundation models are large models, trained on broad data that deliver high accuracy in many downstream tasks, often without fine-tuning. For this reason, models such as CLIP , DINO or Vision...

MalGEN: a Generative Agent Framework for Modeling Malicious Software in Cybersecurity

The dual use nature of Large Language Models LLMs presents a growing challenge in cybersecurity. While LLM enhances automation and reasoning for defenders, they also introduce new risks, particularly their potential to be misused for generating evasive, AI crafted malware. Despite this emerging...

CAPAA: Classifier-Agnostic Projector-Based Adversarial Attack

Projector-based adversarial attack aims to project carefully designed light patterns i.e., adversarial projections onto scenes to deceive deep image classifiers. It has potential applications in privacy protection and the development of more robust classifiers. However, existing approaches...

D2R: Dual Regularization Loss with Collaborative Adversarial Generation for Model Robustness

The robustness of Deep Neural Network models is crucial for defending models against adversarial attacks. Recent defense methods have employed collaborative learning frameworks to enhance model robustness. Two key limitations of existing methods are i insufficient guidance of the target model via...

Can In-Context Reinforcement Learning Recover from Reward Poisoning Attacks?

We study the corruption-robustness of in-context reinforcement learning ICRL, focusing on the Decision-Pretrained Transformer DPT, Lee et al., 2023. To address the challenge of reward poisoning attacks targeting the DPT, we propose a novel adversarial training framework, called Adversarially...

Inside the Mind of the Adversary: Why More Security Leaders Are Selecting AEV

Cybersecurity involves both playing the good guy and the bad guy. Diving deep into advanced technologies and yet also going rogue in the Dark Web. Defining technical policies and also profiling attacker behavior. Security teams cannot be focused on just ticking boxes, they need to inhabit the...

Inside the Mind of the Adversary: Why More Security Leaders Are Selecting AEV

Cybersecurity involves both playing the good guy and the bad guy. Diving deep into advanced technologies and yet also going rogue in the Dark Web. Defining technical policies and also profiling attacker behavior. Security teams cannot be focused on just ticking boxes, they need to inhabit the...

SATversary: Adversarial Attacks on Satellite Fingerprinting

As satellite systems become increasingly vulnerable to physical layer attacks via SDRs, novel countermeasures are being developed to protect critical systems, particularly those lacking cryptographic protection, or those which cannot be upgraded to support modern cryptography. Among these is...

Adapting under Fire: Multi-Agent Reinforcement Learning for Adversarial Drift in Network Security

Evolving attacks are a critical challenge for the long-term success of Network Intrusion Detection Systems NIDS. The rise of these changing patterns has exposed the limitations of traditional network security methods. While signature-based methods are used to detect different types of attacks, th...

Breaking the Gaussian Barrier: Residual-PAC Privacy for Automatic Privatization

The Probably Approximately Correct PAC Privacy framework 1 provides a powerful instance-based methodology for certifying privacy in complex data-driven systems. However, existing PAC Privacy algorithms rely on a Gaussian mutual information upper bound. We show that this is in general too...