Weakest Link in the Chain: Security Vulnerabilities in Advanced Reasoning Models

The introduction of advanced reasoning capabilities have improved the problem-solving performance of large language models, particularly on math and coding benchmarks. However, it remains unclear whether these reasoning models are more or less vulnerable to adversarial prompt attacks than their...

AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructions

The rapid advancement of vision-language models VLMs and their integration into embodied agents have unlocked powerful capabilities for decision-making. However, as these systems are increasingly deployed in real-world environments, they face mounting safety concerns, particularly when responding...

A Theory of Lending Protocols in DeFi

Lending protocols are one of the main applications of Decentralized Finance DeFi, enabling crypto-assets loan markets with a total value estimated in the tens of billions of dollars. Unlike traditional lending systems, these protocols operate without relying on trusted authorities or off-chain...

Unlearning-Enhanced Website Fingerprinting Attack: against Backdoor Poisoning in Anonymous Networks

Website Fingerprinting WF is an effective tool for regulating and governing the dark web. However, its performance can be significantly degraded by backdoor poisoning attacks in practical deployments. This paper aims to address the problem of hidden backdoor poisoning attacks faced by Website...

AdRo-FL: Informed and Secure Client Selection for Federated Learning in the Presence of Adversarial Aggregator

Whitepaper called AdRo-FL: Informed And Secure Client Selection For Federated Learning In The Presence Of Adversarial Aggregator...

Analyzing PDFs like Binaries: Adversarially Robust PDF Malware Analysis Via Intermediate Representation and Language Model

Malicious PDF files have emerged as a persistent threat and become a popular attack vector in web-based attacks. While machine learning-based PDF malware classifiers have shown promise, these classifiers are often susceptible to adversarial attacks, undermining their reliability. To address this...

Safety Interventions against Adversarial Patches in an Open-Source Driver Assistance System

Drivers are becoming increasingly reliant on advanced driver assistance systems ADAS as autonomous driving technology becomes more popular and developed with advanced safety features to enhance road safety. However, the increasing complexity of the ADAS makes autonomous vehicles AVs more exposed ...

Probing the Robustness of Large Language Models Safety to Latent Perturbations

Safety alignment is a key requirement for building reliable Artificial General Intelligence. Despite significant advances in safety alignment, we observe that minor latent shifts can still trigger unsafe responses in aligned models. We argue that this stems from the shallow nature of existing...

Bias Amplification in RAG: Poisoning Knowledge Retrieval to Steer LLMs

In Large Language Models, Retrieval-Augmented Generation RAG systems can significantly enhance the performance of large language models by integrating external knowledge. However, RAG also introduces new security risks. Existing research focuses mainly on how poisoning attacks in RAG systems affe...

Chain-Of-Code Collapse: Reasoning Failures in LLMs Via Adversarial Prompting in Code Generation

Large Language Models LLMs have achieved remarkable success in tasks requiring complex reasoning, such as code generation, mathematical problem solving, and algorithmic synthesis -- especially when aided by reasoning tokens and Chain-of-Thought prompting. Yet, a core question remains: do these...

LLMs Cannot Reliably Judge (Yet?): a Comprehensive Assessment on the Robustness of LLM-As-A-Judge

Large Language Models LLMs have demonstrated remarkable intelligence across various tasks, which has inspired the development and widespread adoption of LLM-as-a-Judge systems for automated model testing, such as red teaming and benchmarking. However, these systems are susceptible to adversarial...

TRIDENT -- a Three-Tier Privacy-Preserving Propaganda Detection Model in Mobile Networks Using Transformers, Adversarial Learning, and Differential Privacy

The proliferation of propaganda on mobile platforms raises critical concerns around detection accuracy and user privacy. To address this, we propose TRIDENT - a three-tier propaganda detection model implementing transformers, adversarial learning, and differential privacy which integrates syntact...

GenBreak: Red Teaming Text-To-Image Generators Using Large Language Models

Text-to-image T2I models such as Stable Diffusion have advanced rapidly and are now widely used in content creation. However, these models can be misused to generate harmful content, including nudity or violence, posing significant safety risks. While most platforms employ content moderation...

Adversarial Text Generation with Dynamic Contextual Perturbation

Adversarial attacks on Natural Language Processing NLP models expose vulnerabilities by introducing subtle perturbations to input text, often leading to misclassification while maintaining human readability. Existing methods typically focus on word-level or local text segment alterations,...

Attacking Attention of Foundation Models Disrupts Downstream Tasks

Foundation models represent the most prominent and recent paradigm shift in artificial intelligence. Foundation models are large models, trained on broad data that deliver high accuracy in many downstream tasks, often without fine-tuning. For this reason, models such as CLIP , DINO or Vision...

MalGEN: a Generative Agent Framework for Modeling Malicious Software in Cybersecurity

The dual use nature of Large Language Models LLMs presents a growing challenge in cybersecurity. While LLM enhances automation and reasoning for defenders, they also introduce new risks, particularly their potential to be misused for generating evasive, AI crafted malware. Despite this emerging...

CAPAA: Classifier-Agnostic Projector-Based Adversarial Attack

Projector-based adversarial attack aims to project carefully designed light patterns i.e., adversarial projections onto scenes to deceive deep image classifiers. It has potential applications in privacy protection and the development of more robust classifiers. However, existing approaches...

D2R: Dual Regularization Loss with Collaborative Adversarial Generation for Model Robustness

The robustness of Deep Neural Network models is crucial for defending models against adversarial attacks. Recent defense methods have employed collaborative learning frameworks to enhance model robustness. Two key limitations of existing methods are i insufficient guidance of the target model via...

Can In-Context Reinforcement Learning Recover from Reward Poisoning Attacks?

We study the corruption-robustness of in-context reinforcement learning ICRL, focusing on the Decision-Pretrained Transformer DPT, Lee et al., 2023. To address the challenge of reward poisoning attacks targeting the DPT, we propose a novel adversarial training framework, called Adversarially...

Inside the Mind of the Adversary: Why More Security Leaders Are Selecting AEV

Cybersecurity involves both playing the good guy and the bad guy. Diving deep into advanced technologies and yet also going rogue in the Dark Web. Defining technical policies and also profiling attacker behavior. Security teams cannot be focused on just ticking boxes, they need to inhabit the...