Backdoor Attacks and Defenses in Computer Vision Domain: a Survey

Backdoor trojan attacks embed hidden, controllable behaviors into machine-learning models so that models behave normally on benign inputs but produce attacker-chosen outputs when a trigger is present. This survey reviews the rapidly growing literature on backdoor attacks and defenses in the...

E-PhishGen: Unlocking Novel Research in Phishing Email Detection

Every day, our inboxes are flooded with unsolicited emails, ranging between annoying spam to more subtle phishing scams. Unfortunately, despite abundant prior efforts proposing solutions achieving near-perfect accuracy, the reality is that countering malicious emails still remains an unsolved...

Cheating on Quantum Computing Benchmarks

Peter Gutmann and Stephan Neuhaus have a new paper--I think it's new, even though it has a March 2025 date--that makes the argument that we shouldn't trust any of the quantum factorization benchmarks, because everyone has been cooking the books: Similarly, quantum factorisation is performed using...

PRISM: Programmatic Reasoning with Image Sequence Manipulation for LVLM Jailbreaking

The increasing sophistication of large vision-language models LVLMs has been accompanied by advances in safety alignment mechanisms designed to prevent harmful content generation. However, these defenses remain vulnerable to sophisticated adversarial attacks. Existing jailbreak methods typically...

Benchmarking Fraud Detectors on Private Graph Data

We introduce the novel problem of benchmarking fraud detectors on private graph-structured data. Currently, many types of fraud are managed in part by automated detection algorithms that operate over graphs. We consider the scenario where a data holder wishes to outsource development of fraud...

Vulnerability Mitigation System (VMS): LLM Agent and Evaluation Framework for Autonomous Penetration Testing

As the frequency of cyber threats increases, conventional penetration testing is failing to capture the entirety of todays complex environments. To solve this problem, we propose the Vulnerability Mitigation System VMS, a novel agent based on a Large Language Model LLM capable of performing...

PhreshPhish: a Real-World, High-Quality, Large-Scale Phishing Website Dataset and Benchmark

Phishing remains a pervasive and growing threat, inflicting heavy economic and reputational damage. While machine learning has been effective in real-time detection of phishing attacks, progress is hindered by lack of large, high-quality datasets and benchmarks. In addition to poor-quality due to...

ARMOR: Aligning Secure and Safe Large Language Models Via Meticulous Reasoning

Large Language Models LLMs have demonstrated remarkable generative capabilities. However, their susceptibility to misuse has raised significant safety concerns. While post-training safety alignment methods have been widely adopted, LLMs remain vulnerable to malicious instructions that can bypass...

Efficient Private Inference Based on Helper-Assisted Malicious Security Dishonest Majority MPC

Private inference based on Secure Multi-Party Computation MPC addresses data privacy risks in Machine Learning as a Service MLaaS. However, existing MPC-based private inference frameworks focuses on semi-honest or honest majority models, whose threat models are overly idealistic, while malicious...

Agent Safety Alignment Via Reinforcement Learning

The emergence of autonomous Large Language Model LLM agents capable of tool usage has introduced new safety risks that go beyond traditional conversational misuse. These agents, empowered to execute external functions, are vulnerable to both user-initiated threats e.g., adversarial prompts and...

Evaluating the Critical Risks of Amazon'S Nova Premier under the Frontier Model Safety Framework

Nova Premier is Amazon's most capable multimodal foundation model and teacher for model distillation. It processes text, images, and video with a one-million-token context window, enabling analysis of large codebases, 400-page documents, and 90-minute videos in a single prompt. We present the fir...

NAP-Tuning: Neural Augmented Prompt Tuning for Adversarially Robust Vision-Language Models

Vision-Language Models VLMs such as CLIP have demonstrated remarkable capabilities in understanding relationships between visual and textual data through joint embedding spaces. Despite their effectiveness, these models remain vulnerable to adversarial attacks, particularly in the image modality,...

Bridging Unsupervised and Semi-Supervised Anomaly Detection: a Theoretically-Grounded and Practical Framework with Synthetic Anomalies

Anomaly detection AD is a critical task across domains such as cybersecurity and healthcare. In the unsupervised setting, an effective and theoretically-grounded principle is to train classifiers to distinguish normal data from synthetic anomalies. We extend this principle to semi-supervised AD,...

SoK: Machine Unlearning for Large Language Models

Large language model LLM unlearning has become a critical topic in machine learning, aiming to eliminate the influence of specific training data or knowledge without retraining the model from scratch. A variety of techniques have been proposed, including Gradient Ascent, model editing, and...

The Scales of Justitia: a Comprehensive Survey on Safety Evaluation of LLMs

With the rapid advancement of artificial intelligence technology, Large Language Models LLMs have demonstrated remarkable potential in the field of Natural Language Processing NLP, including areas such as content generation, human-computer interaction, machine translation, and code generation,...

Sentinel: SOTA Model to Protect against Prompt Injections

Large Language Models LLMs are increasingly powerful but remain vulnerable to prompt injection attacks, where malicious inputs cause the model to deviate from its intended instructions. This paper introduces Sentinel, a novel detection model, qualifire/prompt-injection-sentinel, based on the...

Comprehensive Vulnerability Analysis Is Necessary for Trustworthy LLM-MAS

This paper argues that a comprehensive vulnerability analysis is essential for building trustworthy Large Language Model-based Multi-Agent Systems LLM-MAS. These systems, which consist of multiple LLM-powered agents working collaboratively, are increasingly deployed in high-stakes applications bu...

SafeCOMM: What about Safety Alignment in Fine-Tuned Telecom Large Language Models?

Fine-tuning large language models LLMs for telecom tasks and datasets is a common practice to adapt general-purpose models to the telecom domain. However, little attention has been paid to how this process may compromise model safety. Recent research has shown that even benign fine-tuning can...

Jailbreak Distillation: Renewable Safety Benchmarking

Large language models LLMs are rapidly deployed in critical applications, raising urgent needs for robust safety benchmarking. We propose Jailbreak Distillation JBDistill, a novel benchmark construction framework that "distills" jailbreak attacks into high-quality and easily-updatable safety...

MAPS: a Multilingual Benchmark for Global Agent Performance and Security

Agentic AI systems, which build on Large Language Models LLMs and interact with tools and memory, have rapidly advanced in capability and scope. Yet, since LLMs have been shown to struggle in multilingual settings, typically resulting in lower performance and reduced safety, agentic systems risk...