autoguardrails

autoguardrails Open source by Santander AI Lab. An LLM / AI-safety guardrail research library / evaluation harness autoresearch-style: it searches over a single mutable policy.md surface to minimize attack success rate ASR against a fixed evaluation suite, with a benign-pass floor. Part of...

Where Rules End and Judges Begin: Measuring the Judgment Boundary in Multi-Agent Systems Security

LLM-based multi-agent systems MAS engage tools, share memory, and delegate tasks, often encountering adversarial content. Current defenses for MAS are typically evaluated in isolation, focusing on one attack type at a time, which can lead to costly and hard-to-audit outcomes. This study organizes...

Red-TTT: Test-Time Training for Automated Jailbreaking Large Language Models

Large language models remain vulnerable to jailbreaks, and automated red teaming is the standard way to find jailbreaks in large language models at scale. Current methods either draw more samples at test time through search, rewriting, and tree expansion, or train a stronger attacker offline with...

Threat-Preserving Representation Sensitivity in Agent-Security Benchmarks

Security benchmarks for LLM-based agents often report the attack success rate ASR as a measure of model robustness and use these scores to compare different models and defense mechanisms, assuming that they describe the security of the agent. In this paper, we explore whether it also influences t...

Attack Success Rate Is Not a Number: On Measurement Validity in Agentic AI Security Evaluation

Attack success rate ASR is the headline metric in nearly every published evaluation of attacks on, and defenses for, LLM agents. We argue that ASR as currently used is not a single quantity but a family of metrics parameterized by six design choices that papers seldom specify and never hold...

Rethinking Backdoor Repair Evaluation: Distinguishing Aggregate Clean Utility from Benign Performance Preservation

Backdoor repair aims to suppress malicious behavior in compromised models while preserving benign task performance. Existing studies typically evaluate these objectives using Attack Success Rate ASR and Overall Clean Accuracy, but aggregate clean accuracy can obscure substantial degradation...

Pick Your Poison: Learning to Select Poison Sets for Stronger LLM Backdoor Attacks

Backdoor poisoning attacks add poisoned examples to otherwise-clean finetuning data, pairing a trigger with a target behavior that the model learns to produce when the trigger appears. Existing evaluations typically fix the number of poisoned examples and sample them at random from a candidate...

The Fragility of Jailbreak Robustness across Operational States

Existing jailbreak evaluations typically characterize robustness using a single attack success rate ASR measured in a default configuration the vanilla state. However, user-LLM interactions can induce diverse operational states beyond the vanilla state. In this work, we find that jailbreak...

Beyond the Payload: How User Invocation Shapes Coding Agent Vulnerability to Repository Poisoning

Coding agents are increasingly used for software engineering tasks, including bootstrapping projects from third-party repositories whose integrity cannot be assumed. Prior work on repository poisoning largely focuses on attacker-controlled injection and disguise, but developers also shape risk...

Workspace Topology As an Attack Vector in Agentic Coding Assistants

Agentic coding assistants are finding widespread use, not just in new code development but in quickly ingesting and leveraging third-party code. This opens up a risk of malicious code being ingested as these coding tools operate with broad filesystem access inside developer workspaces. In this...

A Multimodal Automatic Redteaming Evaluation Based on Atomic Jailbreak Strategy Decoupling and Combination

Multimodal Large Language Models MLLMs have achieved impressive progress in image-text comprehension and generation, yet they remain susceptible to jailbreak attacks that can trigger harmful outputs and pose serious safety concerns. Existing multimodal jailbreak attacks have shown the feasibility...

Decoy Images Amplify Caption-Mediated Defenses against Encoded Jailbreaks

We report a counter-intuitive interaction between image inputs and existing black-box defenses on Vision--Language Models VLMs: pairing an encoded jailbreak prompt with an unrelated decoy image can sharply lower attack success rate ASR. The operative change is in the defense pipeline, not in the...

ALIBI: Adaptive Agentic Attacks on LLM-Based Vulnerability Detectors Via Adversarial Code Comments

Large language models are increasingly deployed for security-sensitive tasks such as vulnerability detection and code review. Their reliance on natural-language context embedded in source code exposes a previously underexplored attack surface: adversarial comments that can influence a detector's...

Defense against LLM Backdoors Using Critical Neuron Isolation Pruning

Large language models LLMs are vulnerable to backdoor attacks, where hidden triggers induce malicious outputs. Existing defenses generally fall into inference-time detection or training-time mitigation, but face two key limitations. First, they focus on fine-tuning-based backdoors e.g., PEFT...

Natural Backdoor Attacks on Speech Recognition Models

With the rapid development of deep learning, its vulnerability has gradually emerged in recent years. This work focuses on backdoor attacks on speech recognition systems. We adopt sounds that are ordinary in nature or in our daily life as triggers for natural backdoor attacks. We conduct...

HarmQ: Harmonic Backdoor Attacks against Quantum Neural Networks

Quantum Neural Networks QNNs have emerged as a promising paradigm for quantum machine learning in the Noisy Intermediate-Scale Quantum NISQ era, leveraging quantum phenomena such as superposition and entanglement to process information in exponentially large Hilbert spaces. However, QNNs inherit...

Understanding and Evaluating Claw-Like Agent Security through a Computer-Systems Lens

Claw-like AI agents e.g., OpenClaw are always-on processes with persistent access to credentials, files, tools, and external services. They take on system-level responsibilities -- installing packages, maintaining state, scheduling subtasks, and mediating I/O -- making security failures far more...

Model Poisoning against Federated Model Adaptation with Chain of Bit-Flips

Federated Learning FL allows a set of clients to collectively train a global model without sharing local training data. Giving the responsibility of the training to decentralized actors may lead to poisoning attacks: clients controlled by malicious third party potentially poison the training...

POISE: Position-Aware Undetectable Skill Injection on LLM Agents

Agent skills provide a lightweight mechanism for extending general-purpose agents, but their open format exposes them to skill-poisoning attacks. A practically dangerous injection must stay invisible: if executing the payload derails the user's legitimate task, the resulting failure signal invite...

MaskForge: Structure-Aware Adaptive Attacks for Jailbreaking Diffusion Large Language Models

Diffusion large language models dLLMs generate text by iteratively denoising partially masked sequences under bidirectional context, exposing a safety surface distinct from autoregressive LLMs. Because mask tokens are native inputs and tokens are committed by confidence rather than position,...