14 matches found
autoguardrails
autoguardrails Open source by Santander AI Lab. An LLM / AI-safety guardrail research library / evaluation harness autoresearch-style: it searches over a single mutable policy.md surface to minimize attack success rate ASR against a fixed evaluation suite, with a benign-pass floor. Part of...
redeval
RedEval - LLM Safety Evaluation Framework A comprehensive framework for evaluating the safety of Large Language Models LLMs through systematic attack and refusal testing. RedEval provides a unified, secure, and extensible platform for assessing LLM robustness against adversarial prompts and harmf...
meta-ai-support-prompt
Meta AI Support Assistant System Prompt Extracted system prompt from Meta's AI Support Assistant on June 1, 2026. Files system-prompt.md — Extracted system prompt ⚠️ Disclaimer & Legal Notice Purpose This repository is published strictly for educational and authorized security research purposes...
saffron
Saffron-1: Inference Scaling for LLM Safety Assurance 📖 Paper 🛠️ Dependencies The code was tested under the following dependencies: Python 3.12.3 CUDA 12.2 typingextensions==4.14.0 numpy==2.2.6 torch==2.5.1 huggingfacehub==0.30.2 accelerate==1.1.1 datasets==3.1.0 evaluate==0.4.3...
DARWIN
🧬 DARWIN Evolving Jailbreak Adversary and Guardrail for LLM Safety Evaluation and Protection An evolutionary attack–defense framework that couples DARWIN-Attack with DARWIN-Guard through online adversarial training. Paper · Guard Checkpoint · Installation · Use DARWIN-Attack · Use DARWIN-Guard ·...
TIER: Threat Implicitness Benchmark for Evaluating LLM Safety Behaviors
Current LLM safety benchmarks largely rely on binary metrics, overlooking how models respond to harmful prompts with varying threat implicitness. We introduce TIER, a Threat Implicitness Benchmark for behavioral safety evaluation of LLMs. TIER covers four risk domains and four threat levels, from...
Russia-Aligned UAC-0099 Plants Nuclear Weapon Prompt in Malware to Disrupt AI Analysis
Cybersecurity researchers have disclosed a new technique dubbed GuardBreaker that's been put to use by a Russia-aligned threat actor known as UAC-0099 against a target in Ukraine with an aim to interfere with artificial intelligence AI-assisted analysis. The idea, ESET said in a series of posts o...
NeuronFuzz: Safety Neuron Guided Fuzzing for LLM Safety Evaluation
Safety evaluation is critical for assessing whether aligned Large Language Models LLMs remain robust against jailbreak attacks. Existing automated testing methods, however, largely rely on response-level feedback: each candidate prompt typically requires generating a target-model response to...
MobileWorldSafety: Benchmarking GUI Agent Safety against Environmental Injection Attacks in Android Apps
LLM-powered GUI agents that autonomously operate smartphones are rapidly transitioning from research prototypes to early real-world deployment. However, because these agents routinely process untrusted environmental content, they are highly vulnerable to environmental injection attacks, which...
Measuring the Wrong Thing: Internal Harmfulness Scores Anti-Rank Successful Jailbreaks
Internal safety scores judge a prompt before any text is generated, and they are validated by how well they separate harmful prompts from benign ones. That separation is then read as evidence that the score will also catch the attacks that succeed. Harmful intent is a property of the prompt...
delirium-ai-safety-benchmark — Updated!
Delirium AI Safety Benchmark A diagnostic framework for measuring LLM vulnerability to Affective Contextual Erosion ACE and related liminal attack vectors. Delirium is not an exploitation tool. It is a standardized benchmark designed to detect the precise moment when a language model's attention...
Quality-Diversity Evolution for Discovering Diverse Vulnerabilities in LLM Safety
Current approaches to LLM adversarial testing suffer from coverage gaps: manual red-teaming does not scale, LLM-as-attacker methods exhibit mode collapse, and gradient-based approaches produce uninterpretable gibberish. We introduce a quality-diversity evolutionary framework that operates at the...
A Wolf in Sheep's Clothing: Bypassing Commercial LLM Guardrails Via Harmless Prompt Weaving and Adaptive Tree Search
Large language models LLMs remain vulnerable to jailbreak attacks that bypass safety guardrails to elicit harmful outputs. Existing approaches overwhelmingly operate within the prompt-optimization paradigm: whether through traditional algorithmic search or recent agent-based workflows, the...
