llm-security

New: Demonstrating Indirect Injection attacks on Bing Chat Compromising LLMs using Indirect Prompt Injection "... a language model is a Turing-complete weird machine running programs written in natural language; when you do retrieval, you are not 'plugging updated facts into your AI', you are...

Awesome-LLMs-for-Vulnerability-Detection

Awesome Large Language Models for Vulnerability Detection A curated list of papers, projects, and agent skills on using LLMs for vulnerability detection and discovery. 📄 Papers Only showing 2025 and later. For earlier work, see Papers Archive 2024 and earlier. Title| Venue| Year| Paper| Github...

Perturbed-Embedding-Vectors

Jailbreaking Open-Weight LLMs via Random Embedding Perturbations: runs, labels and code Code and data for the paper Jailbreaking Open-Weight LLMs via Random Embedding Perturbations. Authored by the amazing people of the Theoretical Computer Science Department of University of California, Santa...

Securiscan

Securiscan Securiscan Securiscan is a robust, lightweight, and native standard-library security triage engine built to analyze vulnerabilities across Web Applications, APIs, LLMs, and Mobile Packages completely using Python standard built-ins. 🚀 Core Execution Modes 1. Passive External Inspection...

Jailbreaking Open-Weight LLMs Via Random Embedding Perturbations

While open-weight models have enjoyed steady progress in capabilities and wide adoption across multiple domains, their safety remains an important concern. One key feature is the ability to refuse or deflect harmful, malicious, or insensitive prompts. In this paper, we expose safety vulnerabiliti...

Cross-Lingual Transferability of Training Data Extraction Attacks to Recover Memorized PII

The robustness of Personally Identifiable Information PII protection in Large Language Models LLMs is a critical concern, yet the risks associated with cross-lingual data extraction remain under-explored. This study evaluates the vulnerability of English-centric and multilingual models to Trainin...

Reflections and Fragments: Securing LLMs against Sequential Mosaic Attacks

Self-play red-teaming improves language-model safety by pitting attacker and defender roles against each other in a zero-sum game. However, real adversaries increasingly use mosaic attacks: multi-turn sequences whose individual fragments are innocuous in isolation yet assemble into a harmful...

Reactivating Alignment: Defending LLMs from Jailbreaks Via Intention-Aware Input-Output Matching

Large language models LLMs remain vulnerable to jailbreak attacks that conceal harmful intent within complex adversarial prompts. Existing defenses primarily rely on input perturbation or harmful-output suppression, but they rarely model where malicious intent resides, resulting in brittle...

Invisible Ink, Visible Lies: How Production Watermarking Causes LLMs to Hallucinate

Text watermarking helps identify AI-generated content, but its effect on factual reliability remains underexplored. In this paper, we study watermarking hallucination: factual errors induced or amplified by watermarking even when the required evidence is present in the context and the unwatermark...

High-Quality Data Do Not Mean Safe! Poisoning LLMs after Data Selection

Safety-aligned Large Language Models remain vulnerable to fine-tuning on small sets of harmful or benign-looking samples. However, prior studies typically assume that poisoned samples directly enter downstream fine-tuning, overlooking quality-based selection in practical training pipelines. To fi...

CITADEL: CWE-Guided Insertion of Hardware Trojans Via Analysis of DFG-Enabled LLMs

The increasing sophistication of Hardware Trojans HTs and system-level vulnerabilities poses significant risks to modern integrated circuits. However, constructing realistic HT scenarios, remains a substantial burden: researchers must manually analyze complex RTL structures, identify plausible...

Mitigating Private Data Leakage in LLMs with Whiteout

Modern large language models LLMs are trained on massive, largely unfiltered datasets, including content scraped from nearly every accessible website and user inputs. As a result, LLMs often memorize and reproduce personally sensitive information PSI such as birth dates, phone numbers, and home...

Backdoor Containment Via Expert Quarantine and Shutdown in LLMs

Backdoored large language models LLMs can behave normally on benign inputs while producing attacker-specified outputs under hidden triggers. Existing defenses span four stages--prior-training, in-training, post-training, and inference-time--and share one of two underlying strategies: either...

CodeMimicry: Exploiting Safety Generalization Lag in Large Language Models Via Structured Code Completion

Large language models have achieved remarkable capabilities across diverse domains, yet their safety alignment remains vulnerable to jailbreak attacks. In this work, we identify a previously underexplored failure mode - safety generalization lag - where alignment trained predominantly on natural...

Backdoor Purification for LoRA-Tuned LLMs Via Null-Space Projection

With the rapid adoption of large language models LLMs and parameter-efficient fine-tuning PEFT methods, the risk of backdoor attacks has become more severe. Existing backdoor purification methods typically rely on at least one of the strong assumptions, such as prior knowledge of triggers, access...

SKILLLITE: Evidence-Guided Malicious Skill Auditing with Compact LLMs

As LLM-based agents perform increasingly complex tasks, Agent Skills have emerged as a flexible mechanism for extending their capabilities. An Agent Skill packages task-specific instructions with executable components and auxiliary resources to provide specialized functionalities. However, the...

Render Before Reading: Visual Rendering As a Prompt Injection Defense

Large language models are vulnerable to prompt injection attacks, where third-party adversarial content can hijack the model's behavior. In this paper, we study the role played by the adversarial data's input modality, and identify a systematic asymmetry: multimodal LLMs are more likely to follow...

RAISE: Reinforcing Access Control Policy Synthesis in LLMs Via Symbolic Evaluation

Translating natural-language access-control requirements into policies requires careful reasoning about permissions, constraints, and exceptions, and even frontier LLMs often produce policies that violate the intended authorization semantics. We construct CedarInstruct, to our knowledge the first...

PentestChain: A Cost-Aware, MCP-Orchestrated Framework for Automated Penetration Testing with Free-Tier LLMs

AI-driven penetration testing has been demonstrated with premium frontier models such as GPT-4, but the per-engagement token cost makes continuous, automated testing unaffordable for the smaller organisations that need it most. This paper presents PentestChain, a ten-phase automated penetration...

TIER: Threat Implicitness Benchmark for Evaluating LLM Safety Behaviors

Current LLM safety benchmarks largely rely on binary metrics, overlooking how models respond to harmful prompts with varying threat implicitness. We introduce TIER, a Threat Implicitness Benchmark for behavioral safety evaluation of LLMs. TIER covers four risk domains and four threat levels, from...