721 matches found
CoTGuard: Using Chain-Of-Thought Triggering for Copyright Protection in Multi-Agent LLM Systems
As large language models LLMs evolve into autonomous agents capable of collaborative reasoning and task execution, multi-agent LLM systems have emerged as a powerful paradigm for solving complex problems. However, these systems pose new challenges for copyright protection, particularly when...
Strong Membership Inference Attacks on Massive Datasets and (Moderately) Large Language Models
State-of-the-art membership inference attacks MIAs typically require training many reference models, making it difficult to scale these attacks to large pre-trained language models LLMs. As a result, prior research has either relied on weaker attacks that avoid training reference models e.g.,...
One Model Transfer to All: on Robust Jailbreak Prompts Generation against LLMs
Safety alignment in large language models LLMs is increasingly compromised by jailbreak attacks, which can manipulate these models to generate harmful or unintended content. Investigating these attacks is crucial for uncovering model vulnerabilities. However, many existing jailbreak strategies fa...
Invisible Tokens, Visible Bills: the Urgent Need to Audit Hidden Operations in Opaque LLM Services
Whitepaper called Invisible Tokens, Visible Bills: The Urgent Need To Audit Hidden Operations In Opaque LLM Services...
ACSE-Eval: Can LLMs Threat Model Real-World Cloud Infrastructure?
While Large Language Models have shown promise in cybersecurity applications, their effectiveness in identifying security threats within cloud deployments remains unexplored. This paper introduces AWS Cloud Security Engineering Eval, a novel dataset for evaluating LLMs cloud security threat...
Mapping the Future of AI Security
AI security is one of the most pressing challenges facing the world today. Artificial intelligence is extraordinarily powerful, and, especially considering the advent of Agentic AI, growing more so by the day. But it is for this reason that securing it is so important. AI handles massive amounts ...
When Safety Detectors Aren'T Enough: a Stealthy and Effective Jailbreak Attack on LLMs Via Steganographic Techniques
Jailbreak attacks pose a serious threat to large language models LLMs by bypassing built-in safety mechanisms and leading to harmful outputs. Studying these attacks is crucial for identifying vulnerabilities and improving model security. This paper presents a systematic survey of jailbreak method...
Advancing Security with Digital Twins: a Comprehensive Survey
The proliferation of electronic devices has greatly transformed every aspect of human life, such as communication, healthcare, transportation, and energy. Unfortunately, the global electronics supply chain is vulnerable to various attacks, including piracy of intellectual properties, tampering,...
CAIN: Hijacking LLM-Humans Conversations Via a Two-Stage Malicious System Prompt Generation and Refining Framework
Large language models LLMs have advanced many applications, but are also known to be vulnerable to adversarial attacks. In this work, we introduce a novel security threat: hijacking AI-human conversations by manipulating LLMs' system prompts to produce malicious answers only to specific targeted...
Mitigating Fine-Tuning Risks in LLMs Via Safety-Aware Probing Optimization
The significant progress of large language models LLMs has led to remarkable achievements across numerous applications. However, their ability to generate harmful content has sparked substantial safety concerns. Despite the implementation of safety alignment techniques during the pre-training...
Unlearning Isn'T Deletion: Investigating Reversibility of Machine Unlearning in LLMs
Unlearning in large language models LLMs is intended to remove the influence of specific data, yet current evaluations rely heavily on token-level metrics such as accuracy and perplexity. We show that these metrics can be misleading: models often appear to forget, but their original behavior can ...
MTSA: Multi-Turn Safety Alignment for LLMs through Multi-Round Red-Teaming
Whitepaper called MTSA: Multi-Turn Safety Alignment For LLMs Through Multi-Round Red-Teaming...
CoTSRF: Utilize Chain of Thought As Stealthy and Robust Fingerprint of Large Language Models
Despite providing superior performance, open-source large language models LLMs are vulnerable to abusive usage. To address this issue, recent works propose LLM fingerprinting methods to identify the specific source LLMs behind suspect applications. However, these methods fail to provide stealthy...
DuFFin: a Dual-Level Fingerprinting Framework for LLMs IP Protection
Whitepaper called DuFFin: A Dual-Level Fingerprinting Framework For LLMs IP Protection...
Blind Spot Navigation: Evolutionary Discovery of Sensitive Semantic Concepts for LVLMs
Whitepaper called Blind Spot Navigation: Evolutionary Discovery Of Sensitive Semantic Concepts For LVLMs...
Scalable Defense against In-The-Wild Jailbreaking Attacks with Safety Context Retrieval
Large Language Models LLMs are known to be vulnerable to jailbreaking attacks, wherein adversaries exploit carefully engineered prompts to induce harmful or unethical responses. Such threats have raised critical concerns about the safety and reliability of LLMs in real-world deployment. While...
FragFake: a Dataset for Fine-Grained Detection of Edited Images with Vision Language Models
Fine-grained edited image detection of localized edits in images is crucial for assessing content authenticity, especially given that modern diffusion models and image editing methods can produce highly realistic manipulations. However, this domain faces three challenges: 1 Binary classifiers yie...
Are Vision-Language Models Safe in the Wild? A Meme-Based Benchmark Study
Rapid deployment of vision-language models VLMs magnifies safety risks, yet most evaluations rely on artificial images. This study asks: How safe are current VLMs when confronted with meme images that ordinary users share? To investigate this question, we introduce MemeSafetyBench, a...
Leveraging Large Language Models for Command Injection Vulnerability Analysis in Python: an Empirical Study on Popular Open-Source Projects
Command injection vulnerabilities are a significant security threat in dynamic languages like Python, particularly in widely used open-source projects where security issues can have extensive impact. With the proven effectiveness of Large Language ModelsLLMs in code-related tasks, such as testing...