Optimized Couplings for Watermarking Large Language Models

Large-language models LLMs are now able to produce text that is, in many cases, seemingly indistinguishable from human-generated content. This has fueled the development of watermarks that imprint a signal'' in LLM-generated text with minimal perturbation of an LLM's output. This paper provides a...

Improved Algorithms for Differentially Private Language Model Alignment

Language model alignment is crucial for ensuring that large language models LLMs align with human preferences, yet it often involves sensitive user data, raising significant privacy concerns. While prior work has integrated differential privacy DP with alignment techniques, their performance...

RuleGenie: SIEM Detection Rule Set Optimization

SIEM systems serve as a critical hub, employing rule-based logic to detect and respond to threats. Redundant or overlapping rules in SIEM systems lead to excessive false alerts, degrading analyst performance due to alert fatigue, and increase computational overhead and response latency for actual...

Security Steerability Is All You Need

The adoption of Generative AI GenAI in various applications inevitably comes with expanding the attack surface, combining new security threats along with the traditional ones. Consequently, numerous research and industrial initiatives aim to mitigate these security threats in GenAI by developing...

LATENT: LLM-Augmented Trojan Insertion and Evaluation Framework for Analog Netlist Topologies

Analog and mixed-signal A/MS integrated circuits ICs are integral to safety-critical applications. However, the globalization and outsourcing of A/MS ICs to untrusted third-party foundries expose them to security threats, particularly analog Trojans. Unlike digital Trojans which have been...

Large Language Model-Driven Security Assistant for Internet of Things Via Chain-Of-Thought

The rapid development of Internet of Things IoT technology has transformed people's way of life and has a profound impact on both production and daily activities. However, with the rapid advancement of IoT technology, the security of IoT devices has become an unavoidable issue in both research an...

An LLM-Based Self-Evolving Security Framework for 6G Space-Air-Ground Integrated Networks

Recently emerged 6G space-air-ground integrated networks SAGINs, which integrate satellites, aerial networks, and terrestrial communications, offer ubiquitous coverage for various mobile applications. However, the highly dynamic, open, and heterogeneous nature of SAGINs poses severe security...

Directed Greybox Fuzzing Via Large Language Model

Directed greybox fuzzing DGF focuses on efficiently reaching specific program locations or triggering particular behaviors, making it essential for tasks like vulnerability detection and crash reproduction. However, existing methods often suffer from path explosion and randomness in input mutatio...

Towards Effective Identification of Attack Techniques in Cyber Threat Intelligence Reports Using Large Language Models

This work evaluates the performance of Cyber Threat Intelligence CTI extraction methods in identifying attack techniques from threat reports available on the web using the MITRE ATT&CK framework. We analyse four configurations utilising state-of-the-art tools, including the Threat Report ATT&CK...

LLM Watermarking Using Mixtures and Statistical-To-Computational Gaps

Given a text, can we determine whether it was generated by a large language model LLM or by a human? A widely studied approach to this problem is watermarking. We propose an undetectable and elementary watermarking scheme in the closed setting. Also, in the harder open setting, where the adversar...

LLM Security: Vulnerabilities, Attacks, Defenses, and Countermeasures

As large language models LLMs continue to evolve, it is critical to assess the security threats and vulnerabilities that may arise both during their training phase and after models have been deployed. This survey seeks to define and categorize the various attacks targeting LLMs, distinguishing...

Spill the Beans: Exploiting CPU Cache Side-Channels to Leak Tokens from Large Language Models

Side-channel attacks on shared hardware resources increasingly threaten confidentiality, especially with the rise of Large Language Models LLMs. In this work, we introduce Spill The Beans, a novel application of cache side-channels to leak tokens generated by an LLM. By co-locating an attack...

vLLM 安全漏洞

vLLM is a vLLM open source high throughput and memory efficient reasoning and service engine for LLM. A security vulnerability exists in vLLM versions prior to 0.5.2 through 0.8.5, which stems from ZeroMQ could lead to denial of service and data exposure...

XBreaking: Explainable Artificial Intelligence for Jailbreaking LLMs

Large Language Models are fundamental actors in the modern IT landscape dominated by AI solutions. However, security threats associated with them might prevent their reliable adoption in critical application scenarios such as government organizations and medical institutions. For this reason,...

Applying Security Engineering to Prompt Injection Security

This seems like an important advance in LLM security against prompt injection: Google DeepMind has unveiled CaMeL CApabilities for MachinE Learning, a new approach to stopping prompt-injection attacks that abandons the failed strategy of having AI models police themselves. Instead, CaMeL treats...

The Hidden Risks of LLM-Generated Web Application Code: a Security-Centric Evaluation of Code Generation Capabilities in Large Language Models

The rapid advancement of Large Language Models LLMs has enhanced software development processes, minimizing the time and effort required for coding and enhancing developer productivity. However, despite their potential benefits, code generated by LLMs has been shown to generate insecure code in...

The Automation Advantage in AI Red Teaming

This paper analyzes Large Language Model LLM security vulnerabilities based on data from Crucible, encompassing 214,271 attack attempts by 1,674 users across 30 LLM challenges. Our findings reveal automated approaches significantly outperform manual techniques 69.5% vs 47.6% success rate, despite...

Leveraging LLM to Strengthen ML-Based Cross-Site Scripting Detection

According to the Open Web Application Security Project OWASP, Cross-Site Scripting XSS is a critical security vulnerability. Despite decades of research, XSS remains among the top 10 security vulnerabilities. Researchers have proposed various techniques to protect systems from XSS attacks, with...

Llama-3.1-FoundationAI-SecurityLLM-Base-8B Technical Report

As transformer-based large language models LLMs increasingly permeate society, they have revolutionized domains such as software engineering, creative writing, and digital arts. However, their adoption in cybersecurity remains limited due to challenges like scarcity of specialized training data a...

dify 安全漏洞

dify is an open source LLM application development platform from LangGenius Open Source. A security vulnerability exists in versions of dify prior to 1.3.0, which stems from a clickjacking vulnerability in the default settings that could lead to unauthorized operations...