Unveiling the Landscape of LLM Deployment in the Wild: an Empirical Study

Background: Large language models LLMs are increasingly deployed via open-source and commercial frameworks, enabling individuals and organizations to self-host advanced AI capabilities. However, insecure defaults and misconfigurations often expose LLM services to the public Internet, posing...

A Survey on Privacy Risks and Protection in Large Language Models

Although Large Language Models LLMs have become increasingly integral to diverse applications, their capabilities raise significant privacy concerns. This survey offers a comprehensive overview of privacy risks associated with LLMs and examines current solutions to mitigate these challenges. Firs...

Good News for Script Kiddies? Evaluating Large Language Models for Automated Exploit Generation

Large Language Models LLMs have demonstrated remarkable capabilities in code-related tasks, raising concerns about their potential for automated exploit generation AEG. This paper presents the first systematic study on LLMs' effectiveness in AEG, evaluating both their cooperativeness and technica...

From Texts to Shields: Convergence of Large Language Models and Cybersecurity

This report explores the convergence of large language models LLMs and cybersecurity, synthesizing interdisciplinary insights from network security, artificial intelligence, formal methods, and human-centered design. It examines emerging applications of LLMs in software and network security, 5G...

Can Differentially Private Fine-Tuning LLMs Protect against Privacy Attacks?

Fine-tuning large language models LLMs has become an essential strategy for adapting them to specialized tasks; however, this process introduces significant privacy challenges, as sensitive training data may be inadvertently memorized and exposed. Although differential privacy DP offers strong...

OET: Optimization-Based Prompt Injection Evaluation Toolkit

Large Language Models LLMs have demonstrated remarkable capabilities in natural language understanding and generation, enabling their widespread adoption across various domains. However, their susceptibility to prompt injection attacks poses significant security risks, as adversarial inputs can...

LASHED: LLMs and Static Hardware Analysis for Early Detection of RTL Bugs

While static analysis is useful in detecting early-stage hardware security bugs, its efficacy is limited because it requires information to form checks and is often unable to explain the security impact of a detected vulnerability. Large Language Models can be useful in filling these gaps by...

An Empirical Study on the Effectiveness of Large Language Models for Binary Code Understanding

Binary code analysis plays a pivotal role in the field of software security and is widely used in tasks such as software maintenance, malware detection, software vulnerability discovery, patch analysis, etc. However, unlike source code, reverse engineers face significant challenges in understandi...

Unlocking User-Oriented Pages: Intention-Driven Black-Box Scanner for Real-World Web Applications

Black-box scanners have played a significant role in detecting vulnerabilities for web applications. A key focus in current black-box scanning is increasing test coverage i.e., accessing more web pages. However, since many web applications are user-oriented, some deep pages can only be accessed...

XBreaking: Explainable Artificial Intelligence for Jailbreaking LLMs

Large Language Models are fundamental actors in the modern IT landscape dominated by AI solutions. However, security threats associated with them might prevent their reliable adoption in critical application scenarios such as government organizations and medical institutions. For this reason,...

Hoist with His Own Petard: Inducing Guardrails to Facilitate Denial-Of-Service Attacks on Retrieval-Augmented Generation of LLMs

Whitepaper called Hoist With His Own Petard: Inducing Guardrails To Facilitate Denial-Of-Service Attacks On Retrieval-Augmented Generation Of LLMs...

Guard Against GenAI and LLM Risks from Development to Deployment with Qualys TotalAI

Artificial intelligence is fundamentally reshaping the enterprise. From automating customer service to accelerating code generation, large language models LLMs are rapidly becoming embedded in how businesses operate and compete. But as organizations embrace this innovation, they are also opening...

Robustness Via Referencing: Defending against Prompt Injection Attacks by Referencing the Executed Instruction

Large language models LLMs have demonstrated impressive performance and have come to dominate the field of natural language processing NLP across various tasks. However, due to their strong instruction-following capabilities and inability to distinguish between instructions and data content, LLMs...

Token-Efficient Prompt Injection Attack: Provoking Cessation in LLM Reasoning Via Adaptive Token Compression

While reasoning large language models LLMs demonstrate remarkable performance across various tasks, they also contain notable security vulnerabilities. Recent research has uncovered a "thinking-stopped" vulnerability in DeepSeek-R1, where model-generated reasoning tokens can forcibly interrupt th...

Enhancing Leakage Attacks on Searchable Symmetric Encryption Using LLM-Based Synthetic Data Generation

Searchable Symmetric Encryption SSE enables efficient search capabilities over encrypted data, allowing users to maintain privacy while utilizing cloud storage. However, SSE schemes are vulnerable to leakage attacks that exploit access patterns, search frequency, and volume information. Existing...

Prefill-Based Jailbreak: a Novel Approach of Bypassing LLM Safety Boundary

Large Language Models LLMs are designed to generate helpful and safe content. However, adversarial attacks, commonly referred to as jailbreak, can bypass their safety protocols, prompting LLMs to generate harmful content or reveal sensitive data. Consequently, investigating jailbreak methodologie...

Hybrid Privacy Policy-Code Consistency Check Using Knowledge Graphs and LLMs

The increasing concern in user privacy misuse has accelerated research into checking consistencies between smartphone apps' declared privacy policies and their actual behaviors. Recent advances in Large Language Models LLMs have introduced promising techniques for semantic comparison, but these...

Graph of Attacks: Improved Black-Box and Interpretable Jailbreaks for LLMs

The challenge of ensuring Large Language Models LLMs align with societal standards is of increasing interest, as these models are still prone to adversarial jailbreaks that bypass their safety mechanisms. Identifying these vulnerabilities is crucial for enhancing the robustness of LLMs against su...

CipherBank: Exploring the Boundary of LLM Reasoning Capabilities through Cryptography Challenges

Large language models LLMs have demonstrated remarkable capabilities, especially the recent advancements in reasoning, such as o1 and o3, pushing the boundaries of AI. Despite these impressive achievements in mathematics and coding, the reasoning abilities of LLMs in domains requiring cryptograph...

LLMpatronous: Harnessing the Power of LLMs for Vulnerability Detection

Despite the transformative impact of Artificial Intelligence AI across various sectors, cyber security continues to rely on traditional static and dynamic analysis tools, hampered by high false positive rates and superficial code comprehension. While generative AI offers promising automation...