OBLIVIATE: Robust and Practical Machine Unlearning for Large Language Models

Large language models LLMs trained over extensive corpora risk memorizing sensitive, copyrighted, or toxic content. To address this, we propose OBLIVIATE, a robust unlearning framework that removes targeted data while preserving model utility. The framework follows a structured process: extractin...

Weaponizing Language Models for Cybersecurity Offensive Operations: Automating Vulnerability Assessment Report Validation; a Review Paper

This, with the ever-increasing sophistication of cyberwar, calls for novel solutions. In this regard, Large Language Models LLMs have emerged as a highly promising tool for defensive and offensive cybersecurity-related strategies. While existing literature has focused much on the defensive use of...

Winning at All Cost: a Small Environment for Eliciting Specification Gaming Behaviors in Large Language Models

This study reveals how frontier Large Language Models LLMs can "game the system" when faced with impossible situations, a critical security and alignment concern. Using a novel textual simulation approach, we presented three leading LLMs o1, o3-mini, and r1 with a tic-tac-toe scenario designed to...

Large Language Models Are Autonomous Cyber Defenders

Fast and effective incident response is essential to prevent adversarial cyberattacks. Autonomous Cyber Defense ACD aims to automate incident response through Artificial Intelligence AI agents that plan and execute actions. Most ACD approaches focus on single-agent scenarios and leverage...

RAP-SM: Robust Adversarial Prompt Via Shadow Models for Copyright Verification of Large Language Models

Recent advances in large language models LLMs have underscored the importance of safeguarding intellectual property rights through robust fingerprinting techniques. Traditional fingerprint verification approaches typically focus on a single model, seeking to improve the robustness of its...

A Proposal for Evaluating the Operational Risk for ChatBots Based on Large Language Models

The emergence of Generative AI Gen AI and Large Language Models LLMs has enabled more advanced chatbots capable of human-like interactions. However, these conversational agents introduce a broader set of operational risks that extend beyond traditional cybersecurity considerations. In this work, ...

Safeguard-By-Development: a Privacy-Enhanced Development Paradigm for Multi-Agent Collaboration Systems

Multi-agent collaboration systems MACS, powered by large language models LLMs, solve complex problems efficiently by leveraging each agent's specialization and communication between agents. However, the inherent exchange of information between agents and their interaction with external...

NVIDIA TensorRT-LLM python executor code issue vulnerability

NVIDIA TensorRT-LLM is a high-performance inference acceleration library from NVIDIA for defining, optimizing, and executing inference in production environments for large language models LLMs. A code issue vulnerability exists in NVIDIA TensorRT-LLM that stems from insufficient data validation a...

BadLingual: a Novel Lingual-Backdoor Attack against Large Language Models

In this paper, we present a new form of backdoor attack against Large Language Models LLMs: lingual-backdoor attacks. The key novelty of lingual-backdoor attacks is that the language itself serves as the trigger to hijack the infected LLMs to generate inflammatory speech. They enable the precise...

The Steganographic Potentials of Language Models

The potential for large language models LLMs to hide messages within plain text steganography poses a challenge to detection and thwarting of unaligned AI agents, and undermines faithfulness of LLMs reasoning. We explore the steganographic capabilities of LLMs fine-tuned via reinforcement learnin...

Bridging Expertise Gaps: the Role of LLMs in Human-AI Collaboration for Cybersecurity

This study investigates whether large language models LLMs can function as intelligent collaborators to bridge expertise gaps in cybersecurity decision-making. We examine two representative tasks-phishing email detection and intrusion detection-that differ in data modality, cognitive complexity,...

LLMs' Suitability for Network Security: a Case Study of STRIDE Threat Modeling

Artificial Intelligence AI is expected to be an integral part of next-generation AI-native 6G networks. With the prevalence of AI, researchers have identified numerous use cases of AI in network security. However, there are almost nonexistent studies that analyze the suitability of Large Language...

Towards a Standardized Methodology and Dataset for Evaluating LLM-Based Digital Forensic Timeline Analysis

Large language models LLMs have seen widespread adoption in many domains including digital forensics. While prior research has largely centered on case studies and examples demonstrating how LLMs can assist forensic investigations, deeper explorations remain limited, i.e., a standardized approach...

Unveiling the Landscape of LLM Deployment in the Wild: an Empirical Study

Background: Large language models LLMs are increasingly deployed via open-source and commercial frameworks, enabling individuals and organizations to self-host advanced AI capabilities. However, insecure defaults and misconfigurations often expose LLM services to the public Internet, posing...

A Survey on Privacy Risks and Protection in Large Language Models

Although Large Language Models LLMs have become increasingly integral to diverse applications, their capabilities raise significant privacy concerns. This survey offers a comprehensive overview of privacy risks associated with LLMs and examines current solutions to mitigate these challenges. Firs...

Good News for Script Kiddies? Evaluating Large Language Models for Automated Exploit Generation

Large Language Models LLMs have demonstrated remarkable capabilities in code-related tasks, raising concerns about their potential for automated exploit generation AEG. This paper presents the first systematic study on LLMs' effectiveness in AEG, evaluating both their cooperativeness and technica...

Can Differentially Private Fine-Tuning LLMs Protect against Privacy Attacks?

Fine-tuning large language models LLMs has become an essential strategy for adapting them to specialized tasks; however, this process introduces significant privacy challenges, as sensitive training data may be inadvertently memorized and exposed. Although differential privacy DP offers strong...

From Texts to Shields: Convergence of Large Language Models and Cybersecurity

This report explores the convergence of large language models LLMs and cybersecurity, synthesizing interdisciplinary insights from network security, artificial intelligence, formal methods, and human-centered design. It examines emerging applications of LLMs in software and network security, 5G...

OET: Optimization-Based Prompt Injection Evaluation Toolkit

Large Language Models LLMs have demonstrated remarkable capabilities in natural language understanding and generation, enabling their widespread adoption across various domains. However, their susceptibility to prompt injection attacks poses significant security risks, as adversarial inputs can...

Unlocking User-Oriented Pages: Intention-Driven Black-Box Scanner for Real-World Web Applications

Black-box scanners have played a significant role in detecting vulnerabilities for web applications. A key focus in current black-box scanning is increasing test coverage i.e., accessing more web pages. However, since many web applications are user-oriented, some deep pages can only be accessed...