Winning at All Cost: a Small Environment for Eliciting Specification Gaming Behaviors in Large Language Models

This study reveals how frontier Large Language Models LLMs can "game the system" when faced with impossible situations, a critical security and alignment concern. Using a novel textual simulation approach, we presented three leading LLMs o1, o3-mini, and r1 with a tic-tac-toe scenario designed to...

OBLIVIATE: Robust and Practical Machine Unlearning for Large Language Models

Large language models LLMs trained over extensive corpora risk memorizing sensitive, copyrighted, or toxic content. To address this, we propose OBLIVIATE, a robust unlearning framework that removes targeted data while preserving model utility. The framework follows a structured process: extractin...

Weaponizing Language Models for Cybersecurity Offensive Operations: Automating Vulnerability Assessment Report Validation; a Review Paper

This, with the ever-increasing sophistication of cyberwar, calls for novel solutions. In this regard, Large Language Models LLMs have emerged as a highly promising tool for defensive and offensive cybersecurity-related strategies. While existing literature has focused much on the defensive use of...

Large Language Models Are Autonomous Cyber Defenders

Fast and effective incident response is essential to prevent adversarial cyberattacks. Autonomous Cyber Defense ACD aims to automate incident response through Artificial Intelligence AI agents that plan and execute actions. Most ACD approaches focus on single-agent scenarios and leverage...

RAP-SM: Robust Adversarial Prompt Via Shadow Models for Copyright Verification of Large Language Models

Recent advances in large language models LLMs have underscored the importance of safeguarding intellectual property rights through robust fingerprinting techniques. Traditional fingerprint verification approaches typically focus on a single model, seeking to improve the robustness of its...

NVIDIA TensorRT-LLM python executor code issue vulnerability

NVIDIA TensorRT-LLM is a high-performance inference acceleration library from NVIDIA for defining, optimizing, and executing inference in production environments for large language models LLMs. A code issue vulnerability exists in NVIDIA TensorRT-LLM that stems from insufficient data validation a...

A Proposal for Evaluating the Operational Risk for ChatBots Based on Large Language Models

The emergence of Generative AI Gen AI and Large Language Models LLMs has enabled more advanced chatbots capable of human-like interactions. However, these conversational agents introduce a broader set of operational risks that extend beyond traditional cybersecurity considerations. In this work, ...

Safeguard-By-Development: a Privacy-Enhanced Development Paradigm for Multi-Agent Collaboration Systems

Multi-agent collaboration systems MACS, powered by large language models LLMs, solve complex problems efficiently by leveraging each agent's specialization and communication between agents. However, the inherent exchange of information between agents and their interaction with external...

MergeGuard: Efficient Thwarting of Trojan Attacks in Machine Learning Models

This paper proposes MergeGuard, a novel methodology for mitigation of AI Trojan attacks. Trojan attacks on AI models cause inputs embedded with triggers to be misclassified to an adversary's target class, posing a significant threat to model usability trained by an untrusted third party. The core...

BadLingual: a Novel Lingual-Backdoor Attack against Large Language Models

In this paper, we present a new form of backdoor attack against Large Language Models LLMs: lingual-backdoor attacks. The key novelty of lingual-backdoor attacks is that the language itself serves as the trigger to hijack the infected LLMs to generate inflammatory speech. They enable the precise...

The Steganographic Potentials of Language Models

The potential for large language models LLMs to hide messages within plain text steganography poses a challenge to detection and thwarting of unaligned AI agents, and undermines faithfulness of LLMs reasoning. We explore the steganographic capabilities of LLMs fine-tuned via reinforcement learnin...

Detecting Quishing Attacks with Machine Learning Techniques through QR Code Analysis

The rise of QR code based phishing "Quishing" poses a growing cybersecurity threat, as attackers increasingly exploit QR codes to bypass traditional phishing defenses. Existing detection methods predominantly focus on URL analysis, which requires the extraction of the QR code payload, and may...

Bridging Expertise Gaps: the Role of LLMs in Human-AI Collaboration for Cybersecurity

This study investigates whether large language models LLMs can function as intelligent collaborators to bridge expertise gaps in cybersecurity decision-making. We examine two representative tasks-phishing email detection and intrusion detection-that differ in data modality, cognitive complexity,...

LLMs' Suitability for Network Security: a Case Study of STRIDE Threat Modeling

Artificial Intelligence AI is expected to be an integral part of next-generation AI-native 6G networks. With the prevalence of AI, researchers have identified numerous use cases of AI in network security. However, there are almost nonexistent studies that analyze the suitability of Large Language...

Towards Dataset Copyright Evasion Attack against Personalized Text-To-Image Diffusion Models

Text-to-image T2I diffusion models have rapidly advanced, enabling high-quality image generation conditioned on textual prompts. However, the growing trend of fine-tuning pre-trained models for personalization raises serious concerns about unauthorized dataset usage. To combat this, dataset...

Antifragility of RIS-Assisted Communication Systems under Jamming Attacks

Antifragility of communication systems is defined as measure of benefits gained from the adverse events and variability of its environment. In this paper, we introduce the notion of antifragility in Reconfigurable Intelligent Surface RIS assisted communication systems affected by a jamming attack...

Towards a Standardized Methodology and Dataset for Evaluating LLM-Based Digital Forensic Timeline Analysis

Large language models LLMs have seen widespread adoption in many domains including digital forensics. While prior research has largely centered on case studies and examples demonstrating how LLMs can assist forensic investigations, deeper explorations remain limited, i.e., a standardized approach...

Unveiling the Landscape of LLM Deployment in the Wild: an Empirical Study

Background: Large language models LLMs are increasingly deployed via open-source and commercial frameworks, enabling individuals and organizations to self-host advanced AI capabilities. However, insecure defaults and misconfigurations often expose LLM services to the public Internet, posing...

RepliBench: Evaluating the Autonomous Replication Capabilities of Language Model Agents

Uncontrollable autonomous replication of language model agents poses a critical safety risk. To better understand this risk, we introduce RepliBench, a suite of evaluations designed to measure autonomous replication capabilities. RepliBench is derived from a decomposition of these capabilities...

A Comprehensive Analysis of Adversarial Attacks against Spam Filters

Deep learning has revolutionized email filtering, which is critical to protect users from cyber threats such as spam, malware, and phishing. However, the increasing sophistication of adversarial attacks poses a significant challenge to the effectiveness of these filters. This study investigates t...