A Proposal for Evaluating the Operational Risk for ChatBots Based on Large Language Models

The emergence of Generative AI Gen AI and Large Language Models LLMs has enabled more advanced chatbots capable of human-like interactions. However, these conversational agents introduce a broader set of operational risks that extend beyond traditional cybersecurity considerations. In this work, ...

Weaponizing Language Models for Cybersecurity Offensive Operations: Automating Vulnerability Assessment Report Validation; a Review Paper

This, with the ever-increasing sophistication of cyberwar, calls for novel solutions. In this regard, Large Language Models LLMs have emerged as a highly promising tool for defensive and offensive cybersecurity-related strategies. While existing literature has focused much on the defensive use of...

Large Language Models Are Autonomous Cyber Defenders

Fast and effective incident response is essential to prevent adversarial cyberattacks. Autonomous Cyber Defense ACD aims to automate incident response through Artificial Intelligence AI agents that plan and execute actions. Most ACD approaches focus on single-agent scenarios and leverage...

NVIDIA TensorRT-LLM python executor code issue vulnerability

NVIDIA TensorRT-LLM is a high-performance inference acceleration library from NVIDIA for defining, optimizing, and executing inference in production environments for large language models LLMs. A code issue vulnerability exists in NVIDIA TensorRT-LLM that stems from insufficient data validation a...

LLMs' Suitability for Network Security: a Case Study of STRIDE Threat Modeling

Artificial Intelligence AI is expected to be an integral part of next-generation AI-native 6G networks. With the prevalence of AI, researchers have identified numerous use cases of AI in network security. However, there are almost nonexistent studies that analyze the suitability of Large Language...

BadLingual: a Novel Lingual-Backdoor Attack against Large Language Models

In this paper, we present a new form of backdoor attack against Large Language Models LLMs: lingual-backdoor attacks. The key novelty of lingual-backdoor attacks is that the language itself serves as the trigger to hijack the infected LLMs to generate inflammatory speech. They enable the precise...

Bridging Expertise Gaps: the Role of LLMs in Human-AI Collaboration for Cybersecurity

This study investigates whether large language models LLMs can function as intelligent collaborators to bridge expertise gaps in cybersecurity decision-making. We examine two representative tasks-phishing email detection and intrusion detection-that differ in data modality, cognitive complexity,...

The Steganographic Potentials of Language Models

The potential for large language models LLMs to hide messages within plain text steganography poses a challenge to detection and thwarting of unaligned AI agents, and undermines faithfulness of LLMs reasoning. We explore the steganographic capabilities of LLMs fine-tuned via reinforcement learnin...

MergeGuard: Efficient Thwarting of Trojan Attacks in Machine Learning Models

This paper proposes MergeGuard, a novel methodology for mitigation of AI Trojan attacks. Trojan attacks on AI models cause inputs embedded with triggers to be misclassified to an adversary's target class, posing a significant threat to model usability trained by an untrusted third party. The core...

Detecting Quishing Attacks with Machine Learning Techniques through QR Code Analysis

The rise of QR code based phishing "Quishing" poses a growing cybersecurity threat, as attackers increasingly exploit QR codes to bypass traditional phishing defenses. Existing detection methods predominantly focus on URL analysis, which requires the extraction of the QR code payload, and may...

RepliBench: Evaluating the Autonomous Replication Capabilities of Language Model Agents

Uncontrollable autonomous replication of language model agents poses a critical safety risk. To better understand this risk, we introduce RepliBench, a suite of evaluations designed to measure autonomous replication capabilities. RepliBench is derived from a decomposition of these capabilities...

Towards Dataset Copyright Evasion Attack against Personalized Text-To-Image Diffusion Models

Text-to-image T2I diffusion models have rapidly advanced, enabling high-quality image generation conditioned on textual prompts. However, the growing trend of fine-tuning pre-trained models for personalization raises serious concerns about unauthorized dataset usage. To combat this, dataset...

Unveiling the Landscape of LLM Deployment in the Wild: an Empirical Study

Background: Large language models LLMs are increasingly deployed via open-source and commercial frameworks, enabling individuals and organizations to self-host advanced AI capabilities. However, insecure defaults and misconfigurations often expose LLM services to the public Internet, posing...

Towards a Standardized Methodology and Dataset for Evaluating LLM-Based Digital Forensic Timeline Analysis

Large language models LLMs have seen widespread adoption in many domains including digital forensics. While prior research has largely centered on case studies and examples demonstrating how LLMs can assist forensic investigations, deeper explorations remain limited, i.e., a standardized approach...

Antifragility of RIS-Assisted Communication Systems under Jamming Attacks

Antifragility of communication systems is defined as measure of benefits gained from the adverse events and variability of its environment. In this paper, we introduce the notion of antifragility in Reconfigurable Intelligent Surface RIS assisted communication systems affected by a jamming attack...

A Comprehensive Analysis of Adversarial Attacks against Spam Filters

Deep learning has revolutionized email filtering, which is critical to protect users from cyber threats such as spam, malware, and phishing. However, the increasing sophistication of adversarial attacks poses a significant challenge to the effectiveness of these filters. This study investigates t...

A Survey on Privacy Risks and Protection in Large Language Models

Although Large Language Models LLMs have become increasingly integral to diverse applications, their capabilities raise significant privacy concerns. This survey offers a comprehensive overview of privacy risks associated with LLMs and examines current solutions to mitigate these challenges. Firs...

xAI Dev Leaks API Key for Private SpaceX, Tesla LLMs

An employee at Elon Musk's artificial intelligence company xAI leaked a private key on GitHub that for the past two months could have allowed anyone to query private xAI large language models LLMs which appear to have been custom made for working with internal data from Musk's companies, includin...

Good News for Script Kiddies? Evaluating Large Language Models for Automated Exploit Generation

Large Language Models LLMs have demonstrated remarkable capabilities in code-related tasks, raising concerns about their potential for automated exploit generation AEG. This paper presents the first systematic study on LLMs' effectiveness in AEG, evaluating both their cooperativeness and technica...

VIDSTAMP: a Temporally-Aware Watermark for Ownership and Integrity in Video Diffusion Models

The rapid rise of video diffusion models has enabled the generation of highly realistic and temporally coherent videos, raising critical concerns about content authenticity, provenance, and misuse. Existing watermarking approaches, whether passive, post-hoc, or adapted from image-based techniques...