MorphMark: Flexible Adaptive Watermarking for Large Language Models

Watermarking by altering token sampling probabilities based on red-green list is a promising method for tracing the origin of text generated by large language models LLMs. However, existing watermark methods often struggle with a fundamental dilemma: improving watermark effectiveness the...

Fragments to Facts: Partial-Information Fragment Inference from LLMs

Large language models LLMs can leak sensitive training data through memorization and membership inference attacks. Prior work has primarily focused on strong adversarial assumptions, including attacker access to entire samples or long, ordered prefixes, leaving open the question of how vulnerable...

Benchmarking LLMs in an Embodied Environment for Blue Team Threat Hunting

As cyber threats continue to grow in scale and sophistication, blue team defenders increasingly require advanced tools to proactively detect and mitigate risks. Large Language Models LLMs offer promising capabilities for enhancing threat analysis. However, their effectiveness in real-world blue...

On Membership Inference Attacks in Knowledge Distillation

Nowadays, Large Language Models LLMs are trained on huge datasets, some including sensitive information. This poses a serious privacy concern because privacy attacks such as Membership Inference Attacks MIAs may detect this sensitive information. While knowledge distillation compresses LLMs into...

ProxyPrompt: Securing System Prompts against Prompt Extraction Attacks

The integration of large language models LLMs into a wide range of applications has highlighted the critical role of well-crafted system prompts, which require extensive testing and domain expertise. These prompts enhance task performance but may also encode sensitive information and filtering...

S3C2 Summit 2024-09: Industry Secure Software Supply Chain Summit

While providing economic and software development value, software supply chains are only as strong as their weakest link. Over the past several years, there has been an exponential increase in cyberattacks, specifically targeting vulnerable links in critical software supply chains. These attacks...

On Technique Identification and Threat-Actor Attribution Using LLMs and Embedding Models

Attribution of cyber-attacks remains a complex but critical challenge for cyber defenders. Currently, manual extraction of behavioral indicators from dense forensic documentation causes significant attribution delays, especially following major incidents at the international scale. This research...

Cape: Context-Aware Prompt Perturbation Mechanism with Differential Privacy

Large Language Models LLMs have gained significant popularity due to their remarkable capabilities in text understanding and generation. However, despite their widespread deployment in inference services such as ChatGPT, concerns about the potential leakage of sensitive user data have arisen...

Adversarial Attack on Large Language Models Using Exponentiated Gradient Descent

As Large Language Models LLMs are widely used, understanding them systematically is key to improving their safety and realizing their full potential. Although many models are aligned using techniques such as reinforcement learning from human feedback RLHF, they are still vulnerable to jailbreakin...

Red Teaming the Mind of the Machine: a Systematic Evaluation of Prompt Injection and Jailbreak Vulnerabilities in LLMs

Large Language Models LLMs are increasingly integrated into consumer and enterprise applications. Despite their capabilities, they remain susceptible to adversarial attacks such as prompt injection and jailbreaks that override alignment safeguards. This paper provides a systematic investigation o...

Federated Large Language Models: Feasibility, Robustness, Security and Future Directions

The integration of Large Language Models LLMs and Federated Learning FL presents a promising solution for joint training on distributed data while preserving privacy and addressing data silo issues. However, this emerging field, known as Federated Large Language Models FLLM, faces significant...

POISONCRAFT: Practical Poisoning of Retrieval-Augmented Generation for Large Language Models

Large language models LLMs have achieved remarkable success in various domains, primarily due to their strong capabilities in reasoning and generating human-like text. Despite their impressive performance, LLMs are susceptible to hallucinations, which can lead to incorrect or misleading outputs...

CVE-2025-30165

A flaw was found in vLLM's multi-node configuration, which is vulnerable to remote code execution due to unsafe deserialization using pickle over a ZeroMQ SUB socket. If the primary vLLM host is compromised, attackers can escalate privileges and execute arbitrary code on connected secondary hosts...

Winning at All Cost: a Small Environment for Eliciting Specification Gaming Behaviors in Large Language Models

This study reveals how frontier Large Language Models LLMs can "game the system" when faced with impossible situations, a critical security and alignment concern. Using a novel textual simulation approach, we presented three leading LLMs o1, o3-mini, and r1 with a tic-tac-toe scenario designed to...

NVIDIA TensorRT-LLM python executor code issue vulnerability

NVIDIA TensorRT-LLM is a high-performance inference acceleration library from NVIDIA for defining, optimizing, and executing inference in production environments for large language models LLMs. A code issue vulnerability exists in NVIDIA TensorRT-LLM that stems from insufficient data validation a...

Large Language Models Are Autonomous Cyber Defenders

Fast and effective incident response is essential to prevent adversarial cyberattacks. Autonomous Cyber Defense ACD aims to automate incident response through Artificial Intelligence AI agents that plan and execute actions. Most ACD approaches focus on single-agent scenarios and leverage...

A Proposal for Evaluating the Operational Risk for ChatBots Based on Large Language Models

The emergence of Generative AI Gen AI and Large Language Models LLMs has enabled more advanced chatbots capable of human-like interactions. However, these conversational agents introduce a broader set of operational risks that extend beyond traditional cybersecurity considerations. In this work, ...

RAP-SM: Robust Adversarial Prompt Via Shadow Models for Copyright Verification of Large Language Models

Recent advances in large language models LLMs have underscored the importance of safeguarding intellectual property rights through robust fingerprinting techniques. Traditional fingerprint verification approaches typically focus on a single model, seeking to improve the robustness of its...

Safeguard-By-Development: a Privacy-Enhanced Development Paradigm for Multi-Agent Collaboration Systems

Multi-agent collaboration systems MACS, powered by large language models LLMs, solve complex problems efficiently by leveraging each agent's specialization and communication between agents. However, the inherent exchange of information between agents and their interaction with external...

BadLingual: a Novel Lingual-Backdoor Attack against Large Language Models

In this paper, we present a new form of backdoor attack against Large Language Models LLMs: lingual-backdoor attacks. The key novelty of lingual-backdoor attacks is that the language itself serves as the trigger to hijack the infected LLMs to generate inflammatory speech. They enable the precise...