Can Federated Learning Safeguard Private Data in LLM Training? Vulnerabilities, Attacks, and Defense Evaluation

Fine-tuning large language models LLMs with local data is a widely adopted approach for organizations seeking to adapt LLMs to their specific domains. Given the shared characteristics in data across different organizations, the idea of collaboratively fine-tuning an LLM using data from multiple...

Investigating Security Implications of Automatically Generated Code on the Software Supply Chain

In recent years, various software supply chain SSC attacks have posed significant risks to the global community. Severe consequences may arise if developers integrate insecure code snippets that are vulnerable to SSC attacks into their products. Particularly, code generation techniques, such as...

CyberSOCEval: Benchmarking LLMs Capabilities for Malware Analysis and Threat Intelligence Reasoning

Today's cyber defenders are overwhelmed by a deluge of security alerts, threat intelligence signals, and shifting business context, creating an urgent need for AI systems to enhance operational security work. While Large Language Models LLMs have the potential to automate and scale Security...

Bi-GRPO: Bidirectional Optimization for Jailbreak Backdoor Injection on LLMs

With the rapid advancement of large language models LLMs, their robustness against adversarial manipulations, particularly jailbreak backdoor attacks, has become critically important. Existing approaches to embedding jailbreak triggers--such as supervised fine-tuning SFT, model editing, and...

DecipherGuard: Understanding and Deciphering Jailbreak Prompts for a Safer Deployment of Intelligent Software Systems

Intelligent software systems powered by Large Language Models LLMs are increasingly deployed in critical sectors, raising concerns about their safety during runtime. Through an industry-academic collaboration when deploying an LLM-powered virtual customer assistant, a critical software engineerin...

"Digital Camouflage": the LLVM Challenge in LLM-Based Malware Detection

Large Language Models LLMs have emerged as promising tools for malware detection by analyzing code semantics, identifying vulnerabilities, and adapting to evolving threats. However, their reliability under adversarial compiler-level obfuscation is yet to be discovered. In this study, we empirical...

Time-of-Check Time-of-Use Attacks Against LLMs

This is a nice piece of research: "Mind the Gap: Time-of-Check to Time-of-Use Vulnerabilities in LLM-Enabled Agents".: Abstract: Large Language Model LLM-enabled agents are rapidly emerging across a wide range of applications, but their deployment introduces vulnerabilities with security...

Evil Vizier: Vulnerabilities of LLM-Integrated XR Systems

Extended reality XR applications increasingly integrate Large Language Models LLMs to enhance user experience, scene understanding, and even generate executable XR content, and are often called "AI glasses". Despite these potential benefits, the integrated XR-LLM pipeline makes XR applications...

Important: Red Hat Security Advisory: Red Hat Enterprise Linux AI 1.5 (Intel Gaudi)

Red Hat Enterprise Linux AI 1.5 Intel Gaudi is now available. Red Hat® Enterprise Linux® AI is a foundation model platform to seamlessly develop, test, and run Granite family large language models LLMs for enterprise applications. This container provides NVIDIA hardware enablement and the...

Early Approaches to Adversarial Fine-Tuning for Prompt Injection Defense: a 2022 Study of GPT-3 and Contemporary Models

This paper documents early research conducted in 2022 on defending against prompt injection attacks in large language models, providing historical context for the evolution of this critical security domain. This research focuses on two adversarial attacks against Large Language Models LLMs: promp...

Large Language Models for Security Operations Centers: a Comprehensive Survey

Large Language Models LLMs have emerged as powerful tools capable of understanding and generating human-like text, offering transformative potential across diverse domains. The Security Operations Center SOC, responsible for safeguarding digital infrastructure, represents one of these domains. SO...

What You Code Is What We Prove: Translating BLE App Logic into Formal Models with LLMs for Vulnerability Detection

The application layer of Bluetooth Low Energy BLE is a growing source of security vulnerabilities, as developers often neglect to implement critical protections such as encryption, authentication, and freshness. While formal verification offers a principled way to check these properties, the manu...

Fraud Detection and Risk Assessment of Online Payment Transactions on E-Commerce Platforms Based on LLM and GCN Frameworks

With the rapid growth of e-commerce, online payment fraud has become increasingly complex, posing serious threats to financial security and consumer trust. Traditional detection methods often struggle to capture the intricate relational structures inherent in transactional data. This study presen...

PatchSeeker: Mapping NVD Records to Their Vulnerability-Fixing Commits with LLM Generated Commits and Embeddings

Software vulnerabilities pose serious risks to modern software ecosystems. While the National Vulnerability Database NVD is the authoritative source for cataloging these vulnerabilities, it often lacks explicit links to the corresponding Vulnerability-Fixing Commits VFCs. VFCs encode precise code...

Breaking Android with AI: a Deep Dive into LLM-Powered Exploitation

The rapid evolution of Artificial Intelligence AI and Large Language Models LLMs has opened up new opportunities in the area of cybersecurity, especially in the exploitation automation landscape and penetration testing. This study explores Android penetration testing automation using LLM-based...

Guided Reasoning in LLM-Driven Penetration Testing Using Structured Attack Trees

Recent advances in Large Language Models LLMs have driven interest in automating cybersecurity penetration testing workflows, offering the promise of faster and more consistent vulnerability assessment for enterprise systems. Existing LLM agents for penetration testing primarily rely on self-guid...

LLMs in Cybersecurity: Friend or Foe in the Human Decision Loop?

Large Language Models LLMs are transforming human decision-making by acting as cognitive collaborators. Yet, this promise comes with a paradox: while LLMs can improve accuracy, they may also erode independent reasoning, promote over-reliance and homogenize decisions. In this paper, we investigate...

Multimodal Prompt Injection Attacks: Risks and Defenses for Modern LLMs

Large Language Models LLMs have seen rapid adoption in recent years, with industries increasingly relying on them to maintain a competitive advantage. These models excel at interpreting user instructions and generating human-like responses, leading to their integration across diverse domains,...

Behind the Mask: Benchmarking Camouflaged Jailbreaks in Large Language Models

Large Language Models LLMs are increasingly vulnerable to a sophisticated form of adversarial prompting known as camouflaged jailbreaking. This method embeds malicious intent within seemingly benign language to evade existing safety mechanisms. Unlike overt attacks, these subtle prompts exploit...

Breaking to Build: a Threat Model of Prompt-Based Attacks for Securing LLMs

The proliferation of Large Language Models LLMs has introduced critical security challenges, where adversarial actors can manipulate input prompts to cause significant harm and circumvent safety alignments. These prompt-based attacks exploit vulnerabilities in a model's design, training, and...