SEC-Bench: Automated Benchmarking of LLM Agents on Real-World Software Security Tasks

Rigorous security-focused evaluation of large language model LLM agents is imperative for establishing trust in their safe deployment throughout the software development lifecycle. However, existing benchmarks largely rely on synthetic challenges or simplified vulnerability datasets that fail to...

Organizational Adaptation to Generative AI in Cybersecurity: a Systematic Review

Cybersecurity organizations are adapting to GenAI integration through modified frameworks and hybrid operational processes, with success influenced by existing security maturity, regulatory requirements, and investments in human capital and infrastructure. This qualitative research employs...

SmartHome-Bench: a Comprehensive Benchmark for Video Anomaly Detection in Smart Homes Using Multi-Modal Large Language Models

Video anomaly detection VAD is essential for enhancing safety and security by identifying unusual events across different environments. Existing VAD benchmarks, however, are primarily designed for general-purpose scenarios, neglecting the specific characteristics of smart home applications. To...

SAVANT: Vulnerability Detection in Application Dependencies through Semantic-Guided Reachability Analysis

The integration of open-source third-party library dependencies in Java development introduces significant security risks when these libraries contain known vulnerabilities. Existing Software Composition Analysis SCA tools struggle to effectively detect vulnerable API usage from these libraries d...

deepSURF: Detecting Memory Safety Vulnerabilities in Rust through Fuzzing LLM-Augmented Harnesses

Although Rust ensures memory safety by default, it also permits the use of unsafe code, which can introduce memory safety vulnerabilities if misused. Unfortunately, existing tools for detecting memory bugs in Rust typically exhibit limited detection capabilities, inadequately handle Rust-specific...

Analyzing PDFs like Binaries: Adversarially Robust PDF Malware Analysis Via Intermediate Representation and Language Model

Malicious PDF files have emerged as a persistent threat and become a popular attack vector in web-based attacks. While machine learning-based PDF malware classifiers have shown promise, these classifiers are often susceptible to adversarial attacks, undermining their reliability. To address this...

Rubber Mallet: a Study of High Frequency Localized Bit Flips and Their Impact on Security

The increasing density of modern DRAM has heightened its vulnerability to Rowhammer attacks, which induce bit flips by repeatedly accessing specific memory rows. This paper presents an analysis of bit flip patterns generated by advanced Rowhammer techniques that bypass existing hardware defenses...

LLM Jailbreak Oracle

As large language models LLMs become increasingly deployed in safety-critical applications, the lack of systematic methods to assess their vulnerability to jailbreak attacks presents a critical security gap. We introduce the jailbreak oracle problem: given a model, prompt, and decoding strategy,...

I Know What You Said: Unveiling Hardware Cache Side-Channels in Local Large Language Model Inference

Large Language Models LLMs that can be deployed locally have recently gained popularity for privacy-sensitive tasks, with companies such as Meta, Google, and Intel playing significant roles in their development. However, the security of local LLMs through the lens of hardware cache side-channels...

SOFT: Selective Data Obfuscation for Protecting LLM Fine-Tuning against Membership Inference Attacks

Whitepaper called SOFT: Selective Data Obfuscation For Protecting LLM Fine-Tuning Against Membership Inference Attacks...

Uncovering Reliable Indicators: Improving IoC Extraction from Threat Reports

Indicators of Compromise IoCs are critical for threat detection and response, marking malicious activity across networks and systems. Yet, the effectiveness of automated IoC extraction systems is fundamentally limited by one key issue: the lack of high-quality ground truth. Current extraction too...

LLMail-Inject: a Dataset from a Realistic Adaptive Prompt Injection Challenge

Indirect Prompt Injection attacks exploit the inherent limitation of Large Language Models LLMs to distinguish between instructions and data in their inputs. Despite numerous defense proposals, the systematic evaluation against adaptive adversaries remains limited, even when successful attacks ca...

Expert-In-The-Loop Systems with Cross-Domain and In-Domain Few-Shot Learning for Software Vulnerability Detection

As cyber threats become more sophisticated, rapid and accurate vulnerability detection is essential for maintaining secure systems. This study explores the use of Large Language Models LLMs in software vulnerability assessment by simulating the identification of Python code with known Common...

Lean and Mean: How We Fine-Tuned a Small Language Model for Secret Detection in Code

Building an efficient small language model for cybersecurity, from data prep to deployment...

Auditing Black-Box LLM APIs with a Rank-Based Uniformity Test

As API access becomes a primary interface to large language models LLMs, users often interact with black-box systems that offer little transparency into the deployed model. To reduce costs or maliciously alter model behaviors, API providers may discreetly serve quantized or fine-tuned variants,...

Your Agent Can Defend Itself against Backdoor Attacks

Despite their growing adoption across domains, large language model LLM-powered agents face significant security risks from backdoor attacks during training and fine-tuning. These compromised agents can subsequently be manipulated to execute malicious operations when presented with specific...

Mono: Is Your "Clean" Vulnerability Dataset Really Solvable? Exposing and Trapping Undecidable Patches and Beyond

The quantity and quality of vulnerability datasets are essential for developing deep learning solutions to vulnerability-related tasks. Due to the limited availability of vulnerabilities, a common approach to building such datasets is analyzing security patches in source code. However, existing...

How Good LLM-Generated Password Policies Are?

Generative AI technologies, particularly Large Language Models LLMs, are rapidly being adopted across industry, academia, and government sectors, owing to their remarkable capabilities in natural language processing. However, despite their strengths, the inconsistency and unpredictability of LLM...

A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment

The remarkable success of Large Language Models LLMs has illuminated a promising pathway toward achieving Artificial General Intelligence for both academic and industrial communities, owing to their unprecedented performance across various applications. As LLMs continue to gain prominence in both...

MARVEL: Multi-Agent RTL Vulnerability Extraction Using Large Language Models

Hardware security verification is a challenging and time-consuming task. For this purpose, design engineers may utilize tools such as formal verification, linters, and functional simulation tests, coupled with analysis and a deep understanding of the hardware design being inspected. Large Languag...