Bastet: A Fine-Grained Expert-Labeled Dataset for DeFi Smart Contract Vulnerability Detection

Smart contract vulnerabilities in Decentralized Finance DeFi protocols resulted in over 1.49 billion USD in confirmed losses in 2024 alone, across 192 incidents 1. As LLM-based vulnerability detection emerges as a promising approach to address these threats, the quality of evaluation datasets has...

Towards Demystifying and Repairing LLM-In-The-Loop Vulnerabilities

Large Language ModelsLLMs have been actively integrated into modern software systems as critical components. LLM-in-the-loop vulnerabilities, where vulnerabilities are introduced by LLMs and their dependent downstream components, such as frameworks, introduce new risks. Although some benchmark...

VulnScout-C: A Lightweight Transformer for C Code Vulnerability Detection

Vulnerability detection in C programs is a critical challenge in software security. Although large language models LLMs achieve strong detection performance, their multi-billion-parameter scale makes them impractical for integration into development workflows requiring low latency and continuous...

Reading between the Code Lines: On the Use of Self-Admitted Technical Debt for Security Analysis

Static Analysis Tools SATs are central to security engineering activities, as they enable early identification of code weaknesses without requiring execution. However, their effectiveness is often limited by high false-positive rates and incomplete coverage of vulnerability classes. At the same...

AutoVulnPHP: LLM-Powered Two-Stage PHP Vulnerability Detection and Automated Localization

PHP's dominance in web development is undermined by security challenges: static analysis lacks semantic depth, causing high false positives; dynamic analysis is computationally expensive; and automated vulnerability localization suffers from coarse granularity and imprecise context. Additionally,...

VulnRepairEval: an Exploit-Based Evaluation Framework for Assessing Large Language Model Vulnerability Repair Capabilities

The adoption of Large Language Models LLMs for automated software vulnerability patching has shown promising outcomes on carefully curated evaluation sets. Nevertheless, existing datasets predominantly rely on superficial validation methods rather than exploit-based verification, leading to...

Resource-Efficient Automatic Software Vulnerability Assessment Via Knowledge Distillation and Particle Swarm Optimization

The increasing complexity of software systems has led to a surge in cybersecurity vulnerabilities, necessitating efficient and scalable solutions for vulnerability assessment. However, the deployment of large pre-trained models in real-world scenarios is hindered by their substantial computationa...

Out of Distribution, out of Luck: How Well Can LLMs Trained on Vulnerability Datasets Detect Top 25 CWE Weaknesses?

Automated vulnerability detection research has made substantial progress, yet its real-world impact remains limited. Current vulnerability datasets suffer from issues including label inaccuracy rates of 20-71%, extensive duplication, and poor coverage of critical CWE types. These issues create a...

Leveraging GPT-4 for Vulnerability-Witnessing Unit Test Generation

In the life-cycle of software development, testing plays a crucial role in quality assurance. Proper testing not only increases code coverage and prevents regressions but it can also ensure that any potential vulnerabilities in the software are identified and effectively fixed. However, creating...

Everything You Wanted to Know about LLM-Based Vulnerability Detection but Were Afraid to Ask

Large Language Models are a promising tool for automated vulnerability detection, thanks to their success in code generation and repair. However, despite widespread adoption, a critical question remains: Are LLMs truly effective at detecting real-world vulnerabilities? Current evaluations, which...