A Comprehensive Real-World Assessment of Audio Watermarking Algorithms: Will They Survive Neural Codecs?

We introduce the Robust Audio Watermarking Benchmark RAW-Bench, a benchmark for evaluating deep learning-based audio watermarking methods with standardized and systematic comparisons. To simulate real-world usage, we introduce a comprehensive audio attack pipeline with various distortions such as...

VideoMarkBench: Benchmarking Robustness of Video Watermarking

The rapid development of video generative models has led to a surge in highly realistic synthetic videos, raising ethical concerns related to disinformation and copyright infringement. Recently, video watermarking has been proposed as a mitigation strategy by embedding invisible marks into...

Capability-Based Scaling Laws for LLM Red-Teaming

As large language models grow in capability and agency, identifying vulnerabilities through red-teaming becomes vital for safe deployment. However, traditional prompt-engineering approaches may prove ineffective once red-teaming turns into a weak-to-strong problem, where target models surpass...

PandaGuard: Systematic Evaluation of LLM Safety against Jailbreaking Attacks

Large language models LLMs have achieved remarkable capabilities but remain vulnerable to adversarial prompts known as jailbreaks, which can bypass safety alignment and elicit harmful outputs. Despite growing efforts in LLM safety research, existing evaluations are often fragmented, focused on...

Fixing 7,400 Bugs for 1$: Cheap Crash-Site Program Repair

The rapid advancement of bug-finding techniques has led to the discovery of more vulnerabilities than developers can reasonably fix, creating an urgent need for effective Automated Program Repair APR methods. However, the complexity of modern bugs often makes precise root cause analysis difficult...

LAMDA: a Longitudinal Android Malware Benchmark for Concept Drift Analysis

Machine learning ML-based malware detection systems often fail to account for the dynamic nature of real-world training and test data distributions. In practice, these distributions evolve due to frequent changes in the Android ecosystem, adversarial development of new malware families, and the...

Benchmarking Poisoning Attacks against Retrieval-Augmented Generation

Retrieval-Augmented Generation RAG has proven effective in mitigating hallucinations in large language models by incorporating external knowledge during inference. However, this integration introduces new security vulnerabilities, particularly to poisoning attacks. Although prior work has explore...

CVE-2024-27508

Atheme 7.2.12 contains a memory leak vulnerability in /atheme/src/crypto-benchmark/main.c...

CVE-2024-31360

Cross-Site Request Forgery CSRF vulnerability in Coded Commerce, LLC Benchmark Email Lite.This issue affects Benchmark Email Lite: from n/a through 4.1...

CVE-2024-31922

Cross-Site Request Forgery CSRF vulnerability in Anton Aleksandrov WordPress Hosting Benchmark tool.This issue affects WordPress Hosting Benchmark tool: from n/a through 1.3.6...

CVE-2023-32114

SAP NetWeaver Change and Transport System - versions 702, 731, 740, 750, 751, 752, 753, 754, 755, 756, 757, allows an authenticated user with admin privileges to maliciously run a benchmark program repeatedly in intent to slowdown or make the server unavailable which may lead to a limited impact ...

Are Vision-Language Models Safe in the Wild? A Meme-Based Benchmark Study

Rapid deployment of vision-language models VLMs magnifies safety risks, yet most evaluations rely on artificial images. This study asks: How safe are current VLMs when confronted with meme images that ordinary users share? To investigate this question, we introduce MemeSafetyBench, a...

Evaluating the Efficacy of LLM Safety Solutions : the Palit Benchmark Dataset

Large Language Models LLMs are increasingly integrated into critical systems in industries like healthcare and finance. Users can often submit queries to LLM-enabled chatbots, some of which can enrich responses with information retrieved from internal databases storing sensitive data. This gives...

Improving LLM Outputs against Jailbreak Attacks with Expert Model Integration

Using LLMs in a production environment presents security challenges that include vulnerabilities to jailbreaks and prompt injections, which can result in harmful outputs for humans or the enterprise. The challenge is amplified when working within a specific domain, as topics generally accepted fo...

Benchmarking LLMs in an Embodied Environment for Blue Team Threat Hunting

As cyber threats continue to grow in scale and sophistication, blue team defenders increasingly require advanced tools to proactively detect and mitigate risks. Large Language Models LLMs offer promising capabilities for enhancing threat analysis. However, their effectiveness in real-world blue...

WASP: Benchmarking Web Agent Security against Prompt Injection Attacks

Autonomous UI agents powered by AI have tremendous potential to boost human productivity by automating routine tasks such as filing taxes and paying bills. However, a major challenge in unlocking their full potential is security, which is exacerbated by the agent's ability to take action on their...

GenoArmory: a Unified Evaluation Framework for Adversarial Attacks on Genomic Foundation Models

We propose the first unified adversarial attack benchmark for Genomic Foundation Models GFMs, named GenoArmory. Unlike existing GFM benchmarks, GenoArmory offers the first comprehensive evaluation framework to systematically assess the vulnerability of GFMs to adversarial attacks. Methodologicall...

AutoRAN: Weak-To-Strong Jailbreaking of Large Reasoning Models

This paper presents AutoRAN, the first automated, weak-to-strong jailbreak attack framework targeting large reasoning models LRMs. At its core, AutoRAN leverages a weak, less-aligned reasoning model to simulate the target model's high-level reasoning structures, generates narrative prompts, and...

CVE-2025-37841

In the Linux kernel, the following vulnerability has been resolved: pm: cpupower: bench: Prevent NULL dereference on malloc failure If malloc returns NULL due to low memory, 'config' pointer can be NULL. Add a check to prevent NULL dereference...

Enable AIDE

Advanced intrusion detection environment AIDE is an intrusion detection tool that checks the integrity of system files and directories and identifies those maliciously tampered with. In principle, the integrity check can be performed only after an AIDE benchmark database is constructed, which...