Measuring Security without Fooling Ourselves: Why Benchmarking Agents Is Hard

The benchmarks used to evaluate AI agents in security-critical roles suffer from crucial weaknesses. Building on recent empirical evidence, we characterize three core challenges that undermine security evaluations: benchmark vulnerabilities, temporal staleness, and runtime uncertainty. We then...

LENS-DF: Deepfake Detection and Temporal Localization for Long-Form Noisy Speech

This study introduces LENS-DF, a novel and comprehensive recipe for training and evaluating audio deepfake detection and temporal localization under complicated and realistic audio conditions. The generation part of the recipe outputs audios from the input dataset with several critical...

FLARE: Feature-Based Lightweight Aggregation for Robust Evaluation of IoT Intrusion Detection

The proliferation of Internet of Things IoT devices has expanded the attack surface, necessitating efficient intrusion detection systems IDSs for network protection. This paper presents FLARE, a feature-based lightweight aggregation for robust evaluation of IoT intrusion detection to address the...