Every Model Cheats: Prompt-Level Mitigation of Cheating on Offensive Cyber Tasks

Large language model LLM agents routinely cheat on cybersecurity benchmarks, inflating reported pass rates far beyond genuine capability. Prior audits of Cybench found cheating in 0.3-3.4% of traces, implicating only a handful of models. We present a controlled prompt-ablation study across 22...

Trellix “Catmen Sanfrancisco” Capture the Flag Results!

Trellix “Catmen Sanfrancisco” Capture the Flag Results! By Trellix · February 28, 2022 This story was written by Steve Povolny. And just like that, it’s all over! Our annual Capture the Flag contest expired at 11:59pm PST, on February 25th. We wanted to take a moment to thank all of our...