Attack Success Rate Is Not a Number: On Measurement Validity in Agentic AI Security Evaluation

Attack success rate ASR is the headline metric in nearly every published evaluation of attacks on, and defenses for, LLM agents. We argue that ASR as currently used is not a single quantity but a family of metrics parameterized by six design choices that papers seldom specify and never hold...