ToolHazard: Scaling Adversarial Environments for Security Evaluation and Alignment of LLM-Based Agents

Large language model LLM agents integrated with external tools are vulnerable to indirect prompt injections embedded in environmental states. However, existing studies largely rely on manually implemented or reused environments, stochastic LLM-based tool simulation, and predefined injection...

Why LLM Safety Guardrails Collapse after Fine-Tuning: a Similarity Analysis between Alignment and Fine-Tuning Datasets

Recent advancements in large language models LLMs have underscored their vulnerability to safety alignment jailbreaks, particularly when subjected to downstream fine-tuning. However, existing mitigation strategies primarily focus on reactively addressing jailbreak incidents after safety guardrail...