Alignment under Pressure: the Case for Informed Adversaries When Evaluating LLM Defenses

Large language models LLMs are rapidly deployed in real-world applications ranging from chatbots to agentic systems. Alignment is one of the main approaches used to defend against attacks such as prompt injection and jailbreaks. Recent defenses report near-zero Attack Success Rates ASR even again...

Improved Algorithms for Differentially Private Language Model Alignment

Language model alignment is crucial for ensuring that large language models LLMs align with human preferences, yet it often involves sensitive user data, raising significant privacy concerns. While prior work has integrated differential privacy DP with alignment techniques, their performance...