AiXamine: Unified Black-Box Evaluation of Cross-Dimensional Trade-Offs in LLM Safety, Security, and Privacy

The critical failure modes in deployed large language models LLMs are cross-dimensional: a model can score 99.3 in safety alignment while refusing one in three benign queries, or improve across every capability metric while losing 21 points in privacy. Existing evaluation frameworks that assess...

AutoRISE: Agent-Driven Strategy Evolution for Red-Teaming Large Language Models

Automated red-teaming methods for large language models typically optimize attack prompts within a fixed, human-designed strategy, leaving the attack strategy itself unchanged. We instead optimize the strategy. We propose AutoRISE, a method that searches over executable attack programs rather tha...

Towards Unveiling Vulnerabilities of Large Reasoning Models in Machine Unlearning

Large language models LLMs possess strong semantic understanding, driving significant progress in data mining applications. This is further enhanced by large reasoning models LRMs, which provide explicit multi-step reasoning traces. On the other hand, the growing need for the right to be forgotte...

Pushing the Limits of Safety: a Technical Report on the ATLAS Challenge 2025

Multimodal Large Language Models MLLMs have enabled transformative advancements across diverse applications but remain susceptible to safety threats, especially jailbreak attacks that induce harmful outputs. To systematically evaluate and improve their safety, we organized the Adversarial Testing...