30 matches found
Amazon Nova AI Challenge -- Trusted AI: Advancing Secure, AI-Assisted Software Development
AI systems for software development are rapidly gaining prominence, yet significant challenges remain in ensuring their safety. To address this, Amazon launched the Trusted AI track of the Amazon Nova AI Challenge, a global competition among 10 university teams to drive advances in secure AI. In...
ScamAgents: How AI Agents Can Simulate Human-Level Scam Calls
Large Language Models LLMs have demonstrated impressive fluency and reasoning capabilities, but their potential for misuse has raised growing concern. In this paper, we present ScamAgent, an autonomous multi-turn agent built on top of LLMs, capable of generating highly realistic scam call scripts...
Large Reasoning Models Are Autonomous Jailbreak Agents
Jailbreaking -- bypassing built-in safety mechanisms in AI models -- has traditionally required complex technical procedures or specialized human expertise. In this study, we show that the persuasive capabilities of large reasoning models LRMs simplify and scale jailbreaking, converting it into a...
RedCoder: Automated Multi-Turn Red Teaming for Code LLMs
Large Language Models LLMs for code generation i.e., Code LLMs have demonstrated impressive capabilities in AI-assisted software development and testing. However, recent studies have shown that these models are prone to generating vulnerable or even malicious code under adversarial settings...
Echo Chamber Jailbreak Tricks LLMs Like OpenAI and Google into Generating Harmful Content
Cybersecurity researchers are calling attention to a new jailbreaking method called Echo Chamber that could be leveraged to trick popular large language models LLMs into generating undesirable responses, irrespective of the safeguards put in place. "Unlike traditional jailbreaks that rely on...
MTSA: Multi-Turn Safety Alignment for LLMs through Multi-Round Red-Teaming
Whitepaper called MTSA: Multi-Turn Safety Alignment For LLMs Through Multi-Round Red-Teaming...
Are Vision-Language Models Safe in the Wild? A Meme-Based Benchmark Study
Rapid deployment of vision-language models VLMs magnifies safety risks, yet most evaluations rely on artificial images. This study asks: How safe are current VLMs when confronted with meme images that ordinary users share? To investigate this question, we introduce MemeSafetyBench, a...
X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agents
Multi-turn interactions with language models LMs pose critical safety risks, as harmful intent can be strategically spread across exchanges. Yet, the vast majority of prior work has focused on single-turn safety, while adaptability and diversity remain among the key challenges of multi-turn...
New AI Jailbreak Method 'Bad Likert Judge' Boosts Attack Success Rates by Over 60%
Cybersecurity researchers have shed light on a new jailbreak technique that could be used to get past a large language model's LLM safety guardrails and produce potentially harmful or malicious responses. The multi-turn aka many-shot attack strategy has been codenamed Bad Likert Judge by Palo Alt...
