Evaluating the Critical Risks of Amazon'S Nova Premier under the Frontier Model Safety Framework

Nova Premier is Amazon's most capable multimodal foundation model and teacher for model distillation. It processes text, images, and video with a one-million-token context window, enabling analysis of large codebases, 400-page documents, and 90-minute videos in a single prompt. We present the fir...

RedCoder: Automated Multi-Turn Red Teaming for Code LLMs

Large Language Models LLMs for code generation i.e., Code LLMs have demonstrated impressive capabilities in AI-assisted software development and testing. However, recent studies have shown that these models are prone to generating vulnerable or even malicious code under adversarial settings...

AIRTBench: Measuring Autonomous AI Red Teaming Capabilities in Language Models

We introduce AIRTBench, an AI red teaming benchmark for evaluating language models' ability to autonomously discover and exploit Artificial Intelligence and Machine Learning AI/ML security vulnerabilities. The benchmark consists of 70 realistic black-box capture-the-flag CTF challenges from the...

A Red Teaming Roadmap Towards System-Level Safety

Large Language Model LLM safeguards, which implement request refusals, have become a widely adopted mitigation strategy against misuse. At the intersection of adversarial machine learning and AI safety, safeguard red teaming has effectively identified critical vulnerabilities in state-of-the-art...

Towards Secure MLOps: Surveying Attacks, Mitigation Strategies, and Research Challenges

The rapid adoption of machine learning ML technologies has driven organizations across diverse sectors to seek efficient and reliable methods to accelerate model development-to-deployment. Machine Learning Operations MLOps has emerged as an integrative approach addressing these requirements by...

A Reward-Driven Automated Webshell Malicious-Code Generator for Red-Teaming

Whitepaper called A Reward-Driven Automated Webshell Malicious-Code Generator For Red-Teaming...

SafeCOMM: What about Safety Alignment in Fine-Tuned Telecom Large Language Models?

Fine-tuning large language models LLMs for telecom tasks and datasets is a common practice to adapt general-purpose models to the telecom domain. However, little attention has been paid to how this process may compromise model safety. Recent research has shown that even benign fine-tuning can...

Red-Teaming Text-To-Image Systems by Rule-Based Preference Modeling

Text-to-image T2I models raise ethical and safety concerns due to their potential to generate inappropriate or harmful images. Evaluating these models' security through red-teaming is vital, yet white-box approaches are limited by their need for internal access, complicating their use with...

MTSA: Multi-Turn Safety Alignment for LLMs through Multi-Round Red-Teaming

Whitepaper called MTSA: Multi-Turn Safety Alignment For LLMs Through Multi-Round Red-Teaming...

Security Practices in AI Development

What makes safety claims about general purpose AI systems such as large language models trustworthy? We show that rather than the capabilities of security tools such as alignment and red teaming procedures, it is security practices based on these tools that contributed to reconfiguring the image ...

Red Teaming the Mind of the Machine: a Systematic Evaluation of Prompt Injection and Jailbreak Vulnerabilities in LLMs

Large Language Models LLMs are increasingly integrated into consumer and enterprise applications. Despite their capabilities, they remain susceptible to adversarial attacks such as prompt injection and jailbreaks that override alignment safeguards. This paper provides a systematic investigation o...

Offensive Security for AI Systems: Concepts, Practices, and Applications

As artificial intelligence AI systems become increasingly adopted across sectors, the need for robust, proactive security strategies is paramount. Traditional defensive measures often fall short against the unique and evolving threats facing AI-driven technologies, making offensive security an...

The Automation Advantage in AI Red Teaming

This paper analyzes Large Language Model LLM security vulnerabilities based on data from Crucible, encompassing 214,271 attack attempts by 1,674 users across 30 LLM challenges. Our findings reveal automated approaches significantly outperform manual techniques 69.5% vs 47.6% success rate, despite...

Top Lessons from Take Command 2025

The live sessions may be over, but with every talk now available on demand, it’s the perfect time to reflect on the biggest takeaways from this year’s summit—and how they can help security teams move faster, act smarter, and take control of their attack surface. From red teaming tactics to...

Why AI Systems Need Red Teaming Now More Than Ever

AI systems are becoming a huge part of our lives, but they are not perfect. Red teaming helps…...

Take Command | Rapid7’s 2025 Cybersecurity Summit: First Look at Our Speaker Lineup

Take Command Summit 2025 is shaping up to be one of the most impactful cybersecurity events of the year, bringing together Rapid7’s own security experts alongside leading industry voices for a full day of insights into today’s evolving attack landscape. This virtual summit will offer actionable...

Vector Command Opportunistic Phishing Blog

Gone Phishing with Vector Command During one of our customer engagements, our red team will continuously attack your network to see if we can exploit a vulnerability. One of the tactics, techniques and proceduresTTPs we use is “Opportunistic Phishing”. First, let’s share a quick reminder about...

On Generative AI Security

Microsoft's AI Red Team just published "Lessons from Red Teaming 100 Generative AI Products." Their blog post lists "three takeaways," but the eight lessons in the report itself are more useful: 1. Understand what the system can do and where it is applied. 2. You don't have to compute gradients t...

How Red Teaming Helps Meet DORA Requirements

The Digital Operational Resilience Act DORA sets strict EU rules for financial institutions and IT providers, emphasizing strong…...

Hackers Abuse EDRSilencer Tool to Bypass Security and Hide Malicious Activity

Threat actors are attempting to abuse the open-source EDRSilencer tool as part of efforts to tamper endpoint detection and response EDR solutions and hide malicious activity. Trend Micro said it detected "threat actors attempting to integrate EDRSilencer in their attacks, repurposing it as a mean...