AI-Powered Villager Pen Testing Tool Hits 11,000 PyPI Downloads Amid Abuse Concerns

A new artificial intelligence AI-powered penetration testing tool linked to a China-based company has attracted nearly 11,000 downloads on the Python Package Index PyPI repository, raising concerns that it could be repurposed by cybercriminals for malicious purposes. Dubbed Villager, the framewor...

offensiveinterview

It is an offensive tool for penetration testing and red teaming. The repository contains a collection of interview questions to screen offensive red team/pentest candidates, categorized into open-ended, knowledge-based, and scenario-based questions. The questions cover various topics such as...

From Firewalls to Frontiers: AI Red-Teaming Is a Domain-Specific Evolution of Cyber Red-Teaming

A red team simulates adversary attacks to help defenders find effective strategies to defend their systems in a real-world operational setting. As more enterprise systems adopt AI, red-teaming will need to evolve to address the unique vulnerabilities and risks posed by AI systems. We take the...

Amazon Nova AI Challenge -- Trusted AI: Advancing Secure, AI-Assisted Software Development

AI systems for software development are rapidly gaining prominence, yet significant challenges remain in ensuring their safety. To address this, Amazon launched the Trusted AI track of the Amazon Nova AI Challenge, a global competition among 10 university teams to drive advances in secure AI. In...

Prompt Optimization and Evaluation for LLM Automated Red Teaming

Applications that use Large Language Models LLMs are becoming widespread, making the identification of system vulnerabilities increasingly important. Automated Red Teaming accelerates this effort by using an LLM to generate and execute attacks against target systems. Attack generators are evaluat...

Security Challenges in AI Agent Deployment: Insights from a Large Scale Public Competition

Recent advances have enabled LLM-powered AI agents to autonomously execute complex tasks by combining language model reasoning with tools, memory, and web access. But can these systems be trusted to follow deployment policies in realistic environments, especially under attack? To investigate, we...

PurpCode: Reasoning for Safer Code Generation

We introduce PurpCode, the first post-training recipe for training safe code reasoning models towards generating secure code and defending against malicious cyberactivities. PurpCode trains a reasoning model in two stages: i Rule Learning, which explicitly teaches the model to reference cybersafe...

DREAM: Scalable Red Teaming for Text-To-Image Generative Systems Via Distribution Modeling

Despite the integration of safety alignment and external filters, text-to-image T2I generative models are still susceptible to producing harmful content, such as sexual or violent imagery. This raises serious concerns about unintended exposure and potential misuse. Red teaming, which aims to...

Key Takeaways from the Take Command Summit 2025: Outpacing the Adversary – Red Teaming in a Complex Threat Landscape

The evolving complexity of modern infrastructures calls for more than traditional pen testing. In this session from the Take Command 2025 Virtual Cybersecurity Summit, red team experts shared how organizations are using continuous testing to outpace attackers — and better prepare their teams to...

PRM-Free Security Alignment of Large Models Via Red Teaming and Adversarial Training

Large Language Models LLMs have demonstrated remarkable capabilities across diverse applications, yet they pose significant security risks that threaten their safe deployment in critical domains. Current security alignment methodologies predominantly rely on Process Reward Models PRMs to evaluate...

Evaluating the Critical Risks of Amazon'S Nova Premier under the Frontier Model Safety Framework

Nova Premier is Amazon's most capable multimodal foundation model and teacher for model distillation. It processes text, images, and video with a one-million-token context window, enabling analysis of large codebases, 400-page documents, and 90-minute videos in a single prompt. We present the fir...

Red Teaming AI Red Teaming

Red teaming has evolved from its origins in military applications to become a widely adopted methodology in cybersecurity and AI. In this paper, we take a critical look at the practice of AI red teaming. We argue that despite its current popularity in AI governance, there exists a significant gap...

RedCoder: Automated Multi-Turn Red Teaming for Code LLMs

Large Language Models LLMs for code generation i.e., Code LLMs have demonstrated impressive capabilities in AI-assisted software development and testing. However, recent studies have shown that these models are prone to generating vulnerable or even malicious code under adversarial settings...

AIRTBench: Measuring Autonomous AI Red Teaming Capabilities in Language Models

We introduce AIRTBench, an AI red teaming benchmark for evaluating language models' ability to autonomously discover and exploit Artificial Intelligence and Machine Learning AI/ML security vulnerabilities. The benchmark consists of 70 realistic black-box capture-the-flag CTF challenges from the...

A Red Teaming Roadmap Towards System-Level Safety

Large Language Model LLM safeguards, which implement request refusals, have become a widely adopted mitigation strategy against misuse. At the intersection of adversarial machine learning and AI safety, safeguard red teaming has effectively identified critical vulnerabilities in state-of-the-art...

Towards Secure MLOps: Surveying Attacks, Mitigation Strategies, and Research Challenges

The rapid adoption of machine learning ML technologies has driven organizations across diverse sectors to seek efficient and reliable methods to accelerate model development-to-deployment. Machine Learning Operations MLOps has emerged as an integrative approach addressing these requirements by...

A Reward-Driven Automated Webshell Malicious-Code Generator for Red-Teaming

Whitepaper called A Reward-Driven Automated Webshell Malicious-Code Generator For Red-Teaming...

SafeCOMM: What about Safety Alignment in Fine-Tuned Telecom Large Language Models?

Fine-tuning large language models LLMs for telecom tasks and datasets is a common practice to adapt general-purpose models to the telecom domain. However, little attention has been paid to how this process may compromise model safety. Recent research has shown that even benign fine-tuning can...

Red-Teaming Text-To-Image Systems by Rule-Based Preference Modeling

Text-to-image T2I models raise ethical and safety concerns due to their potential to generate inappropriate or harmful images. Evaluating these models' security through red-teaming is vital, yet white-box approaches are limited by their need for internal access, complicating their use with...

MTSA: Multi-Turn Safety Alignment for LLMs through Multi-Round Red-Teaming

Whitepaper called MTSA: Multi-Turn Safety Alignment For LLMs Through Multi-Round Red-Teaming...