RvB: Automating AI System Hardening Via Iterative Red-Blue Games

The dual offensive and defensive utility of Large Language Models LLMs highlights a critical gap in AI security: the lack of unified frameworks for dynamic, iterative adversarial adaptation hardening. To bridge this gap, we propose the Red Team vs. Blue Team RvB framework, formulated as a...

Securing agentic AI: Your guide to the Microsoft Ignite sessions catalog

Security is a core focus at Microsoft Ignite 2025, reflected in dedicated sessions and hands-on experiences designed for security professionals and leaders. Whether you’re shaping strategy or working on the front lines, Microsoft Ignite offers direct access to the latest advancements and practica...

Securing agentic AI: Your guide to the Microsoft Ignite sessions catalog

Security is a core focus at Microsoft Ignite 2025, reflected in dedicated sessions and hands-on experiences designed for security professionals and leaders. Whether you’re shaping strategy or working on the front lines, Microsoft Ignite offers direct access to the latest advancements and practica...

Why AI Systems Use Vector Databases to Improve Performance and Reduce Costs

...

The Age of Integrity

We need to talk about data integrity. Narrowly, the term refers to ensuring that data isn’t tampered with, either in transit or in storage. Manipulating account balances in bank databases, removing entries from criminal records, and murder by removing notations about allergies from medical record...

AI Vulnerability Finding

Microsoft is reporting that its AI systems are able to find new vulnerabilities in source code: Microsoft discovered eleven vulnerabilities in GRUB2, including integer and buffer overflows in filesystem parsers, command flaws, and a side-channel in cryptographic comparison. Additionally, 9 buffer...

The vulnerability of the .NET Remoting technology in systems for visualization and analysis of clinical data based on artificial intelligence, IntelliSpace Portal. This allows a intruder to execute arbitrary code.

The vulnerability of the .NET Remoting technology in systems for visualization and analysis of clinical data based on artificial intelligence, IntelliSpace Portal, is related to deficiencies in the deserialization mechanism. Exploiting this vulnerability could allow an attacker to execute arbitra...

The vulnerability of software for optimizing and deploying AI-based applications on AMD Ryzen AI systems, related to integer overflow, allows attackers to compromise the confidentiality, integrity, and accessibility of protected information.

The vulnerability of software for optimizing and deploying AI-based applications on AMD Ryzen AI systems is related to a numerical overflow condition. Exploiting this vulnerability could allow an attacker to compromise the confidentiality, integrity, and accessibility of protected information...

Why AI Systems Need Red Teaming Now More Than Ever

AI systems are becoming a huge part of our lives, but they are not perfect. Red teaming helps…...

Jailbreaking is (mostly) simpler than you think

Content warning: This blog post contains discussions of sensitive topics. These subjects may be distressing or triggering for some readers. Reader discretion is advised. Today, we are sharing insights on a simple, optimization-free jailbreak method called Context Compliance Attack CCA, that has...

CVE-2024-52524 ReDoS in Giskard Scan text perturbation

Giskard is an evaluation and testing framework for AI systems. A Remote Code Execution ReDoS vulnerability was discovered in Giskard component by the GitHub Security Lab team. When processing datasets with specific text patterns with Giskard detectors, this vulnerability could trigger exponential...

Apple Opens PCC Source Code for Researchers to Identify Bugs in Cloud AI Security

Apple has publicly made available its Private Cloud Compute PCC Virtual Research Environment VRE, allowing the research community to inspect and verify the privacy and security guarantees of its offering. PCC, which Apple unveiled earlier this June, has been marketed as the "most advanced securit...

Delivering Proactive Protection Against Critical Threats to NVIDIA-powered AI Systems

On Wednesday, NVIDA released updates to fix a critical vulnerability in its NVIDIA Container Toolkit, which, if exploited, could put a wide range of AI infrastructure and underlying data/secrets at risk...

LLMs Acting Deceptively

New research: "Deception abilities emerged in large language models": Abstract: Large language models LLMs are currently at the forefront of intertwining AI systems with human communication and everyday life. Thus, aligning them with human values is of great importance. However, given the steady...

CISA Joins ACSC-led Guidance on How to Use AI Systems Securely

CISA has collaborated with the Australian Signals Directorate’s Australian Cyber Security Centre ASD’s ACSC on Engaging with Artificial Intelligencelink is external—joint guidance, led by ACSC, on how to use AI systems securely. The following organizations also collaborated with ACSC on the...

A Robot the Size of the World

In 2016, I wrote about an Internet that affected the world in a direct, physical manner. It was connected to your smartphone. It had sensors like cameras and thermostats. It had actuators: Drones, autonomous cars. And it had smarts in the middle, using sensor data to figure out what to do and the...

CISA and UK NCSC Unveil Joint Guidelines for Secure AI System Development

Today, in a landmark collaboration, the U.S. Cybersecurity and Infrastructure Security Agency CISA and the UK National Cyber Security Centre NCSC are proud to announce the release of the Guidelines for Secure AI System Developmentlink is external. Co-sealed by 23 domestic and international...

Microsoft AI Red Team building future of safer AI

An essential part of shipping software securely is red teaming. It broadly refers to the practice of emulating real-world adversaries and their tools, tactics, and procedures to identify risks, uncover blind spots, validate assumptions, and improve the overall security posture of systems. Microso...

Prompt Injection/Extraction Attacks against AI Systems

This is an interesting attack I had not previously considered. The variants are interesting, and I think were just starting to understand their implications...

Best practices for AI security risk management

Today, we are releasing an AI security risk assessment framework as a step to empower organizations to reliably audit, track, and improve the security of the AI systems. In addition, we are providing new updates to Counterfit, our open-source tool to simplify assessing the security posture of AI...