OpenAI Pauses Frontier RL Training as It Tightens Defenses Against Unsafe AI Behavior

OpenAI on Tuesday revealed that it paused reinforcement learning RL training for its latest artificial intelligence AI models for two weeks while it shored up additional defenses and increased the scope of its monitoring to avert another Hugging Face-like incident. "As models become more capable,...

If the Markets Reject OpenAI and Anthropic, the US Should Nationalize Them

This essay was written with Nathan E. Sanders, and originally appeared inThe Guardian. OpenAI, and then Anthropic, were each formed by AI developers who feared unrestrained corporate AI development--specifically, that companies like Google and Meta would steer the technology towards deleterious,...

Malicious code in intel-ai-safety-explainer (PyPI)

--- -= Per source details. Do not edit below this line.=- Source: kam193 7561bb0b816a4521b6de43bce01afa55516a7201b6daa7696de4924623557f90 Installing the package or importing the module exfiltrates basic information about the host, and the package has no other purpose. --- Category: PROBABLYPENTES...

Backchaining Loss of Control Mitigations from Mission-Specific Benchmarks in National Security

Affordances and permissions are promising and timely safety levers for mitigating Loss of Control LoC threats in high-stakes deployment contexts, such as national security. Deployers in defense and intelligence could rely on several approaches to identify which affordances and permissions should ...

SecureCAI: Injection-Resilient LLM Assistants for Cybersecurity Operations

Large Language Models have emerged as transformative tools for Security Operations Centers, enabling automated log analysis, phishing triage, and malware explanation; however, deployment in adversarial cybersecurity environments exposes critical vulnerabilities to prompt injection attacks where...

Whispering poetry at AI can make it break its own rules

Most of the big AI makers don't like people using their models for unsavory activity. Ask one of the mainstream AI models how to make a bomb or create nerve gas and you'll get the standard "I don't help people do harmful things" response. That has spawned a cat-and-mouse game of people who try to...

Hammering the Diagnosis: Rowhammer-Induced Stealthy Trojan Attacks on ViT-Based Medical Imaging

Vision Transformers ViTs have emerged as powerful architectures in medical image analysis, excelling in tasks such as disease detection, segmentation, and classification. However, their reliance on large, attention-driven models makes them vulnerable to hardware-level attacks. In this paper, we...

A week in security (October 20 – October 26)

Last week on Malwarebytes Labs: Is AI moving faster than its safety net? Thousands of online stores at risk as SessionReaper attacks spread Apple may have to open its walled garden to outside app stores Meta boosts scam protection on WhatsApp and Messenger Home Depot Halloween phish gives users a...

Replit AI Agent Deletes Sensitive Data Despite Explicit Instructions

Replit AI agent deleted data from 1,200+ executives and companies without permission, raising concerns about AI safety and control in live environments...

Meta AI chatbot bug could have allowed anyone to see private conversations

A researcher has disclosed to TechCrunch that he received a $10,000 bounty for reporting a bug that let anyone access private prompts and responses with the Meta AI chatbot. On June 13, we reported that the Meta AI app publicly exposes user conversations, often without users realizing it. In thes...

Risks & Benefits of LLMs & GenAI for Platform Integrity, Healthcare Diagnostics, Cybersecurity, Privacy & AI Safety: a Comprehensive Survey, Roadmap & Implementation Blueprint

Large Language Models LLMs and generative AI GenAI systems such as ChatGPT, Claude, Gemini, LLaMA, and Copilot, developed by OpenAI, Anthropic, Google, Meta, and Microsoft are reshaping digital platforms and app ecosystems while introducing key challenges in cybersecurity, privacy, and platform...

Risks and Benefits of LLMs and GenAI for Platform Integrity, Healthcare Diagnostics, Cybersecurity, Privacy and AI Safety: a Comprehensive Survey, Roadmap and Implementation Blueprint

Large Language Models LLMs and generative AI GenAI systems such as ChatGPT, Claude, Gemini, LLaMA, and Copilot, developed by OpenAI, Anthropic, Google, Meta, and Microsoft are reshaping digital platforms and app ecosystems while introducing key challenges in cybersecurity, privacy, and platform...

ChatGPT o3 Resists Shutdown Despite Instructions, Study Claims

ChatGPT o3 resists shutdown despite explicit instructions, raising fresh concerns over AI safety, alignment, and reinforcement learning behaviors...

Security Practices in AI Development

What makes safety claims about general purpose AI systems such as large language models trustworthy? We show that rather than the capabilities of security tools such as alignment and red teaming procedures, it is security practices based on these tools that contributed to reconfiguring the image ...

Regulating AI Behavior with a Hypervisor

Interesting research: "Guillotine: Hypervisors for Isolating Malicious AIs." Abstract :As AI models become more embedded in critical sectors like finance, healthcare, and the military, their inscrutable behavior poses ever-greater risks to society. To mitigate this risk, we propose Guillotine, a...

Security-First AI: Foundations for Robust and Trustworthy Systems

The conversation around artificial intelligence AI often focuses on safety, transparency, accountability, alignment, and responsibility. However, AI security i.e., the safeguarding of data, models, and pipelines from adversarial manipulation underpins all of these efforts. This manuscript posits...

On Generative AI Security

Microsoft's AI Red Team just published "Lessons from Red Teaming 100 Generative AI Products." Their blog post lists "three takeaways," but the eight lessons in the report itself are more useful: 1. Understand what the system can do and where it is applied. 2. You don't have to compute gradients t...

California AI Safety Bill Vetoed

Governor Newsom has vetoed the state's AI safety bill. I have mixed feelings about the bill. There's a lot to like about it, and I want governments to regulate in this space. But, for now, it's all EU. Related, the Council of Europe treaty on AI is ready for signature. It'll be legally binding wh...

Understanding HyperCycle’s HyperShare Smart Contract Feature

HyperCycle enhances AI safety and efficiency with cryptographic proofs and peer-to-peer nodes. HyperShare supports decentralized governance and income…...

AI jailbreaks: What they are and how they can be mitigated

Generative AI systems are made up of multiple components that interact to provide a rich user experience between the human and the AI models. As part of a responsible AI approach, AI models are protected by layers of defense mechanisms to prevent the production of harmful content or being used to...