DoomArena: a Framework for Testing AI Agents against Evolving Security Threats

We present DoomArena, a security evaluation framework for AI agents. DoomArena is designed on three principles: 1 It is a plug-in framework and integrates easily into realistic agentic frameworks like BrowserGym for web agents and $τ$-bench for tool calling agents; 2 It is configurable and allows...

DualBreach: Efficient Dual-Jailbreaking Via Target-Driven Initialization and Multi-Target Optimization

Recent research has focused on exploring the vulnerabilities of Large Language Models LLMs, aiming to elicit harmful and/or sensitive content from LLMs. However, due to the insufficient research on dual-jailbreaking -- attacks targeting both LLMs and Guardrails, the effectiveness of existing...

Do You Really Need Public Data? Surrogate Public Data for Differential Privacy on Tabular Data

Differentially private DP machine learning often relies on the availability of public data for tasks like privacy-utility trade-off estimation, hyperparameter tuning, and pretraining. While public data assumptions may be reasonable in text and image domains, they are less likely to hold for tabul...

A Data-Centric Approach for Safe and Secure Large Language Models against Threatening and Toxic Content

Large Language Models LLM have made remarkable progress, but concerns about potential biases and harmful content persist. To address these apprehensions, we introduce a practical solution for ensuring LLM's safe and ethical use. Our novel approach focuses on a post-generation correction mechanism...

Multi-Stage Retrieval for Operational Technology Cybersecurity Compliance Using Large Language Models: a Railway Casestudy

Operational Technology Cybersecurity OTCS continues to be a dominant challenge for critical infrastructure such as railways. As these systems become increasingly vulnerable to malicious attacks due to digitalization, effective documentation and compliance processes are essential to protect these...

Everything You Wanted to Know about LLM-Based Vulnerability Detection but Were Afraid to Ask

Large Language Models are a promising tool for automated vulnerability detection, thanks to their success in code generation and repair. However, despite widespread adoption, a critical question remains: Are LLMs truly effective at detecting real-world vulnerabilities? Current evaluations, which...

Artificial Intelligence – What's all the fuss?

Talking about AI: Definitions Artificial Intelligence AI — AI refers to the simulation of human intelligence in machines, enabling them to perform tasks that typically require human intelligence, such as decision-making and problem-solving. AI is the broadest concept in this field, encompassing...

Provable Secure Steganography Based on Adaptive Dynamic Sampling

The security of private communication is increasingly at risk due to widespread surveillance. Steganography, a technique for embedding secret messages within innocuous carriers, enables covert communication over monitored channels. Provably Secure Steganography PSS is state of the art for making...

Investigating Cybersecurity Incidents Using Large Language Models in Latest-Generation Wireless Networks

The purpose of research: Detection of cybersecurity incidents and analysis of decision support and assessment of the effectiveness of measures to counter information security threats based on modern generative models. The methods of research: Emulation of signal propagation data in MIMO systems,...

Lunary 安全漏洞

lunary is lunary open source a production toolkit for LLM . An access control error vulnerability exists in lunary that stems from improper access control on the /prompts/promptid endpoint, and no detailed vulnerability details are provided at this time...

LlamaIndex 安全漏洞

LlamaIndex is a data framework for LLM applications from the LlamaIndex open source. A security vulnerability exists in LlamaIndex version v0.12.5 that stems from an unhandled thread exception and could lead to a denial of service attack...

12,000+ API Keys and Passwords Found in Public Datasets Used for LLM Training

A dataset used to train large language models LLMs has been found to contain nearly 12,000 live secrets, which allow for successful authentication. The findings once again highlight how hard-coded credentials pose a severe security risk to users and organizations alike, not to mention compounding...

Building Effective Agents with Spring AI (Part 1)

In a recent research publication: Building effective agents, Anthropic shared valuable insights about building effective Large Language Model LLM agents. What makes this research particularly interesting is its emphasis on simplicity and composability over complex frameworks. Let's explore how...

New Research: Enhancing Botnet Detection with AI using LLMs and Similarity Search

As botnets continue to evolve, so do the techniques required to detect them. While Transport Layer Security TLS encryption is widely adopted for secure communications, botnets leverage TLS to obscure command-and-control C2 traffic. These malicious actors often have identifiable characteristics...

AI-supported spear phishing fools more than 50% of targets

One of the first things everyone predicted when artificial intelligence AI became more commonplace was that it would assist cybercriminals in making their phishing campaigns more effective. Now, researchers have conducted a scientific study into the effectiveness of AI supported spear phishing, a...

The vulnerability of Ollama’s system for running and managing large language models lies in the improper restriction on the path name to the restricted-access catalog, which allows a violator to trigger a service failure.

The vulnerability of the Ollama system for running and managing large language models is related to an incorrect restriction on the path name to the restricted-access catalog. Exploiting this vulnerability could allow a malicious actor to trigger a service failure...

The vulnerability of Ollama’s system for launching and managing large language models, related to the exposure of system data to unauthorized individuals, allows a violator to trigger a service failure.

The vulnerability of Ollama’s system for running and managing large language models is related to the exposure of system data to unauthorized individuals. Exploiting this vulnerability could allow a malicious actor to cause service failures...

The vulnerability of Ollama’s system for running and managing large language models, related to uncontrolled resource consumption, allows a hacker to trigger a service failure.

The vulnerability of Ollama’s system for running and managing large language models is related to an uncontrolled consumption of resources. Exploiting this vulnerability could allow a malicious actor to cause service failures...

Secure Your Generative Investments: Qualys Advances Enterprise TruRisk Platform with Qualys TotalAI to Protect Your LLM Investments

Artificial intelligence AI and large language models LLMs are reshaping industries, streamlining enterprise operations, and fueling unprecedented innovation. However, as adoption accelerates, so do the associated risks. While 70% of enterprises plan to deploy LLMs in production within the next 12...

The vulnerability of the software’s user data loading function for running large language models (LLMs) like ChuanhuChatGPT allows a perpetrator to execute arbitrary code.

The vulnerability of the software’s user data loading function for running large language models like ChuanhuChatGPT is related to an incorrect restriction on the path name to the restricted-access directory. Exploiting this vulnerability could allow a malicious actor to execute arbitrary code...