47 matches found
Exploring Jailbreak Attacks on LLMs through Intent Concealment and Diversion
Although large language models LLMs have achieved remarkable advancements, their security remains a pressing concern. One major threat is jailbreak attacks, where adversarial prompts bypass model safeguards to generate harmful or objectionable content. Researchers study jailbreak attacks to...
Adversarial Attack on Large Language Models Using Exponentiated Gradient Descent
As Large Language Models LLMs are widely used, understanding them systematically is key to improving their safety and realizing their full potential. Although many models are aligned using techniques such as reinforcement learning from human feedback RLHF, they are still vulnerable to jailbreakin...
Exploring PLeak: An Algorithmic Method for System Prompt Leakage
What is PLeak, and what are the risks associated with it? We explored this algorithmic technique and how it can be used to jailbreak LLMs, which could be leveraged by threat actors to manipulate systems and steal sensitive data...
PYSEC-2025-63
vLLM is a high-throughput and memory-efficient inference and serving engine for LLMs. When vLLM is configured to use Mooncake, unsafe deserialization exposed directly over ZMQ/TCP on all network interfaces will allow attackers to execute remote code on distributed hosts. This is a remote code...
CVE-2025-29770
CVE-2025-29770 affects vLLM’s guided_decoding outlines backend. The vulnerability arises because outlines_logits_processors.py unconditionally uses outlines’ on-disk grammar cache, enabling a malicious user to send many short decoding requests with unique schemas and exhaust the filesystem, causi...
On Generative AI Security
Microsoft's AI Red Team just published "Lessons from Red Teaming 100 Generative AI Products." Their blog post lists "three takeaways," but the eight lessons in the report itself are more useful: 1. Understand what the system can do and where it is applied. 2. You don't have to compute gradients t...
Exploring Artificial Intelligence: Is AI Overhyped?
Dive into AI technologies like inference, deep learning, and generative models to learn how LLMs and AI are transforming cybersecurity and tech industries...
From Naptime to Big Sleep: Using Large Language Models To Catch Vulnerabilities In Real-World Code
Posted by the Big Sleep team Introduction In our previous post, Project Naptime: Evaluating Offensive Security Capabilities of Large Language Models, we introduced our framework for large-language-model-assisted vulnerability research and demonstrated its potential by improving the state-of-the-a...
How LLMs could help defenders write better and faster detection
Most users will associate large language models LLMs like ChatGPT with answering basic questions or helping to write basics lines of text. But could these tools actually help defenders in the cybersecurity industry write more effective detection content? Several security researchers from across...
GenAI risks to be aware of — and prepare for — according to Gartner®
The deployment of GenAI, LLMs, and chat interfaces expands potential attack surfaces and poses increased security threats...
Project Naptime: Evaluating Offensive Security Capabilities of Large Language Models
Posted by Sergei Glazunov and Mark Brand, Google Project Zero Introduction At Project Zero, we constantly seek to expand the scope and effectiveness of our vulnerability research. Though much of our work still relies on traditional methods like manual source code audits and reverse engineering,...
LLMs Acting Deceptively
New research: "Deception abilities emerged in large language models": Abstract: Large language models LLMs are currently at the forefront of intertwining AI systems with human communication and everyday life. Thus, aligning them with human values is of great importance. However, given the steady...
SASE Threat Report: 8 Key Findings for Enterprise Security
Threat actors are evolving, yet Cyber Threat Intelligence CTI remains confined to each isolated point solution. Organizations require a holistic analysis across external data, inbound and outbound threats and network activity. This will enable evaluating the true state of cybersecurity in the...
CVE-2024-4181
The CVE-2024-4181 issue affects llama_index (RunGptLLM class) version 0.9.47 used by RunGpt framework to connect to LLMs. Root cause: unsafe use of eval enabling a malicious or compromised LLM hosting provider to run arbitrary commands on a client machine. Impact statements in sources indicate th...
Spring AI - Multimodality - Orbis Sensualium Pictus
Humans process knowledge, simultaneously across multiple modes of data inputs. The way we learn, our experiences are all multimodal. We don't have just vision, just audio and just text. These foundational principles of learning were articulated by the father of modern education John Amos Comenius...
CVE-2024-30256
Open WebUI is a user-friendly WebUI for LLMs. Open-webui is vulnerable to authenticated blind server-side request forgery. This vulnerability is fixed in 0.1.117...
CVE-2024-30256
CVE-2024-30256 affects Open WebUI prior to version 0.1.117. The vulnerability is an authenticated blind server-side request forgery (SSRF) in the backend, specifically in the function download_file_stream() inside Open WebUI’s backend/apps/web/routers/utils.py, exploitable via the url parameter. ...
Generative AI Security - Secure Your Business in a World Powered by LLMs
Did you know that 79% of organizations are already leveraging Generative AI technologies? Much like the internet defined the 90s and the cloud revolutionized the 2010s, we are now in the era of Large Language Models LLMs and Generative AI. The potential of Generative AI is immense, yet it brings...