System Prompt Poisoning: Persistent Attacks on Large Language Models beyond User Injection

Large language models LLMs have gained widespread adoption across diverse applications due to their impressive generative capabilities. Their plug-and-play nature enables both developers and end users to interact with these models through simple prompts. However, as LLMs become more integrated in...

Defending against Indirect Prompt Injection by Instruction Detection

The integration of Large Language Models LLMs with external sources is becoming increasingly common, with Retrieval-Augmented Generation RAG being a prominent example. However, this integration introduces vulnerabilities of Indirect Prompt Injection IPI attacks, where hidden instructions embedded...

A Proposal for Evaluating the Operational Risk for ChatBots Based on Large Language Models

The emergence of Generative AI Gen AI and Large Language Models LLMs has enabled more advanced chatbots capable of human-like interactions. However, these conversational agents introduce a broader set of operational risks that extend beyond traditional cybersecurity considerations. In this work, ...

Hoist with His Own Petard: Inducing Guardrails to Facilitate Denial-Of-Service Attacks on Retrieval-Augmented Generation of LLMs

Whitepaper called Hoist With His Own Petard: Inducing Guardrails To Facilitate Denial-Of-Service Attacks On Retrieval-Augmented Generation Of LLMs...

Traceback of Poisoning Attacks to Retrieval-Augmented Generation

Large language models LLMs integrated with retrieval-augmented generation RAG systems improve accuracy by leveraging external knowledge sources. However, recent research has revealed RAG's susceptibility to poisoning attacks, where the attacker injects poisoned texts into the knowledge database,...

LLMpatronous: Harnessing the Power of LLMs for Vulnerability Detection

Despite the transformative impact of Artificial Intelligence AI across various sectors, cyber security continues to rely on traditional static and dynamic analysis tools, hampered by high false positive rates and superficial code comprehension. While generative AI offers promising automation...

On the Feasibility of Using MultiModal LLMs to Execute AR Social Engineering Attacks

Augmented Reality AR and Multimodal Large Language Models LLMs are rapidly evolving, providing unprecedented capabilities for human-computer interaction. However, their integration introduces a new attack surface for social engineering. In this paper, we systematically investigate the feasibility...

DB-GPT 安全漏洞

DB-GPT is an AWEL and agent-based AI native data application development framework open-sourced by eosphoros. A security vulnerability exists in DB-GPT version 0.6.0, which stems from an arbitrary file write vulnerability in the RAG-knowledge endpoint, which allows an attacker to write a file to ...

AI Meets Spring Petclinic: Implementing an AI Assistant with Spring AI (Part II)

Recap of Part I In the first part of this blog series, we explored the basics of integrating Spring AI with large language models. We walked through building a custom ChatClient, leveraging Function Calling for dynamic interactions, and refining our prompts to suit the Spring Petclinic use case. ...

Spring Tips: Spring AI

Hi, Spring fans! In this installment we'll look at the new Spring AI project, which provides convenient integrations with LLMs like the one behind ChatGPT and tools to support the RAG retrieval augmented generation pipeline. ai springboot artificialintelligence java graalvm cloud java21 postgresq...