LangChain 安全漏洞

LangChain is used to build applications using LLM through composability. A security vulnerability exists in Langchain version 0.0.171, which stems from an arbitrary code execution vulnerability...

CVE-2023-34094 ChuanhuChatGPT vulnerable to unauthorized configuration file access

ChuanhuChatGPT is a graphical user interface for ChatGPT and many large language models. A vulnerability in versions 20230526 and prior allows unauthorized access to the config.json file of the privately deployed ChuanghuChatGPT project, when authentication is not configured. The attacker can...

AI-powered content farms start clogging search results with ad-stuffed spam

A recent study by NewsGuard, trackers of online misinformation, makes some alarming discoveries about the role of artificial intelligence AI in content farm generation. If youve previously held your nose at the content mill grind, its probably going to become a lot more unpleasant. Content farms...

What does ChatGPT know about phishing?

Can ChatGPT detect phishing links? Hearing all the buzz about the amazing applications of ChatGPT and other language models, our team could not help but ask this question. We work on applying machine learning technologies to cybersecurity tasks, specifically models that analyze websites to detect...

The Hacking of ChatGPT Is Just Getting Started

Security researchers are jailbreaking large language models to get around safety rules. Things could get much worse...

LLMs and Phishing

Heres an experiment being run by undergraduate computer science students everywhere: Ask ChatGPT to generate phishing emails, and test whether these are better at persuading victims to respond or click on the link than the usual spam. Its an interesting experiment, and the results are likely to...

haystack 安全漏洞

haystack is an open source NLP framework for interacting with your data using Transformer models and LLMs GPT-4, ChatGPT, etc.. A security vulnerability exists in haystack versions prior to 0.1.30 that stems from the use of hard-coded constants...

ChatGPT happy to write ransomware, just really bad at it

This morning I decided to write some ransomware. I've never done it before, and I can't code in C, the language ransomware is mostly commonly written in, but I have a reasonably good idea of what ransomware does. Previously, this lack of technical skills would have served as something of a barrie...

ChatGPT helps both criminals and law enforcement, says Europol report

In a report, Europol says that ChatGPT and other large language models LLMs can help criminals with little technical knowledge to perpetrate criminal activities, but it can also assist law enforcement with investigating and anticipating criminal activities. The report aims to provide an overview ...

Prompt Injection Attacks on Large Language Models

This is a good survey on prompt injection attacks on large language models like ChatGPT. Abstract: We are currently witnessing dramatic advances in the capabilities of Large Language Models LLMs. They are already being adopted in practice and integrated into many systems, including integrated...

Jailbreaking ChatGPT and other large language models while we can

The introduction of ChatGPT launched an arms race between tech giants. The rush to be the first to incorporate a similar large language model LLM into their own offerings read: search engines may have left a lot of opportunities to bypass the active restrictions such as bias, privacy concerns, an...

New Study Uncovers Text-to-SQL Model Vulnerabilities Allowing Data Theft and DoS Attacks

A group of academics has demonstrated novel attacks that leverage Text-to-SQL models to produce malicious code that could enable adversaries to glean sensitive information and stage denial-of-service DoS attacks. "To better interact with users, a wide range of database applications employ AI...

Extracting Personal Information from Large Language Models Like GPT-2

Researchers have been able to find all sorts of personal information within GPT-2. This information was part of the training data, and can be extracted with the right sorts of queries. Paper: "Extracting Training Data from Large Language Models." Abstract: It has become common to publish large...

Attention is All They Need: Combatting Social Media Information Operations With Neural Language Models

Information operations have flourished on social media in part because they can be conducted cheaply, are relatively low risk, have immediate global reach, and can exploit the type of viral amplification incentivized by platforms. Using networks of coordinated accounts, social media-driven...