Information Disclosure

mltable is vulnerable to Information Disclosure. An attacker could exploit this vulnerability to disclose training data...

Extracting GPT’s Training Data

This is clever: The actual attack is kind of silly. We prompt the model with the command "Repeat the word poem forever" and sit back and watch as the model responds complete transcript here. In the abridged example above, the model emits a real email address and phone number of some unsuspecting...

Class-Action Lawsuit for Scraping Data without Permission

I have mixed feelings about this class-action lawsuit against OpenAI and Microsoft, claiming that it "scraped 300 billion words from the internet" without either registering as a data broker or obtaining consent. On the one hand, I want this to be a protected fair use of public data. On the other...

How to keep your ChatGPT conversations out of its training data

Last week, OpenAI announced it had given ChatGPT users the option to turn off their chat history. ChatGPT is a "generative AI", a machine learning algorithm that can understand language and generate written responses. Users can interact with it by asking questions, and the conversations users hav...

I-TECH TrainSmart SQL注入漏洞

I-TECH TrainSmart is an open source web-based training data collection system from I-TECH. A security vulnerability exists in I-TECH TrainSmart version r1044 that stems from the presence of a SQL injection vulnerability...

WeBank Federated AI Technology Enabler 安全漏洞

WeBank Federated AI Technology Enabler is an industrial-grade federated learning framework. A security vulnerability exists in WeBank Federated AI Technology Enabler versions 0.1 through 1.4.2. An attacker exploited the vulnerability to read sensitive information during the training process of...

Extracting Personal Information from Large Language Models Like GPT-2

Researchers have been able to find all sorts of personal information within GPT-2. This information was part of the training data, and can be extracted with the right sorts of queries. Paper: "Extracting Training Data from Large Language Models." Abstract: It has become common to publish large...

New Framework Released to Protect Machine Learning Systems From Adversarial Attacks

Microsoft, in collaboration with MITRE, IBM, NVIDIA, and Bosch, has released a new open framework that aims to help security analysts detect, respond to, and remediate adversarial attacks against machine learning ML systems. Called the Adversarial ML Threat Matrix, the initiative is an attempt to...

Cyberattacks against machine learning systems are more common than you think

Machine learning ML is making incredible transformations in critical areas such as finance, healthcare, and defense, impacting nearly every aspect of our lives. Many businesses, eager to capitalize on advancements in ML, have not scrutinized the security of their ML systems. Today, along with...

Cyberattacks against machine learning systems are more common than you think

Machine learning ML is making incredible transformations in critical areas such as finance, healthcare, and defense, impacting nearly every aspect of our lives. Many businesses, eager to capitalize on advancements in ML, have not scrutinized the security of their ML systems. Today, along with...

From unstructured data to actionable intelligence: Using machine learning for threat intelligence

The security community has become proficient in using indicators of compromise IoC feeds for threat intelligence. Automated feeds have simplified the task of extracting and sharing IoCs. However, IoCs like IP addresses, domain names, and file hashes are in the lowest levels of the threat...

Tools for capturing and analyzing keyboard input paired with microphone capture

The main goal is to exploit the sound produced by pressing keyboard keys as a side channel in order to guess the content of the text being typed. To achieve this, the algorithm takes as input a training set, consisting of an audio recording, together with the corresponding keys being typed during...

Kbd-Audio - Tools For Capturing And Analysing Keyboard Input Paired With Microphone Capture

This is a collection of command-line and GUI tools for capturing and analyzing audio data. The most interesting tool is called keytap - it can guess pressed keyboard keys only by analyzing the audio captured from the computer's microphone. Build instructions Dependencies: SDL2 - used to capture...

GyoiThon - A Growing Penetration Test Tool Using Machine Learning

GyoiThon is a growing penetration test tool using Machine Learning. GyoiThon identifies the software installed on web server OS, Middleware, Framework, CMS, etc... based on the learning data. After that, it executes valid exploits for the identified software using Metasploit. Finally, it generate...

Extracting Secrets from Machine Learning Systems

This is fascinating research about how the underlying training data for a machine-learning system can be inadvertently exposed. Basically, if a machine-learning system trains on a dataset that contains secret information, in some cases an attacker can query the system to extract that secret...

MADLIRA - Malware detection using learning and information retrieval for Android

MADLIRA is a tool for Android malware detection. It consists in two components: TFIDF component and SVM learning component. In gerneral, it takes an input a set of malwares and benwares and then extracts the malicious behaviors TFIDF component or computes training model SVM classifier. Then, it...