Chasing Shadows: Pitfalls in LLM Security Research

Large language models LLMs are increasingly prevalent in security research. Their unique characteristics, however, introduce challenges that undermine established paradigms of reproducibility, rigor, and evaluation. Prior work has identified common pitfalls in traditional machine learning researc...

Vision Transformers: the Threat of Realistic Adversarial Patches

The increasing reliance on machine learning systems has made their security a critical concern. Evasion attacks enable adversaries to manipulate the decision-making processes of AI systems, potentially causing security breaches or misclassification of targets. Vision Transformers ViTs have gained...

IF-GUIDE: Influence Function-Guided Detoxification of LLMs

We study how training data contributes to the emergence of toxic behaviors in large-language models. Most prior work on reducing model toxicity adopts $reactive$ approaches, such as fine-tuning pre-trained and potentially toxic models to align them with human values. In contrast, we propose a...

Training Transformers for Cyber Security Tasks: A Case Study on Malicious URL Prediction

Highlights Perform a case study on using Transformer models to solve cyber security problems Train a Transformer model to detect malicious URLs under multiple training regimes Compare our model against other deep learning methods, and show it performs on-par with other top-scoring models Identify...

Repurposing Neural Networks to Generate Synthetic Media for Information Operations

FireEye’s Data Science and Information Operations Analysis teams released this blog post to coincide with our Black Hat USA 2020 Briefing, which details how open source, pre-trained neural networks can be leveraged to generate synthetic media for malicious purposes. To summarize our presentation,...