When Intelligence Fails: An Empirical Study on Why LLMs Struggle with Password Cracking

The remarkable capabilities of Large Language Models LLMs in natural language understanding and generation have sparked interest in their potential for cybersecurity applications, including password guessing. In this study, we conduct an empirical investigation into the efficacy of pre-trained LL...

The Landscape of Memorization in LLMs: Mechanisms, Measurement, and Mitigation

Large Language Models LLMs have demonstrated remarkable capabilities across a wide range of tasks, yet they also exhibit memorization of their training data. This phenomenon raises critical questions about model behavior, privacy risks, and the boundary between learning and memorization. Addressi...

Counterfactual Influence As a Distributional Quantity

Machine learning models are known to memorize samples from their training data, raising concerns around privacy and generalization. Counterfactual self-influence is a popular metric to study memorization, quantifying how the model's prediction for a sample changes depending on the sample's...

Leaner Training, Lower Leakage: Revisiting Memorization in LLM Fine-Tuning with LoRA

Memorization in large language models LLMs makes them vulnerable to data extraction attacks. While pre-training memorization has been extensively studied, fewer works have explored its impact in fine-tuning, particularly for LoRA fine-tuning, a widely adopted parameter-efficient method. In this...

SoK: Data Reconstruction Attacks against Machine Learning Models: Definition, Metrics, and Benchmark

Data reconstruction attacks, which aim to recover the training dataset of a target model with limited access, have gained increasing attention in recent years. However, there is currently no consensus on a formal definition of data reconstruction attacks or appropriate evaluation metrics for...

Private Memorization Editing: Turning Memorization into a Defense to Strengthen Data Privacy in Large Language Models

Large Language Models LLMs memorize, and thus, among huge amounts of uncontrolled data, may memorize Personally Identifiable Information PII, which should not be stored and, consequently, not leaked. In this paper, we introduce Private Memorization Editing PME, an approach for preventing private...

Upgraded Q -> 2 from #410 [1684435015507]

Judge has assessed an item in Issue 410 as 2 risk. The relevant finding follows: QA-2 Publicly Callable memorializePositions Function Allows Unauthorized memorization of User Positions memorializePositions function in positionManager.sol allows any caller to modify position information of any use...