PIG: Privacy Jailbreak Attack on LLMs Via Gradient-Based Iterative In-Context Optimization

Large Language Models LLMs excel in various domains but pose inherent privacy risks. Existing methods to evaluate privacy leakage in LLMs often use memorized prefixes or simple instructions to extract data, both of which well-alignment models can easily block. Meanwhile, Jailbreak attacks bypass...