How Private Is Your Attention? Bridging Privacy with In-Context Learning

In-context learning ICL-the ability of transformer-based models to perform new tasks from examples provided at inference time-has emerged as a hallmark of modern language models. While recent works have investigated the mechanisms underlying ICL, its feasibility under formal privacy constraints...