Shadow in the Cache: Unveiling and Mitigating Privacy Risks of KV-Cache in LLM Inference

The Key-Value KV cache, which stores intermediate attention computations Key and Value pairs to avoid redundant calculations, is a fundamental mechanism for accelerating Large Language Model LLM inference. However, this efficiency optimization introduces significant yet underexplored privacy risk...

An Attack to Break Permutation-Based Private Third-Party Inference Schemes for LLMs

Recent advances in Large Language Models LLMs have led to the widespread adoption of third-party inference services, raising critical privacy concerns. Existing methods of performing private third-party inference, such as Secure Multiparty Computation SMPC, often rely on cryptographic methods...