Comet: Accelerating Private Inference for Large Language Model by Predicting Activation Sparsity

With the growing use of large language models LLMs hosted on cloud platforms to offer inference services, privacy concerns about the potential leakage of sensitive information are escalating. Secure multi-party computation MPC is a promising solution to protect the privacy in LLM inference...