SpliTEE: Improving LLM Inference on Trusted Hardware with Differentially Private GPU Outsourcing

User prompts provided to large language models LLMs may contain sensitive or private information that can be misused by remotely deployed models, such as through inadvertent memorization during retraining. One way to protect user prompts is to execute the LLM inside a trusted execution environmen...