Defending against Prompt Injection with a Few DefensiveTokens

When large language model LLM systems interact with external data to perform complex tasks, a new attack, namely prompt injection, becomes a significant threat. By injecting instructions into the data accessed by the system, the attacker is able to override the initial user task with an arbitrary...