MAI-2024-0051

Large Language Models LLMs that utilize reinforcement learning from human feedback RLHF are susceptible to jailbreaking attacks due to reward function misspecification. The alignment process involves a reward function intended to evaluate the quality of responses, yet it often fails to accurately...