TuneShield: Mitigating Toxicity in Conversational AI While Fine-Tuning on Untrusted Data

Recent advances in foundation models, such as LLMs, have revolutionized conversational AI. Chatbots are increasingly being developed by customizing LLMs on specific conversational datasets. However, mitigating toxicity during this customization, especially when dealing with untrusted training dat...