DREAM: Scalable Red Teaming for Text-To-Image Generative Systems Via Distribution Modeling

Despite the integration of safety alignment and external filters, text-to-image T2I generative models are still susceptible to producing harmful content, such as sexual or violent imagery. This raises serious concerns about unintended exposure and potential misuse. Red teaming, which aims to...