Pretraining Data Exposure in Large Language Models: A Survey of Membership Inference, Data Contamination, and Security Implications

Large Language Models LLMs have become the predominant paradigm in NLP, advancing both research and industry. As model sizes and pretraining data grow, concerns about Pretraining Data Exposure PDE increase due to the scale and opacity of training datasets. PDE refers to determining whether specif...

STAMP Your Content: Proving Dataset Membership Via Watermarked Rephrasings

Given how large parts of publicly available text are crawled to pretrain large language models LLMs, data creators increasingly worry about the inclusion of their proprietary data for model training without attribution or licensing. Their concerns are also shared by benchmark curators whose...