Leaky Students: Membership Inference against On-Policy Distillation

On-policy distillation OPD trains a student to match a teacher's next-token distributions on student-generated trajectories. However, privileged information supplied to the teacher for OPD training may contain sensitive data. Whether the student leaks private information about the records supplie...