When Data-Free Knowledge Distillation Meets Non-Transferable Teacher: Escaping Out-Of-Distribution Trap Is All You Need

Data-free knowledge distillation DFKD transfers knowledge from a teacher to a student without access the real in-distribution ID data. Its common solution is to use a generator to synthesize fake data and use them as a substitute for real ID data. However, existing works typically assume teachers...