Extraction Meets Distillation - Cryptanalytic Techniques to Recover Parameters from a Deep Neural Network

We study fully connected neural-network reconstruction by combining cryptanalytic weight-direction extraction with learning from predicted labels. We adapt a ReLU extraction implementation to Leaky ReLU, compare sign-search methods, and estimate biases and output layers. We prove conditional boun...