FERRET: Private Deep Learning Faster and Better Than DPSGD

We revisit 1-bit gradient compression through the lens of mutual-information differential privacy MI-DP. Building on signSGD, we propose FERRET--Fast and Effective Restricted Release for Ethical Training--which transmits at most one sign bit per parameter group with Bernoulli masking. Theory: We...