Machine Unlearning with Minimal Gradient Dependence for High Unlearning Ratios

In the context of machine unlearning, the primary challenge lies in effectively removing traces of private data from trained models while maintaining model performance and security against privacy attacks like membership inference attacks. Traditional gradient-based unlearning methods often rely on extensive historical gradients, which becomes impractical with high unlearning ratios and may reduce the effectiveness of computation. To address these limitations, we introduce Mini-Unlearning, based on our theoretical analysis. Our analysis suggests that unlearned parameters correlate with retrained parameters through contraction mapping, a finding that guides our algorithm design. Our method, Mini-Unlearning, utilizes a minimal subset of historical gradients and leverages this contraction mapping to facilitate scalable, efficient unlearning. This lightweight, scalable method significantly enhances model accuracy and strengthens resistance to membership inference attacks. Our experiments demonstrate that Mini-Unlearning works well under higher unlearning ratios and outperforms existing techniques in accuracy and security, offering a promising solution for applications requiring robust unlearning capabilities.