A preprint posted to arXiv on 30 September proposes a method that simultaneously removes unwanted training data from generative models and compresses them into faster one-step generators, according to the paper. arXiv CS.LG1
The approach could reduce the inference cost of diffusion and flow-matching models, requiring only a pretrained teacher and examples from the data to be forgotten, with no access to the original retained training data, the abstract states. arXiv CS.LG1
Automatica reported on 28 September that a separate preprint established generalization bounds for OPTQ quantization. Automatica Press2 The new work extends the efficiency push by jointly tackling model compression (via distillation) and unlearning, according to the preprint. arXiv CS.LG1
The framework, dubbed Inverse Distillation Unlearning (IDU), formulates distillation as a min-max objective over a data distribution and represents that distribution as a mixture of the forget set and the generated data. By comparing this mixture with the teacher's training distribution, the method recovers only the retained data at the optimum, the abstract states. arXiv CS.LG1
Experiments on MNIST and CIFAR-10 under flow-matching and score-based diffusion settings show a substantial reduction in the generation frequency of forgotten classes while maintaining output quality on retained classes, the authors write. They claim IDU is the first unified framework for simultaneous unlearning and distillation in unconditional flow-matching and score-based models. arXiv CS.LG1
The preprint has not been peer-reviewed, and the evaluation is limited to small vision datasets (MNIST and CIFAR-10), as described in the paper. arXiv CS.LG1