A preprint posted to arXiv on 30 September proposes a method that simultaneously removes unwanted training data from generative models and compresses them into faster one-step generators, according to the paper. arXiv CS.LG
The approach could reduce the inference cost of diffusion and flow-matching models, requiring only a pretrained teacher and examples from the data to be forgotten, with no access to the original retained training data, the abstract states. arXiv CS.LG
Automatica reported on 28 September that a separate preprint established generalization bounds for OPTQ quantization. Automatica Press The new work extends the efficiency push by jointly tackling model compression (via distillation) and unlearning, according to the preprint. arXiv CS.LG
The framework, dubbed Inverse Distillation Unlearning (IDU), formulates distillation as a min-max objective over a data distribution and represents that distribution as a mixture of the forget set and the generated data. By comparing this mixture with the teacher's training distribution, the method recovers only the retained data at the optimum, the abstract states. arXiv CS.LG
Experiments on MNIST and CIFAR-10 under flow-matching and score-based diffusion settings show a substantial reduction in the generation frequency of forgotten classes while maintaining output quality on retained classes, the authors write. They claim IDU is the first unified framework for simultaneous unlearning and distillation in unconditional flow-matching and score-based models. arXiv CS.LG
The preprint has not been peer-reviewed, and the evaluation is limited to small vision datasets (MNIST and CIFAR-10), as described in the paper. arXiv CS.LG