Making Deep Neural Networks Robust to Label Noise: A Reweighting Loss and Data Filtration

Abstract

Deep neural networks (DNNs) have achieved astonishing results on a variety of supervised learning tasks owing to a large scale of expert-labeled datasets. However, as recent researches pointed out, the generalization performance of DNNs deteriorates rapidly when training data contains label noise. To alleviate the problem of easily overfitting to label-corrupted data when training DNNs, this paper proposes a robust loss function by reweighting the standard Cross-Entropy loss. For obtaining more robust DNNs under label noise, we further design a framework to jointly optimize model parameters and filtering noisy data during training. Our methods can be thus viewed as online curriculum learning based on both loss function and training datasets. A great deal of experiments have conducted on CIFAR-10, CIFAR-100 and ImageNet under two types of label noise. The results of our proposed method outperforms other state-of-the-art methods.