SigCo: Eliminate the inter-class competition via sigmoid for learning with noisy labels

Abstract

Accurate predictions from deep neural networks are crucial for distinguishing clean data and correcting noisy labels in current label noise learning methods. However, the conventional Softmax classifier used in most relevant works is highly sensitive to label noise due to its inherent competition-prompting mechanism, i.e., similar categories are encouraged to compete for limited confidence scores during class activation, especially between the noisy classes and the ground-truth, which can inevitably lead to suboptimal predictions and eventually hamper model performance. To address this inter-class competition problem, we propose a novel Sigmoid-based Sample Selection and Correction method named SigCo for learning with noisy labels. Different from previous works, we develop a Sigmoid-based network in which each Sigmoid classifier independently predicts its respective class, improving the reliability of the selection and correction process through more accurate predictions. Besides, in order to mitigate the negative impact of noisy labels, we design a noise-adaptive learning strategy by imposing stringent class masking constraints on clean samples to enhance the learning of discriminative features, while adopting a loose masking strategy for noisy data to improve the robustness to label noise. Additionally, we introduce a co-training strategy between our Sigmoid-based network and the conventional Softmax-based network to implicitly boost the generalization capability of the model. Extensive experiments on synthetic and real-world benchmarks show that SigCo consistently outperforms state-of-the-art methods. Especially on CIFAR-100N with 80% and 90% symmetric noise ratios, it improves test accuracy by 5.10% and 18.44%, respectively.