ECLB: Efficient contrastive learning on bi-level for noisy labels

Abstract

For contrastive learning has achieved remarkable success in self-supervised and supervised informative representation, learning with noisy labels based on contrastive learning is becoming the research consensus. However, under noisy labels, how to efficiently leverage informative representation of various levels and how to effectively screen reliable positive pairs for the optimization of the contrastive learning model are still challenges. To address these issues, we innovatively propose a method named efficient contrastive learning on bi-level for noisy labels (ECLB), which is jointly implemented by both self-supervised and supervised contrastive learning. For the accessible informative representation, we propose to perform contrastive learning at two different levels: (1) feature level, where feature representation is jointly optimized by supervised and self-supervised feature contrastive loss; (2) label level, where feature and label representation are optimized by label distribution supervised contrastive loss. Furthermore, to alleviate the impact of noisy labels on the selection of reliable positive pairs in supervised contrastive learning and to reduce labor cost and computational complexity, we propose an efficient adaptive mask, which is dynamically generated by label self-equality mask, prediction self-equality mask, label-prediction equality mask, and feature similarity mask. Extensive experiments show that our proposed method outperforms other state-of-the-art methods in terms of robustness and generalization. Our code is publicly available at: https://github.com/whyandbecause/ECLB