Retaining Diverse Information in Contrastive Learning Through Multiple Projectors

Abstract

<i>Contrastive Learning (CL)</i> achieves great success in learning visual representations by comparing two augmented views of the same images. However, this very design removes transformation-dependent visual information from the pre-training, which leads to incomplete representations and is harmful for downstream tasks. It&#x0027;s still an open question to retain such information in the CL pre-training process. In this paper, we propose a <i>Multi-Projector Contrastive Learning (MPCL)</i> to address this issue, which produces multi-view contrastive candidates to retain more comprehensive visual characteristics. In addition, we introduce a contrast regularization to construct multiple projectors as different as possible, thereby facilitating the diversity of preserved information. Finally, to promote a consistent learning process for multi-projector, we design a projector training balance strategy to adjust the learning preference of different projectors. MPCL can be applied to various CL frameworks to effectively protect visual characteristics. Experimental results show that the method performs well on subsequent tasks such as linear and semi-supervised image classification, object detection, and semantic segmentation. Importantly, the visual transformer trained by MPCL improves 2&#x0025; absolute points of linear evaluation beyond the MoCo-v3 on the ImageNet-100 dataset.