Unsupervised self-training correction learning for 2D image-based 3D model retrieval

Abstract

Existing 2D image-based 3D model retrieval (IBMR) methods usually use the pseudo labels as semantic guidance to reduce the domain-wise and class-wise feature distribution difference across annotated 2D images and unlabeled 3D models. However, they cannot entirely guarantee the quality of pseudo labels, which will decrease prediction discriminability and diversity to affect feature distribution alignment. Therefore, we propose a novel unsupervised self-training correction learning (USTCL) network for the IBMR task. Specifically, we first utilize a CNN to encode 2D images and 3D models (described as multi-view images). Then, we design a noise-corrected self-training learning module (NCST) to denoise pseudo labels in an adversarial manner to make the predicted categories more easily discriminated to improve prediction discriminability. Besides, we employ a target-guided pseudo label refining strategy (TPLR) to progressively refine generated pseudo labels to prevent minority categories from being pushed into majority categories, thereby enhancing prediction diversity. Comprehensive experiments on popular IBMR benchmarks validate the effectiveness and robustness of USTCL, e.g., it can achieve the average gains of 54.20%/22.30%, 62.17%/43.21%, 54.21%/31.18%, 63.48%/43.28%, 59.87%/44.49% in terms of NN, FT, ST, F-measure, DCG and the decrease of 29.50%/35.15% corresponding to ANMRR on MI3DOR (21,000 2D images and 7,690 3D models) and MI3DOR-2 (19,694 2D images and 3,982 3D models), respectively.