Instance-prototype similarity consistency for unsupervised 2D image-based 3D model retrieval

Abstract

The unsupervised 3D model retrieval is designed to joint the information of well-labeled 2D domain and unlabeled 3D domain to learn collaborative representations. Most existing methods adopted semantic alignment, but were inevitably affected by false pseudo-label. In this paper, we design a novel Instance-Prototype Similarity Consistency Network (IPSC) to guide domain alignment with similarity consistency, which can simultaneously suppress the impact of false pseudo-label information and well reduce the domain discrepancy. IPSC contains two similarity strategies, named Single instance vs Multiple prototypes and Instance-pair vs Single prototype. The first strategy utilizes a single instance as an anchor, and measures the similarities between the anchor and multiple prototypes with the same category but from different domains. The minimization between these similarities can better align the cross-domain prototypes with Kullback–Leibler (KL) divergence than traditional Euclidean similarities. The second strategy utilizes a single prototype as an anchor, and measures the similarities between this anchor and an instance-pair with the same category but from different domains. The minimization between these similarities can conduct the instance-level alignment with KL divergence, which can better suppress the negative effect of noisy pseudo-labels. We conduct various experiments on two datasets, MI3DOR-1 (21000 2D images and 7690 3D models) and MI3DOR-2 (19694 2D images and 3982 3D models), to verify the superiority of our algorithm.