Adaptive semantic transfer network for unsupervised 2D image-based 3D model retrieval

Abstract

Unsupervised 2D image-based 3D model retrieval has been a highlighted research topic to enable flexible retrieval from 2D photos to 3D shapes. Although what methods we have so far have been great progress in this aspect, it still exists some issues in learning discriminative features and to well align the distribution diversity from various domains because of the huge cross-domain interval. According to our paper, we propose an adaptive semantic transfer network (ASTN) to improve the discrimination of feature representations and conveniently narrow the discrepancy of different domains by utilizing the intermediate domain to conduct semantic alignment. Our ASTN composes of the adaptive feature encoding module (AFE) and the dynamic semantic alignment module (DSA). To improve the quality of feature representation, the AFE module deploys a new strategy that trains the learnable parameters on multiple convolutional layers, which can adaptively pay different attentions to these layers. The DSA module dynamically constructs an intermediate domain that aims to convert the familiar direct alignment into the sum of two alignments which are source-intermediate and target-intermediate alignments, effectively narrowing the domain gap and further realizing the semantic alignment. On two arduous datasets, MI3DOR-1 and MI3DOR-2, we design abundant experiments that have demonstrated the effectiveness of our suggested method.