Annotating 3D Models and Their Parts via Deep Feature Embedding

Abstract

Need to organize 3D shape data has prompted studies on comparison and retrieval of 3D shape models. Being able to query 3D shape models by words, in addition to 3D model examples and 2D sketches, would be quite beneficial. This paper proposes a method to associate whole 3D models (e.g., automobile) as well as their parts (e.g., tire, body, engine) with word labels so that the 3D model can be queried by words. The associations between 3D shapes and words are learned from a dataset of 3D models whose whole model and segmented parts are labeled with words. Feature vectors of these words (distributed representation) and feature vectors of whole and partial geometries of 3D models are embedded, by Word Shape embedding Network (WSN) into a common feature embedding space. As the word feature vectors are learned by Word2Vec trained on Wikipedia corpus, the common embedding space can be queried by a wide variety of words that are not included in the labeled 3D model dataset. Experimental evaluation has shown that, with the proposed algorithm, 3D shape can be queried by labels of either whole or part shape, or labels that are semantically close but not included in the original 3D model dataset.