Semantic sketch-based 3D model retrieval

Abstract

Query-by-Sketch is an intuitive way to retrieve relevant 3D models given a user sketch. It has many promising application potentials in human computer interaction, 3D animation, game design, etc. However, it is a very challenging task due to the big semantic gap between human-drawn sketches and 3D models. A human-drawn sketch is usually composed of several simplified and exaggerated curves as the iconic representation of an object, while a 3D model of the object is generally an accurate representation of its geometry information. Such big semantic gap makes the search based on a direct 2D-3D comparison suffer low accuracy and high computational cost. In this paper, we propose a novel semantic sketch-based 3D model search to bridge the semantic gap by first recognizing the potential semantic meanings of the user sketch and then performing 2D-3D matching for the 3D models within the predicted categories. The experimental results demonstrate that our approach achieves significant improvements in both search accuracy and efficiency, which further validate that our approach can effectively bridge the semantic gap between user sketches and 3D models.