Learning based 3D keypoint detection with local and global attributes in multi-scale space

Abstract

Over the last few decades various methods have been proposed by researchers to extract 3D keypoints from the surface of 3D mesh models, but most of them are geometric ones, which are not flexible enough for various applications. In this paper, we propose a new 3D keypoint detection method based on multi-scale neural network (MSNN), which is a tiny neural network and can effectively merge multi-scale information to detect 3D keypoints. Traditional end-to-end learning systems usually require large-scale dataset to do training. However, there are not enough 3D data with ground truth of 3D keypoints. To solve this problem, we perform delicate preprocessing, which effectively enhance the performance of the MSNN based approach. Numerical experiments show that the proposed MSNN 3D keypoint detector not only outperforms other six state-of-the-art geometric based methods, but also achieves better performance than a learning-based method using random forest.