3D object recognition based on pairwise Multi-view Convolutional Neural Networks

Abstract

With the development of 3D sensors, it will be much easier for us to obtain 3D models, which is prevailing in our future daily life, but up to now, although many 3D object recognition algorithms have been proposed, there are some limitations, including the lack of training samples, hand-crafted feature representation, feature extraction and recognition separately. In this work, we propose a novel pairwise Multi-View Convolutional Neural Network for 3D Object Recognition (PMV-CNN for short), where automatic feature extraction and object recognition are put into a unify CNN architecture. Moreover, since the pairwise network architecture is utilized in PMV-CNN, thus, the requirement of the number of training samples in the original dataset is not severe. In addition, the latent complementary relationships from different views can be highly explored by view pooling. Large scale experiments demonstrate that the pairwise architecture is very useful when the number of labeled training samples is very small. Moreover, it also makes more robust feature extraction. Furthermore, since the end-to-end network architecture is employed in PMV-CNN, thus, the extracted feature is very suitable for 3D object recognition, whose performance is much better than that of hand-crafted features. In a word, the performance of our proposed method outperforms state-of-the-art methods.