Rotation Prediction Based Representative View Locating Framework for 3D Object Recognition

Abstract

Finding representative views of 3D objects is a key problem in the field of 3D object analysis. We can obtain most of the crucial information of 3D objects from their representative views. In this paper, we propose a framework for learning the features of multi-view rendered images extracted from 3D objects in order to locate representative views of 3D objects. The learning method includes a reinforcement learning based rotation direction prediction (RDP) method and a deep learning based rotation angle prediction (RAP) method. The RDP uses a deep deterministic policy gradient (DDPG) algorithm to learn rotation policies. We improved DDPG to make RDP more applicable for learning 3D object rotation action. RAP uses a convolutional neural network to predict the rotation angles of representative views. We also propose a 3D object classification network. The network reconstructs the rendered images using an encoder–decoder based rendered image reconstruction method and trains the images composed of the original and reconstructed images. Finally, a series of experiments are conducted to validate the feasibility of the proposed methods. Experimental results show the competitive performance of the proposed framework. • We propose a classification reliability discrimination and a reward mechanism for learning 3D object rotation policies. • We improve the classification performance and demonstrate the optimum angles are oblique angles using RDP and RAP. • With the EDIR method, we reconstruct new rendered images without tiny unique details and improve the generalization ability.