Voxel-based 3D face reconstruction and its application to face recognition using sequential deep learning

Abstract

In this paper, a novel 3D face reconstruction technique is proposed along with a sequential deep learning-based framework for face recognition. It uses the voxels generated from the voxelization process. It uses the reflection principle for generating the reconstructed point in 3D using the mid-face plane. From the reconstructed face, a sequential deep learning framework is developed to recognize gender, emotion, occlusion, and person. The developed framework utilizes the concepts of variational autoencoders, bidirectional long short-term memory, and triplet loss training. The sequential deep learning model extracts and refines the reconstructed voxels by generating deep features. The support vector machine is applied to deep features for the final prediction. The proposed 3D face recognition system is compared with the three well-known deep learning approaches over three occluded datasets. Experimental results show that the proposed 3D face recognition technique is invariant to occlusion and facial expression. The proposed technique recognizes the gender with accuracy of 97.28%, 92.12%, and 94.44%, emotion with accuracy of 94.57%, 87.78%, and 89.95%, occlusion with accuracy of 94.02%, 81.26%, and 89.85% and person face with accuracy of 90.01%, 78.21%, and 85.68% for Bosphorus, UMBDB and KinectFaceDB datasets respectively. The proposed framework performs better than state-of-the-art approaches in terms of computational time as well as face recognition accuracy.