Three Dimensional Face Recognition via Surface Harmonic Mapping and Deep Learning

Abstract

In this paper, we propose a general 3D face recognition framework by combining the idea of surface harmonic mapping and deep learning. In particular, given a 3D face scan, we first run the pre-processing pipeline and detect three main facial landmarks (i.e., nose tip and two inner eye corners). Then, harmonic mapping is employed to map the 3D coordinates and differential geometry quantities (e.g., normal vectors, curvatures) of each 3D face scan to a 2D unit disc domain, generating a group of 2D harmonic shape images (HSI). The 2D rotation of the harmonic shape images are removed by using the three detected landmarks. All these pose normalized harmonic shape images are fed into a pre-trained deep convolutional neural network (DCNN) to generate their deep representations. Finally, sparse representation classifier with score-level fusion is used for face similarity measurement and the final decision. The advantage of our method is twofold: (i) it is a general framework and can be easily extended to other surface mapping and deep learning algorithms. (ii) it is registration-free and only needs three landmarks. The effectiveness of the proposed framework was demonstrated on the BU-3DFE database, and reporting a rank-one recognition rate of 89.38% on the whole database.