3D Face Recognition Method Research Articles

PointSurFace: Discriminative point cloud surface feature extraction for 3D face recognition

Due to the geometric information in the 3D face data, the 3D face recognition methods exhibit better robustness against the physical attacks compared to the 2D recognition methods. In this paper, we propose PointSurFace, a 3D face point cloud recognition method integrated with a specially designed facial surface feature extraction (SurF) module. In PointSurFace, SurF comprehensively combines the coordinate positions, normal vectors, and angle relationships, etc., of each patch in the 3D face point cloud, to explore the explicit local geometric information of 3D faces and generate facial surface features. Subsequently, the surface features are fed into an encoder consisting of four set abstraction (SA) modules and three Inverted Residual MLP (InvResMLP) modules to extract more discriminative 3D facial features, where InvResMLP could suppress the overfitting and reduce information loss in the model training. In addition, PointSurFace utilizes channel fusion operations to integrate position information into the extracted 3D facial features, further enhancing the model performance. Experimental results in 3D facial recognition and verification across multiple datasets demonstrate that PointSurFace achieves the best 3D face recognition performance to date. For example, on Lock3DFace, it achieves the state-of-the-art recognition and verification accuracy of 90.03% and 80.01%, respectively, surpassing the previous methods by 2.02% and 3.19%. The ablation studies evaluate each module of PointSurFace, suggesting that the proposed surface feature extraction module significantly enhances the model performance.

Retracted: A 3D Face Modeling and Recognition Method Based on Binocular Stereo Vision and Depth-Sensing Detection

Retracted: A 3D Face Modeling and Recognition Method Based on Binocular Stereo Vision and Depth-Sensing Detection

3D Face Recognition: Two Decades of Progress and Prospects

Three-dimensional (3D) face recognition has been extensively investigated in the last two decades due to its wide range of applications in many areas, such as security and forensics. Numerous methods have been proposed to deal with the challenges faced by 3D face recognition, such as facial expressions, pose variations, and occlusions. These methods have achieved superior performances on several small-scale datasets, including FRGC v2.0, Bosphorus, BU-3DFE, and Gavab. However, deep learning–based 3D face recognition methods are still in their infancy due to the lack of large-scale 3D face datasets. To stimulate future research in this area, we present a comprehensive review of the progress achieved by both traditional and deep learning–based 3D face recognition methods in the last two decades. Comparative results on several publicly available datasets under different challenges of facial expressions, pose variations, and occlusions are also presented.

Face ShapeNets for 3D Face Recognition

In this paper, we present a deep learning-based method for 3D face recognition. Unlike some previous works, our process does not rely on face representation methods as a proxy step to be accepted by Convolutional Neural Networks (CNNs). Applying 2D CNNs to irregular domains such as 3D meshes is challenging. Therefore, we employed 3D ShapeNets to recognize faces covering the full 3D shape since 3D face datasets are available and 3D data augmentation techniques to enlarge 3D datasets are widespread. The reduced size of 3D datasets is overcome by an appropriate 3D data augmentation to train our model. 3D ShapeNets are commonly used to recognize and analyse objects. To the best of the authors’ knowledge, this is the first time they are used for face recognition. This research work focuses on the preprocessing step. Whatever the nature of the face image is (either 2D or 3D representation), and whatever the acquisition conditions are, a 3D regular mesh of each input face image is first generated. Furthermore, meshes are converted to voxels to get the occupancy grid across all possible views. Finally, 3D ShapeNets are trained and recognition tests are performed. Indeed, 3D ShapeNets prove the efficiency and efficacy of 3D shape analysis task in 3D face recognition. The experimental results show that 3D face recognition using deep 3D CNNs such as 3D ShapeNets leads to significant improvement over the state-of-the-art performance on LFPW, BU3DFE, and FRAV3D datasets, with competitive recognition rates of 94.25%, 97.9% and 98.31% respectively.

A 3D Face Modeling and Recognition Method Based on Binocular Stereo Vision and Depth-Sensing Detection

The human face is an important channel for human interaction and is the most expressive part of the human body with personalized and diverse characteristics. To improve the modeling speed as well as recognition speed and accuracy of 3D faces, this paper proposes a laser scanning binocular stereo vision imaging method based on binocular stereo vision and depth-sensing detection method. Existing matching methods have a weak anti-interference capability, cannot identify the spacing of objects quickly and efficiently, and have too large errors. Using the technique mentioned in this paper can remedy these shortcomings by mainly using the laser lines scanned onto the object as strong feature cues for left and right views for binocular vision matching and thus for depth measurement perception of the object. The experiments in this paper first explore the effect of parameter variation of the laser scanning binocular vision imaging system on the accuracy of object measurement depth values in an indoor environment; the experimental data are selected from 68 face feature points for modeling, cameras that photograph faces, stereo calibration of the cameras using calibration methods to obtain the parameters of the binocular vision imaging system, and after stereo correction of the left and right camera images, the laser line scan light is used as a pixel matching strong features for binocular camera matching, which can achieve fast recognition and high accuracy and then select the system parameters with the highest accuracy to perform 3D reconstruction experiments on actual target objects in the indoor environment to achieve faster recognition.

Few-data guided learning upon end-to-end point cloud network for 3D face recognition

Deep-learning-based 3D face recognition methods have developed vigorously in recent years, while the potential of these methods is being exploited in more and more scenarios. In this paper, an end-to-end deep learning network entitled Sur3dNet-Face for point-cloud-based 3D face recognition is proposed. The method uses PointNet, which is a successful point cloud classification solution but performs unexpectedly in face recognition, as the backbone. To adapt the backbone to 3D face recognition, modifications in network architecture and a few-data guided learning framework based on Gaussian process morphable model is supplemented. Instead of mass data in multiple datasets for training, our method takes only Spring2003 subset of FRGC v2.0 for training which contains 943 facial scans and the network is well trained with such a small amount of real data. The processing time to generate face representation is less than 0.15 s. Without fine-tuning on the test set, the Rank-1 Recognition Rate (RR1) is achieved as follows: 98.85% on FRGC v2.0 dataset and 99.33% on Bosphorus dataset, which proves the effectiveness and the potentiality of our method. When facing scenarios with limited resource, the proposed method is expected to give a competitive performance.

A comprehensive survey on 3D face recognition methods

3D face recognition (3DFR) has emerged as an effective means of characterizing facial identity over the past several decades. Depending on the types of techniques used in recognition, these methods are categorized into traditional and modern. The former generally extract distinctive facial features (e.g. global, local, and hybrid features) for matching, whereas the latter rely primarily on deep learning to perform 3DFR in an end-to-end way. Many literature surveys have been carried out reviewing either traditional or modern methods alone, while only a few studies are conducted simultaneously on both of them. This survey presents a state-of-the-art for 3DFR covering both traditional and modern methods, focusing on the techniques used in face processing, feature extraction, and classification. In addition, we review some specific face recognition challenges, including pose, illumination, expression variations, self-occlusion, and spoofing attack. The commonly used 3D face datasets have been summarized as well.

Research on an Improved MB-LBP 3D Face Recognition Method

In order to improve the accuracy and speed of 3D face recognition, this paper proposes an improved MB-LBP 3D face recognition method. First, the MB-LBP algorithm is used to extract the features of 3D face depth image, then the average information entropy algorithm is used to extract the effective feature information of the image, and finallythe Support Vector Machine algorithm is used to identify the extracted effective information. The recognition rate on the Texas 3DFRD database is 96.88%, and the recognition time is 0.025s. The recognition rate in the self-made depth library is 96.36%, and the recognition time is 0.02s.It can be seen from the experimental results that the algorithm in this paper has better performance in terms of accuracy and speed.

3D Expression-Invariant Face Verification Based on Transfer Learning and Siamese Network for Small Sample Size

Three-dimensional (3D) face recognition has become a trending research direction in both industry and academia. However, traditional facial recognition methods carry high computational costs and face data storage costs. Deep learning has led to a significant improvement in the recognition rate, but small sample sizes represent a new problem. In this paper, we present an expression-invariant 3D face recognition method based on transfer learning and Siamese networks that can resolve the small sample size issue. First, a landmark detection method utilizing the shape index was employed for facial alignment. Then, a convolutional network (CNN) was constructed with transfer learning and trained with the aligned 3D facial data for face recognition, enabling the CNN to recognize faces regardless of facial expressions. Following that, the weighted trained CNN was shared by a Siamese network to build a 3D facial recognition model that can identify faces even with a small sample size. Our experimental results showed that the proposed method reached a recognition rate of 0.977 on the FRGC database, and the network can be used for facial recognition with a single sample.

Past, Present, and Future of Face Recognition: A Review

Face recognition is one of the most active research fields of computer vision and pattern recognition, with many practical and commercial applications including identification, access control, forensics, and human-computer interactions. However, identifying a face in a crowd raises serious questions about individual freedoms and poses ethical issues. Significant methods, algorithms, approaches, and databases have been proposed over recent years to study constrained and unconstrained face recognition. 2D approaches reached some degree of maturity and reported very high rates of recognition. This performance is achieved in controlled environments where the acquisition parameters are controlled, such as lighting, angle of view, and distance between the camera–subject. However, if the ambient conditions (e.g., lighting) or the facial appearance (e.g., pose or facial expression) change, this performance will degrade dramatically. 3D approaches were proposed as an alternative solution to the problems mentioned above. The advantage of 3D data lies in its invariance to pose and lighting conditions, which has enhanced recognition systems efficiency. 3D data, however, is somewhat sensitive to changes in facial expressions. This review presents the history of face recognition technology, the current state-of-the-art methodologies, and future directions. We specifically concentrate on the most recent databases, 2D and 3D face recognition methods. Besides, we pay particular attention to deep learning approach as it presents the actuality in this field. Open issues are examined and potential directions for research in facial recognition are proposed in order to provide the reader with a point of reference for topics that deserve consideration.

Research on 3D face recognition method based on LBP and SVM

In order to improve the accuracy and speed of three-dimensional face recognition, this paper proposes a three-dimensional face recognition method combining LBP and SVM. First, the LBP algorithm is used to extract the feature information of the three-dimensional face depth image, then the SVM algorithm is used to classify the feature information. By selecting samples in the Texas 3DFRD three-dimensional face depth database and the self-made depth database, the experiment proves that the algorithm has a better recognition rate and shorter time consumption.

Point cloud based deep convolutional neural network for 3D face recognition

Face recognition is a challenging task as it has to deal with several issues such as illumination orientation and variability among the different faces. Previous works have shown that 3D face is a robust biometric trait and is less sensitive to light and pose variations. Also due to availability of inexpensive sensors and new 3D data acquisition techniques it has become easy to capture 3D data. A 3D depth image of a face is found to be rich in information and biometric recognition performance can be enhanced by using 3D face data along with convolutional neural network. However the shortcoming of this approach is the conversion of 3D data to lower dimensions (depth image) which suffer from loss of geometric information and the network becomes computationally expensive. In this work we endeavor to apply deep learning method for 3D face recognition and propose a deep convolutional neural network based on PointNet architecture which consumes point cloud directly as input and siamese network for similarity learning. Further we propose a solution to the issue of a limited database by applying data augmentation at the point cloud level. Our proposed technique shows encouraging performance on Bosphorus and IIT Indore 3D face databases.

ALCN: Adaptive Local Contrast Normalization

To make Robotics and Augmented Reality applications robust to illumination changes, the current trend is to train a Deep Network with training images captured under many different lighting conditions. Unfortunately, creating such a training set is a very unwieldy and complex task. We therefore propose a novel illumination normalization method that can easily be used for different problems with challenging illumination conditions. Our preliminary experiments show that among current normalization methods, the Difference-of-Gaussians method remains a very good baseline, and we introduce a novel illumination normalization model that generalizes it. Our key insight is then that the normalization parameters should depend on the input image, and we aim to train a Convolutional Neural Network to predict these parameters from the input image. This, however, cannot be done in a supervised manner, as the optimal parameters are not known a priori. We thus designed a method to train this network jointly with another network that aims to recognize objects under different illuminations: The latter network performs well when the former network predicts good values for the normalization parameters. We show that our method significantly outperforms standard normalization methods and would also be appear to be universal since it does not have to be re-trained for each new application. Our method improves the robustness to light changes of state-of-the-art 3D object detection and face recognition methods.

GAN-Based Blur Restoration for Finger Wrinkle Biometrics System

Existing methods for iris, fingerprint, and 3D face recognition in mobile devices have constraints in terms of price and size owing to their use of additional cameras, lighting, and sensors. Additionally, visible light, camera-based 2D face recognition, palm print recognition, touchless fingerprint recognition, and finger knuckle print recognition are difficult to be used in mobile devices due to limitations in recognition performance and user inconvenience. In response to these problems, studies have been conducted on finger wrinkle recognition in mobile devices; however, image quality is often reduced by motion blurring caused by the movement of the camera or the user’s finger, thereby reducing recognition performance. This study proposes a method for restoring and recognizing motion-blurred finger wrinkle images based on a generative adversarial network and deep convolutional neural network. Experiments were performed using two types of finger wrinkle databases, which were custom-made from images of 33 people captured by smart phone cameras (Dongguk mobile finger wrinkle database versions 1 and 2, denoted as DMFW-DB1 and DMFW-DB2, respectively). The results demonstrated high restoration and recognition performance in comparison with the state-of-the-art methods.

3D Face Factorisation for Face Recognition Using Pattern Recognition Algorithms

Abstract The face is the preferable biometrics for person recognition or identification applications because person identifying by face is a human connate habit. In contrast to 2D face recognition, 3D face recognition is practically robust to illumination variance, facial cosmetics, and face pose changes. Traditional 3D face recognition methods describe shape variation across the whole face using holistic features. In spite of that, taking into account facial regions, which are unchanged within expressions, can acquire high performance 3D face recognition system. In this research, the recognition analysis is based on defining a set of coherent parts. Those parts can be considered as latent factors in the face shape space. Non-negative matrix Factorisation technique is used to segment the 3D faces to coherent regions. The best recognition performance is achieved when the vertices of 20 face regions are utilised as a feature vector for recognition task. The region-based 3D face recognition approach provides a 96.4% recognition rate in FRGCv2 dataset.

RGB-D face recognition under various conditions via 3D constrained local model

This research proposes a method for 3D face recognition in various conditions using 3D constrained local model (CLM-Z). In this method, a combination of 2D images (RGBs) and depth images (Ds) captured by Kinect has been used. After detecting the face and smoothing the depth image, CLM-Z model has been used to model and detect the important points of the face. These points are described using Histogram of Oriented Gradients (HOG), Local Binary Patterns (LBP), and 3D Local Binary Patterns (3DLBP). Finally, each face is recognized by a Support Vector Machine (SVM). The challenging situations are changes of lighting, facial expression and head pose. The results on CurtinFaces and IIIT-D datasets demonstrate that the proposed method outperformed state-of-the-art methods under illumination, expression and pitch pose conditions and comparable results were obtained in other cases. Additionally, our proposed method is robust even when the training data has not been carefully collected.

3D face recognition method against gesture and expression change

3D face recognition method against gesture and expression change

Expression-insensitive 3D face recognition by the fusion of multiple subject-specific curves

Abstract This study proposes a 3D face recognition method using multiple subject-specific curves insensitive to intra-subject distortions caused by expression variations. Considering that most sharp variances in facial convex regions are closely related to the bone structure, the convex crest curves are first extracted as the most vital subject-specific facial curves based on the principal curvature extrema in convex local surfaces. Then, the central profile curve and the horizontal contour curve passing through the nose tip are detected by using the precise localization of the nose tip and symmetry plane. Based on their discriminative power and robustness to expression changes, the three types of curves are fused with appropriate weights at the feature-level and used for matching 3D faces with the iterative closest point algorithm. The combination of multiple expression-insensitive curves is complementary and provides sufficient and stable facial surface features for face recognition. In addition, for each convex crest curve, an expression-irrelevant factor is assigned as the adaptive weight to improve the face matching performance. The results of experiments using two public 3D databases, GavabDB and BU-3DFE, demonstrate the effectiveness of the proposed method, and its recognition rates on both databases reflect an encouraging performance.

A survey of local feature methods for 3D face recognition

One of the main modules in a face recognition system is feature extraction, which has a significant effect on the whole system performance. In the past decades, various types of feature extractors and descriptors have been proposed for 3D face recognition. Although several literature reviews have been carried out on 3D face recognition algorithms, only a few studies have been performed on feature extraction methods. The latter have a vital role to overcome degradation conditions, such as face expression variations and occlusions. Depending on the types of features used in 3D face recognition, these methods can be divided into two categories: global and local feature-based methods. Local feature-based methods have been effectively applied in the literature, as they are more robust to occlusions and missing data. This survey presents a state-of-the-art for 3D face recognition using local features, with the main focus being the extraction of these features.

Pose-invariant 3D face recognition using half face

Pose variations are still challenging problems in 3D face recognition because large pose variations will cause self-occlusion and result in missing data. In this paper, a new method for pose-invariant 3D face recognition is proposed to handle significant pose variations. For pose estimation and registration, a coarse-to-fine strategy is proposed to detect landmarks under large yaw variations. At the coarse search step, candidate landmarks are detected using HK curvature analysis and subdivided according to a facial geometrical structure-based classification strategy. At the fine search step, candidate landmarks are identified and labeled by comparing with a Facial Landmark Model. By using the half face matching, we perform the matching step with respect to frontal scans and side scans. Experiments carried out on the Bosphorus and UND/FRGC v2.0 databases show that our method has high accuracy and robustness to pose variations.