Deep Learning Based Face Recognition Research Articles

Face Recognition System for Control Access to Restrictive Domain

Recent progress in computer vision applied to facial analysis has led to state-of-the-art face detection and facial feature extraction models. A cautious implementation of these models into face recognition pipelines can enable achieving superior performances and popularized daily applications of face recognition in a variety of domains. However, modern face recognition system is a multi-steps process including face detection, feature extraction and classification model. Developing a high-performance face recognition application generalizing on local data set remains challenging. In this paper, we present Deep learning based face recognition system employing MTCNN for face detection and FaceNet for feature extraction. We compare KNN and SVM classification models trained on the facial features extracted from prepared labeled faces. Both models demonstrated almost 100% accuracy on static test faces. Moreover, as face pose get more pronounced, far above 30°, both SVM and KNN models demonstrate efficient recognition rate of 95.95% and 96.67% respectively. Real-time evaluation shows less than 1% deviation from the static performances with both classifiers on less 30° tilted images.

Sensitive loss: Improving accuracy and fairness of face representations with discrimination-aware deep learning

We propose a discrimination-aware learning method to improve both the accuracy and fairness of biased face recognition algorithms. The most popular face recognition benchmarks assume a distribution of subjects without paying much attention to their demographic attributes. In this work, we perform a comprehensive discrimination-aware experimentation of deep learning-based face recognition. We also propose a notational framework for algorithmic discrimination with application to face biometrics. The experiments include three popular face recognition models and three public databases composed of 64,000 identities from different demographic groups characterized by sex and ethnicity. We experimentally show that learning processes based on the most used face databases have led to popular pre-trained deep face models that present evidence of strong algorithmic discrimination. Finally, we propose a discrimination-aware learning method, Sensitive Loss, based on the popular triplet loss function and a sensitive triplet generator. Our approach works as an add-on to pre-trained networks and is used to improve their performance in terms of average accuracy and fairness. The method shows results comparable to state-of-the-art de-biasing networks and represents a step forward to prevent discriminatory automatic systems.

Internet of Things with Deep Learning-Based Face Recognition Approach for Authentication in Control Medical Systems.

Internet of Things (IoT) with deep learning (DL) is drastically growing and plays a significant role in many applications, including medical and healthcare systems. It can help users in this field get an advantage in terms of enhanced touchless authentication, especially in spreading infectious diseases like coronavirus disease 2019 (COVID-19). Even though there is a number of available security systems, they suffer from one or more of issues, such as identity fraud, loss of keys and passwords, or spreading diseases through touch authentication tools. To overcome these issues, IoT-based intelligent control medical authentication systems using DL models are proposed to enhance the security factor of medical and healthcare places effectively. This work applies IoT with DL models to recognize human faces for authentication in smart control medical systems. We use Raspberry Pi (RPi) because it has low cost and acts as the main controller in this system. The installation of a smart control system using general-purpose input/output (GPIO) pins of RPi also enhanced the antitheft for smart locks, and the RPi is connected to smart doors. For user authentication, a camera module is used to capture the face image and compare them with database images for getting access. The proposed approach performs face detection using the Haar cascade techniques, while for face recognition, the system comprises the following steps. The first step is the facial feature extraction step, which is done using the pretrained CNN models (ResNet-50 and VGG-16) along with linear binary pattern histogram (LBPH) algorithm. The second step is the classification step which can be done using a support vector machine (SVM) classifier. Only classified face as genuine leads to unlock the door; otherwise, the door is locked, and the system sends a notification email to the home/medical place with detected face images and stores the detected person name and time information on the SQL database. The comparative study of this work shows that the approach achieved 99.56% accuracy compared with some different related methods.

Perturbation Inactivation Based Adversarial Defense for Face Recognition

Deep learning-based face recognition models are vulnerable to adversarial attacks. To curb these attacks, most defense methods aim to improve the robustness of recognition models against adversarial perturbations. However, the generalization capacities of these methods are quite limited. In practice, they are still vulnerable to unseen adversarial attacks. Deep learning models are fairly robust to general perturbations, such as Gaussian noises. A straightforward approach is to inactivate the adversarial perturbations so that they can be easily handled as general perturbations. In this paper, a plug-and-play adversarial defense method, named perturbation inactivation (PIN), is proposed to inactivate adversarial perturbations for adversarial defense. We discover that the perturbations in different subspaces have different influences on the recognition model. There should be a subspace, called the immune space, in which the perturbations have fewer adverse impacts on the recognition model than in other subspaces. Hence, our method estimates the immune space and inactivates the adversarial perturbations by restricting them to this subspace. The proposed method can be generalized to unseen adversarial perturbations since it does not rely on a specific kind of adversarial attack method. This approach not only outperforms several state-of-the-art adversarial defense methods but also demonstrates a superior generalization capacity through exhaustive experiments. Moreover, the proposed method can be successfully applied to four commercial APIs without additional training, indicating that it can be easily generalized to existing face recognition systems. The source code is available at https://github.com/RenMin1991/Perturbation-Inactivate..

A Study of Deep Learning-Based Face Recognition Models for Sibling Identification.

Accurate identification of siblings through face recognition is a challenging task. This is predominantly because of the high degree of similarities among the faces of siblings. In this study, we investigate the use of state-of-the-art deep learning face recognition models to evaluate their capacity for discrimination between sibling faces using various similarity indices. The specific models examined for this purpose are FaceNet, VGGFace, VGG16, and VGG19. For each pair of images provided, the embeddings have been calculated using the chosen deep learning model. Five standard similarity measures, namely, cosine similarity, Euclidean distance, structured similarity, Manhattan distance, and Minkowski distance, are used to classify images looking for their identity on the threshold defined for each of the similarity measures. The accuracy, precision, and misclassification rate of each model are calculated using standard confusion matrices. Four different experimental datasets for full-frontal-face, eyes, nose, and forehead of sibling pairs are constructed using publicly available HQf subset of the SiblingDB database. The experimental results show that the accuracy of the chosen deep learning models to distinguish siblings based on the full-frontal-face and cropped face areas vary based on the face area compared. It is observed that VGGFace is best while comparing the full-frontal-face and eyes—the accuracy of classification being with more than 95% in this case. However, its accuracy degrades significantly when the noses are compared, while FaceNet provides the best result for classification based on the nose. Similarly, VGG16 and VGG19 are not the best models for classification using the eyes, but these models provide favorable results when foreheads are compared.

Model Inversion Attack: Analysis under Gray-box Scenario on Deep Learning based Face Recognition System

In a wide range of ML applications, the training data contains privacy-sensitive information that should be kept secure. Training the ML systems by privacy-sensitive data makes the ML model inherent to the data. As the structure of the model has been fine-tuned by training data, the model can be abused for accessing the data by the estimation in a reverse process called model inversion attack (MIA). Although, MIA has been applied to shallow neural network models of recognizers in literature and its threat in privacy violation has been approved, in the case of a deep learning (DL) model, its efficiency was under question. It was due to the complexity of a DL model structure, big number of DL model parameters, the huge size of training data, big number of registered users to a DL model and thereof big number of class labels. This research work first analyses the possibility of MIA on a deep learning model of a recognition system, namely a face recognizer. Second, despite the conventional MIA under the white box scenario of having partial access to the users' non-sensitive information in addition to the model structure, the MIA is implemented on a deep face recognition system by just having the model structure and parameters but not any user information. In this aspect, it is under a semi-white box scenario or in other words a gray-box scenario. The experimental results in targeting five registered users of a CNN-based face recognition system approve the possibility of regeneration of users' face images even for a deep model by MIA under a gray box scenario. Although, for some images the evaluation recognition score is low and the generated images are not easily recognizable, but for some other images the score is high and facial features of the targeted identities are observable. The objective and subjective evaluations demonstrate that privacy cyber-attack by MIA on a deep recognition system not only is feasible but also is a serious threat with increasing alert state in the future as there is considerable potential for integration more advanced ML techniques to MIA.

Cross-resolution face recognition adversarial attacks

• The performance of face recognition systems degrades in cross-resolution scenarios. • Deep models can be jeopardized by adversarial attacks based on different norms. • The success rate of the attacks is higher, on average, at low input resolutions. • Deep representations attacks represent a severe menace to face recognition systems. • Cross-resolution training can improve the robustness of deep models. Face Recognition is among the best examples of computer vision problems where the supremacy of deep learning techniques compared to standard ones is undeniable. Unfortunately, it has been shown that they are vulnerable to adversarial examples - input images to which a human imperceptible perturbation is added to lead a learning model to output a wrong prediction. Moreover, in applications such as biometric systems and forensics, cross-resolution scenarios are easily met with a non-negligible impact on the recognition performance and adversary’s success. Despite the existence of such vulnerabilities set a harsh limit to the spread of deep learning-based face recognition systems to real-world applications, a comprehensive analysis of their behavior when threatened in a cross-resolution setting is missing in the literature. In this context, we posit our study, where we harness several of the strongest adversarial attacks against deep learning-based face recognition systems considering the cross-resolution domain. To craft adversarial instances, we exploit attacks based on three different metrics, i.e., L 1 , L 2 , and L ∞ , and we study the resilience of the models across resolutions. We then evaluate the performance of the systems against the face identification protocol, open- and close-set. In our study, we find that the deep representation attacks represents a much dangerous menace to a face recognition system than the ones based on the classification output independently from the used metric. Furthermore, we notice that the input image’s resolution has a non-negligible impact on an adversary’s success in deceiving a learning model. Finally, by comparing the performance of the threatened networks under analysis, we show how they can benefit from a cross-resolution training approach in terms of resilience to adversarial attacks.

Low-level features based 2D face recognition using machine learning

In modern times, when deep learning-based face recognition is highly in demand, this paper presents machine learning techniques using the low-level feature extraction. Deep learning has a drawback of getting things done in a black-box, however extraction of low-level features viz. histogram of oriented gradients (HOG), speeded up robust features (SURF), and local binary patterns (LBPs) with a machine learning-based classification model presents higher simplicity. This paper presents the experimental demonstrations using 22 variations of machine learning models. Two face datasets, namely, Bosphorus and UMBDB are used for evaluating different classification models. Four experimentations are shown in the implementation section to demonstrate the effect of feature extraction, discretisation, feature variation, and noise in the image under probe. The subspace discriminant ensemble model yields the highest efficiency in classifying faces using HOG features. The effect of various noise attacks on the probe image is shown in the last experimentation.

Identity-Aware Face Super-Resolution for Low-Resolution Face Recognition

Although deep learning-based face recognition techniques have achieved amazing performance in recent years, low-resolution (LR) face recognition remains challenging. In this letter, we address this problem by proposing an identity-aware face super-resolution network to recover identity information of LR faces. To learn identity-aware features effectively, the identity features are explicitly disentangled to two orthogonal components: the magnitude and angle of features that project identity features to a hypersphere space. We show that the magnitude of features is related to the quality of a face. The proposed approach shows its superiority on recovering identity-related textures which are beneficial to recover identity information for recognition. Extensive experiments demonstrate the effectiveness of the proposed algorithm in LR face recognition.

사용자 인증을 위한 딥러닝 기반 얼굴인식 기술 동향

사용자 인증을 위한 딥러닝 기반 얼굴인식 기술 동향

A survey on deep learning based face recognition

Deep learning, in particular the deep convolutional neural networks, has received increasing interests in face recognition recently, and a number of deep learning methods have been proposed. This paper summarizes about 330 contributions in this area. It reviews major deep learning concepts pertinent to face image analysis and face recognition, and provides a concise overview of studies on specific face recognition problems, such as handling variations in pose, age, illumination, expression, and heterogeneous face matching. A summary of databases used for deep face recognition is given as well. Finally, some open challenges and directions are discussed for future research.

Detecting and Mitigating Adversarial Perturbations for Robust Face Recognition

Deep neural network (DNN) architecture based models have high expressive power and learning capacity. However, they are essentially a black box method since it is not easy to mathematically formulate the functions that are learned within its many layers of representation. Realizing this, many researchers have started to design methods to exploit the drawbacks of deep learning based algorithms questioning their robustness and exposing their singularities. In this paper, we attempt to unravel three aspects related to the robustness of DNNs for face recognition: (i) assessing the impact of deep architectures for face recognition in terms of vulnerabilities to attacks, (ii) detecting the singularities by characterizing abnormal filter response behavior in the hidden layers of deep networks; and (iii) making corrections to the processing pipeline to alleviate the problem. Our experimental evaluation using multiple open-source DNN-based face recognition networks, and three publicly available face databases demonstrates that the performance of deep learning based face recognition algorithms can suffer greatly in the presence of such distortions. We also evaluate the proposed approaches on four existing quasi-imperceptible distortions: DeepFool, Universal adversarial perturbations, $$l_2$$ , and Elastic-Net (EAD). The proposed method is able to detect both types of attacks with very high accuracy by suitably designing a classifier using the response of the hidden layers in the network. Finally, we present effective countermeasures to mitigate the impact of adversarial attacks and improve the overall robustness of DNN-based face recognition.

Unconstrained Face Recognition Using a Set-to-Set Distance Measure on Deep Learned Features

Recently considerable efforts have been dedicated to unconstrained face recognition, which requires to identify faces “in the wild” for a set of images and/or video frames captured without human intervention. Unlike traditional face recognition that compares one-to-one media (either a single image or a video frame) only, we encounter a problem of matching sets with heterogeneous contents containing both images and videos. In this paper, we propose a novel set-to-set (S2S) distance measure to calculate the similarity between two sets with the aim to improve the recognition accuracy for faces with real-world challenges, such as extreme poses or severe illumination conditions. Our S2S distance adopts the kNN -average pooling for the similarity scores computed on all the media in two sets, making the identification far less susceptible to the poor representations (outliers) than traditional feature-average pooling and score-average pooling. Furthermore, we show that various metrics can be embedded into our S2S distance framework, including both predefined and learned ones. This allows to choose the appropriate metric depending on the recognition task in order to achieve the best results. To evaluate the proposed S2S distance, we conduct extensive experiments on the challenging set-based IJB-A face data set, which demonstrate that our algorithm achieves the state-of-the-art results and is clearly superior to the baselines, including several deep learning-based face recognition algorithms.

Unravelling Robustness of Deep Learning Based Face Recognition Against Adversarial Attacks

Deep neural network (DNN) architecture based models have high expressive power and learning capacity. However, they are essentially a black box method since it is not easy to mathematically formulate the functions that are learned within its many layers of representation. Realizing this, many researchers have started to design methods to exploit the drawbacks of deep learning based algorithms questioning their robustness and exposing their singularities. In this paper, we attempt to unravel three aspects related to the robustness of DNNs for face recognition: (i) assessing the impact of deep architectures for face recognition in terms of vulnerabilities to attacks inspired by commonly observed distortions in the real world that are well handled by shallow learning methods along with learning based adversaries; (ii) detecting the singularities by characterizing abnormal filter response behavior in the hidden layers of deep networks; and (iii) making corrections to the processing pipeline to alleviate the problem. Our experimental evaluation using multiple open-source DNN-based face recognition networks, including OpenFace and VGG-Face, and two publicly available databases (MEDS and PaSC) demonstrates that the performance of deep learning based face recognition algorithms can suffer greatly in the presence of such distortions. The proposed method is also compared with existing detection algorithms and the results show that it is able to detect the attacks with very high accuracy by suitably designing a classifier using the response of the hidden layers in the network. Finally, we present several effective countermeasures to mitigate the impact of adversarial attacks and improve the overall robustness of DNN-based face recognition.

Improving Deep Learning Feature with Facial Texture Feature for Face Recognition

Face recognition in the reality, is a challenging problem, due to varieties in illumination, background, pose etc. Recently, the deep learning based face recognition algorithm is able to learn effective face features to obtain a very impressive performance. However, this kind of face recognition algorithm completely relies on the machine learning based face features, while ignores the useful experience in hand-craft features which have been studied in a long period. Therefore, a face recognition based on facial texture feature aided deep learning feature (FTFA-DLF) is proposed in this paper. The proposed FTFA-DLF is able to combine the benefits of deep learning and hand-craft features. In the proposed FTFA-DLF method, the hand-craft features are texture features extracted from the eyes, nose, and mouth regions. Then, the hand-craft features are used to aid deep learning features by adding both deep learning and hand-craft features into the objective function layer, which adaptively adjusts the deep learning features so that it can better cooperate with the hand-craft features and obtain a better face recognition performance. Experimental results show that the proposed face recognition algorithm on the LFW face database to achieve the accuracy rate of 97.02%.

Improving Shadow Suppression for Illumination Robust Face Recognition.

2D face analysis techniques, such as face landmarking, face recognition and face verification, are reasonably dependent on illumination conditions which are usually uncontrolled and unpredictable in the real world. The current massive data-driven approach, e.g., deep learning-based face recognition, requires a huge amount of labeled training face data that hardly cover the infinite lighting variations that can be encountered in real-life applications. An illumination robust preprocessing method thus remains a very interesting but also a significant challenge in reliable face analysis. In this paper we propose a novel model driven approach to improve lighting normalization of face images. Specifically, we propose to build the underlying reflectance model which characterizes interactions between skin surface, lighting source and camera sensor, and elaborate the formation of face color appearance. The proposed illumination processing pipeline enables generation of the Chromaticity Intrinsic Image (CII) in a log chromaticity space which is robust to illumination variations. Moreover, as an advantage over most prevailing methods, a photo-realistic color face image is subsequently reconstructed, which eliminates a wide variety of shadows whilst retaining the color information and identity details. Experimental results under different scenarios and using various face databases show the effectiveness of the proposed approach in dealing with lighting variations, including both soft and hard shadows, in face recognition.

Smart Campus Care and Guiding With Dedicated Video Footprinting Through Internet of Things Technologies

In this paper, we propose a smart campus care and guiding framework with deep learning-based face recognition, called DeepGuiding, for students through Internet of Things technologies. The DeepGuiding framework can construct the dedicated video trajectory of a campus student, where the recorded video for each student can be automatically classified to achieve efficient footprint review as necessary. In addition, DeepGuiding can provide time-efficient indoor and outdoor guiding in a campus to quickly reach places, meet friends, and find students. To the best of our knowledge, DeepGuiding is the first campus care and guiding system which provides the following features: 1) it achieves the seamless outdoor and indoor navigation between buildings in a campus; 2) it keeps additional construction cost low by utilizing existing surveillance cameras in a campus; and 3) it reduces the total searching time for finding a specific event/target in a campus by alleviating time-consuming labor overhead to review a huge amount of video data. An Android-based prototype using iBeacon indoor localization and global positioning system outdoor positioning with surveillance cameras is implemented to verify the feasibility and superiority of our DeepGuiding framework. The Experimental results show that DeepGuiding outperforms existing face recognition methods and can achieve high recognition accuracy for students not close to surveillance cameras.

Robust and efficient face recognition via low-rank supported extreme learning machine

Recently, face recognition algorithms have made great progress in various real-world applications, e.g., authentication and criminal investigation. Deep-learning offers an end-to-end paradigm for vision recognition tasks and achieves good performance. However, designing and training the complex network architecture are time-consuming and labor-intensive. Moreover, under complex scenarios, illumination change, noise or occlusion in images degrade the performance of recognition algorithms. In order to ameliorate these issues, we propose an efficient three-layered low-rank supported extreme learning machine (LSELM) algorithm for face recognition which improves the recognition performance under complex scenarios with high efficiency. In the first layer, a given probe sample is clustered into certain training subspace as pre-clustering. In the second layer, with this subspace, a low-rank subspace of probe sample as robust feature which is insensitive to disguise, noise, variant expression or illumination will be recovered by low-rank decomposition. Furthermore, these low-rank discriminative features are coded to support training a forward neural network termed LSELM. Experimental results indicate that the proposed approach is on par with some deep-learning based face recognition algorithms on recognition performance but with less time complexity over some popular face datasets e.g., AR, Extend Yale-B, CMU PIE and LFW datasets.

Study of assessment of cognitive ability of human brain using deep learning

Most of the biometric techniques are conventional biometrics employing some neural network architecture consisting of suitable number of hidden layers in their training process. The recognition accuracy has no longer become a measure to ensure the method of recognition is robust. Robustness has also been attempted by using hybrid methods for training such as neuro-fuzzy method and applying some optimization technique. The traditional use of the learning concept has almost been saturated in the field of computer vision and face recognition. An emerging concept of learning method that is being researched internationally in several cognitive, computer vision and data classification tasks is deep learning, which is subfield of machine learning inspired by structure and function of artificial neural network. This paper suggests using deep learning in assessment of cognitive ability of human brain. We plan to train thousands of facial images into our image database. Deep learning was compared to shallow learning in face recognition task done for real time applications; cognitive ability assessment and inference for different age group and gender; study of reaction time, etc. A sample size of 380 persons was tested in real time deep learning based face recognition. Response time and correct identification were recorded that shows potent research scope of deep learning in assessment of cognitive ability of human brain at large scale. The cognitive ability of women was found more than that of women.