Labeled Faces In The Wild Research Articles

Orthogonality Loss: Learning Discriminative Representations for Face Recognition

Convolutional neural networks have achieved excellent performance on face recognition (FR) by learning the high discriminative features with advanced loss functions. These improved loss functions share the similar idea for maximizing inter-class variance or minimizing intra-class variance. In this article, from a different perspective, we consider enlarging the inter-class variance by directly penalizing weight vectors of last fully connected layer, which represent the center of classes. To the end, we propose Orthogonality loss as an elegant penalty item appends to common classification loss to learn the discriminative representations. The main idea is that in order for weight vectors to be discriminative, it should be as close as possible to be orthogonal to each other in the vector space. More specifically, the optimization objective of Orthogonality loss is the first moment and second moment of cosine similarity of weight vectors. We performed the empirical studies through simulating the long-tail datasets to show the generalization ability of the proposed approach on long-tail distribution datasets. Further, extensive experiments on large-scale face recognition benchmarks including the Labeled Face in the Wild (LFW), the IARPA Janus Benchmark A (IJB-A), IJB-B, IJB-C, MegaFace Challenge 1 (MF1) and MS-Celeb-1M Low-shot Learning demonstrated that Orthogonality loss outperforms strong baselines, which showcases the extensive suitability and effectiveness of Orthogonality loss.

Hybrid Siamese Network for Unconstrained Face Verification and Clustering under Limited Resources

In this paper, we propose an unconstrained face verification approach that is dependent on Hybrid Siamese architecture under limited resources. The general face verification trend suggests that larger training datasets and/or complex architectures lead to higher accuracy. The proposed approach tends to achieve high accuracy while using a small dataset and a simple architecture by directly learn face’s similarity/dissimilarity from raw face pixels, which is critical for various applications. The proposed architecture has two branches; the first part of these branches is trained independently, while the other parts shared their parameters. A multi-batch algorithm is utilized for training. The training process takes a few hours on a single GPU. The proposed approach achieves near-human accuracy (98.9%) on the Labeled Faces in the Wild (LFW) benchmark, which is competitive with other techniques that are presented in the literature. It reaches 99.1% on the Arabian faces dataset. Moreover, features learned by the proposed architecture are used in building a face clustering system that is based on an updated version of the Density-Based Spatial Clustering of Applications with Noise (DBSCAN). To handle the cluster quality challenge, a novel post-clustering optimization procedure is proposed. It outperforms popular clustering approaches, like K-Means and spectral by 0.098 and up to 0.344 according to F1-measure.

A hybrid deep transfer learning model with machine learning methods for face mask detection in the era of the COVID-19 pandemic

The coronavirus COVID-19 pandemic is causing a global health crisis. One of the effective protection methods is wearing a face mask in public areas according to the World Health Organization (WHO). In this paper, a hybrid model using deep and classical machine learning for face mask detection will be presented. The proposed model consists of two components. The first component is designed for feature extraction using Resnet50. While the second component is designed for the classification process of face masks using decision trees, Support Vector Machine (SVM), and ensemble algorithm. Three face masked datasets have been selected for investigation. The Three datasets are the Real-World Masked Face Dataset (RMFD), the Simulated Masked Face Dataset (SMFD), and the Labeled Faces in the Wild (LFW). The SVM classifier achieved 99.64% testing accuracy in RMFD. In SMFD, it achieved 99.49%, while in LFW, it achieved 100% testing accuracy.

Analysis of Face Recognition Algorithm: Dlib and OpenCV

In face recognition there are two commonly used open-source libraries namely Dlib and OpenCV. Analysis of facial recognition algorithms is needed as reference for software developers who want to implement facial recognition features into an application program. From Dlib algorithm to be analyzed is CNN and HoG, from OpenCV algorithm is DNN and HAAR Cascades. These four algorithms are analyzed in terms of speed and accuracy. The same image dataset will be used to test, along with some actual images to get a more general analysis of how algorithm will appear in real life scenarios. The programming language used for face recognition algorithms is Python. The image dataset will come from LFW (Labeled Faces in the Wild), and AT&T, both of which are available and ready to be downloaded from the internet. Pictures of people around the UIB (Batam International University) is used for actual images dataset. HoG algorithm is fastest in speed test (0.011 seconds / image), but the accuracy rate is lower (FRR = 27.27%, FAR = 0%). DNN algorithm is the highest in level of accuracy (FRR = 11.69%, FAR = 2.6%) but the lowest speed (0.119 seconds / picture). There is no best algorithm, each algorithm has advantages and disadvantages.Keywords: Python, Face Recognition, Analysis, Speed, Accuracy.

Multi-Pose Face Recognition Based on Deep Learning in Unconstrained Scene

At present, deep learning drives the rapid development of face recognition. However, in the unconstrained scenario, the change of facial posture has a great impact on face recognition. Moreover, the current model still has some shortcomings in accuracy and robustness. The existing research has formulated two methods to solve the above problems. One method is to model and train each pose separately. Then, a fusion decision will be made. The other method is to make “frontal” faces on the image or feature level and transform them into “frontal” face recognition. Based on the second idea, we propose a profile to the frontal revise mapping (PTFRM) module. This module realizes the revision of arbitrary poses on the feature level and transforms the multi-pose features into an approximate frontal representation to enhance the recognition ability of the existing recognition models. Finally, we evaluate the PTFRM on unconstrained face validation benchmark datasets such as Labeled Faces in the Wild (LFW), Celebrities in Frontal Profile (CFP), and IARPA Janus Benchmark A(IJB-A). Results show that the chosen method for this study achieves good performance.

Images as graphical password: verification and analysis using non-regular low-density parity check coding

The use of mobile computing devices like laptops, smartphones, tablets, etc. ease user access to work or connect remotely for performing a number of operations. Also, with the advent of these new technologies and devices, one of the major concerns emerges in the form of security for one’s personal data or information. Passwords are very critical to human safety as he or she could be a victim of unauthorized access, data misuse or even they may face some legal trouble because of unconventional activities. Robust passwords are recommended in this act to avoid various outbreaks. But, these passwords are not easy to remember in most of the cases. Therefore, an effort has been made in this paper to present a new graphical password-based method that verifies a user’s authenticity by using images as a password. In this regard, a two-way password verification system is adopted using non-regular low-density parity-check codes and an encryption methodology. The performance of the proposed work was evaluated for three different cases i.e. (a) To validate image for about thirteen of modifications (b) To validate image for different format conversion, and (c) Similarity measures for different face images taken from labeled faces in the wild (LFW) database. Results validated the performance of the proposed method when compared with other state-of-the-art approaches.

A face recognition algorithm based on feature fusion

SummaryIn the process of building a smart city, face recognition can be applied to the transformation of enterprises, communities, and parks. The combination of building security system and face recognition technology can improve the security experience of enterprises and citizens through the solution of hardware and software integration. Face recognition is still facing the challenges of illumination, occlusion, and attitude change in the actual application process. In addition, the end‐to‐end convolutional neural networks (CNN) seldom make use of the hierarchical feature of the network. So, we propose a hierarchy feature fusion method for face recognition, which uses supervisory information to learn shallow and deep facial features. The features are fused to enhance the recognition accuracy of face recognition against illumination and occlusion. The method is applied to the transformation of the visual geometry group network and Lightened CNN. The face recognition experiments are carried out using the hierarchy network. Our method has achieved good recognition results in the labeled faces in the wild (LFW) and AR face databases.

End-to-End Spatial Transform Face Detection and Recognition

BackgroundSeveral face detection and recognition methods have been proposed in the past decades that have excellent performance. The conventional face recognition pipeline comprises the following: (1) face detection, (2) face alignment, (3) feature extraction, and (4) similarity, which are independent of each other. The separate facial analysis stages lead to redundant model calculations, and are difficult for use in end-to-end training. MethodsIn this paper, we propose a novel end-to-end trainable convolutional network framework for face detection and recognition, in which a geometric transformation matrix is directly learned to align the faces rather than predicting the facial landmarks. In the training stage, our single CNN model is supervised only by face bounding boxes and personal identities, which are publicly available from WIDER FACE and CASIA-WebFace datasets. Our model is tested on Face Detection Dataset and Benchmark (FDDB) and Labeled Face in the Wild (LFW) datasets. ResultsThe results show 89.24% recall for face detection tasks and 98.63% accuracy for face recognition tasks.

An end-to-end face recognition method with alignment learning

Many effective methods have been proposed for face recognition in the past decade and the face recognition accuracy is also gradually improved, but these algorithms usually need to perform face alignment process based on the prior knowledge of facial structure before extracting facial features. The face recognition system usually consists of face detection, face alignment, facial feature extraction, etc., which are independent of each other, and it is difficult to design and train the end-to-end face recognition model. In this paper, an end-to-end face recognition method based on spatial transformation layer is proposed. Specifically, the spatial transformation layer is placed in front of the feature extraction layer of the face recognition network, and the face region is aligned by alignment learning which requires neither prior knowledge nor artificially defined geometric transformation. The face identity category information allows the convolutional neural network to automatically learn the most appropriate face alignment. Simulation experiments on CASIA-WebFace, LFW (Labeled Face in the Wild) and YTF (Youtube Face) face database have shown that the suggested alignment learning algorithm in this paper can realize the end-to-end face recognition and can effectively improve the face recognition rate as well.

Improved Single Sample Per Person Face Recognition via Enriching Intra-Variation and Invariant Features

Face recognition using a single sample per person is a challenging problem in computer vision. In this scenario, due to the lack of training samples, it is difficult to distinguish between inter-class variations caused by identity and intra-class variations caused by external factors such as illumination, pose, etc. To address this problem, we propose a scheme to improve the recognition rate by both generating additional samples to enrich the intra-variation and eliminating external factors to extract invariant features. Firstly, a 3D face modeling module is proposed to recover the intrinsic properties of the input image, i.e., 3D face shape and albedo. To obtain the complete albedo, we come up with an end-to-end network to estimate the full albedo UV map from incomplete textures. The obtained albedo UV map not only eliminates the influence of the illumination, pose, and expression, but also retains the identity information. With the help of the recovered intrinsic properties, we then generate images under various illuminations, expressions, and poses. Finally, the albedo and the generated images are used to assist single sample per person face recognition. The experimental results on Face Recognition Technology (FERET), Labeled Faces in the Wild (LFW), Celebrities in Frontal-Profile (CFP) and other face databases demonstrate the effectiveness of the proposed method.

Similar Face Recognition Using the IE-CNN Model

In the field of face recognition, similar face recognition is difficult due to the high degree of similarity of the face structure. The following two factors are needed to make progress in this field: (i) the availability of large scale similar face training datasets, and (ii) a fine-grained face recognition method. With the above factors fulfilled, we make two contributions. First, we show how a large scale similar face dataset (SFD) can be assembled by a combination of automation and human in the loop, and divide the dataset into five grades according to different degrees of similarity. Second, a new fine-grained face feature extraction method is proposed to solve this problem using the attention mechanism which combines the Internal Features and External Features. The Labeled Faces in the Wild (LFW) database, CASIA-WebFace and similar face dataset (SFD) were selected for experiments. It turns out that the true positive rate is improved by 1.94 - 5.66% and the recognition accuracy rate improved by 2.08 - 5.8% for the LFW and CASIA-WebFace database, respectively. Meanwhile for SFD, the recognition accuracy rate improved by 18.80 - 35.84%.

Feature-Improving Generative Adversarial Network for Face Frontalization

Face frontalization can boost the performance of face recognition methods and has made significant progress with the development of Generative Adversarial Networks (GANs). However, many GAN-based face frontalization methods still perform relatively weak on face recognition tasks under large face poses. In this paper, we propose Feature-Improving GAN (FI-GAN) for face frontalization, which aims to improve the recognition performance under large face poses. We assume that there is an inherent mapping between the frontal face and profile face, and their discrepancy in deep representation space can be estimated. The generation module of FI-GAN has a compact module named Feature-Mapping Block that helps to map the features of profile face images to the frontal space. Moreover, we produce a feature discriminator that can distinguish the features of profile face images from those of ground true frontal face images, which guide the generation module to provide high-quality features of profile faces. We conduct experiments on the MultiPIE, Labeled Faces in the Wild (LFW), and Celebrities in Frontal-Profile (CFP) databases. Our method is comparable to state-of-the-art methods under small poses and outperforms them on large pose face recognition.

A Novel Neural Network Method for Face Recognition With a Single Sample Per Person

Face Recognition (FR) problem is one of the significant fields in computer vision. FR is used to identify the faces that appear over distributed cameras over the network. The problem of face recognition can be divided into two categories, the first is recognition with more than one sample per person, which can be called traditional face recognition problem. The second is the recognition of faces using only a Single Sample Per Person (SSPP). The efficiency of face recognition systems decreases because of limited references especially (SSPP) and faces taken in the Operational Domain (OD) different from faces in the Enrollment Domain (ED) in illumination, pose, low-resolution, and blurriness. This paper proposed a method that deals with all problems related to face recognition with SSPP. 3D face reconstruction is used to increase the reference gallery set with different poses and generate a design domain dictionary to overcome the problem of limited reference. Besides, the design domain dictionary is used to feed different deep learning models. Face illumination transfer techniques are utilized to overcome the illumination problem. Labeled Faces in the Wild (LFW) dataset is used to train Super-Resolution Generative Adversarial Network (SRGAN) to overcome the low-resolution problem. Deblur Generative Adversarial Network (DeblurGAN) is trained on the LFW dataset to overcome the problem of blurriness. The proposed method is evaluated using the Chokepoint dataset and COX-S2V dataset. The final results confirm an overall enhancement in accuracy compared to techniques that use SSPP for face recognition (generic learning and face synthesizing approaches). Also, the proposed method outperforms of Traditional and Deep Learning (TDL) method accuracy, which uses SSPP for face recognition.

Rolling in the Deep Convolutional Neural Networks

Over the past years, convolutional neural networks (CNNs) have achieved remarkable success in deep learning. The performance of CNN-based models has caused major advances in a wide range of tasks from computer vision to natural language processing. However, the exposition of the theoretical calculations behind the convolution operation is rarely emphasized. This study aims to provide better understanding the convolution operation entirely by means of diving into the theory of how backpropagation algorithm works for CNNs. In order to explain the training of CNNs clearly, the convolution operation on images is explained in detail and backpropagation in CNNs is highlighted. Besides, Labeled Faces in the Wild (LFW) dataset which is frequently used in face recognition applications is used to visualize what CNNs learn. The intermediate activations of a CNN trained on the LFW dataset are visualized to gain an insight about how CNNs perceive the world. Thus, the feature maps are interpreted visually as well, alongside the training process.

Additive Parameter for Deep Face Recognition

The performance of feature learning for deep convolutional neural networks (DCNNs) is increasing promptly with significant improvement in numerous applications. Recent studies on loss functions clearly describing that better normalization is helpful for improving the performance of face recognition (FR). Several methods based on different loss functions have been proposed for FR to obtain discriminative features. In this paper, we propose an additive parameter depending on multiplicative angular margin to improve the discriminative power of feature embedding that can be easily implemented. In additive parameter approach, an automatic adjustment of the seedling element as the result of angular marginal seed is offered in a particular way for the angular softmax to learn angularly discriminative features. We train the model on publically available dataset CASIA-WebFace, and our experiments on famous benchmarks YouTube Faces (YTF) and labeled face in the wild (LFW) achieve better performance than the various state-of-the-art approaches.

Attention-based convolutional neural network for deep face recognition

Discriminative feature embedding is of essential importance in the field of large scale face recognition. In this paper, we propose an attention-based convolutional neural network (ACNN) for discriminative face feature embedding, which aims to decrease the information redundancy among channels and focus on the most informative components of spatial feature maps. More specifically, the proposed attention module consists of a channel attention block and a spatial attention block which adaptively aggregate the feature maps in both channel and spatial domains to learn the inter-channel relationship matrix and the inter-spatial relationship matrix, then matrix multiplications are conducted for a refined and robust face feature. With the attention module we proposed, we can make standard convolutional neural networks (CNNs), such as ResNet-50, ResNet-101 have more discriminative power for deep face recognition. The experiments on Labelled Faces in the Wild (LFW), Age Database (AgeDB), Celebrities in Frontal Profile (CFP) and MegaFace Challenge 1 (MF1) show that our proposed ACNN architecture consistently outperforms naive CNNs and achieves the state-of-the-art performance.

Learning face similarities for face verification using hybrid convolutional neural networks

<span>Face verification focuses on the task of determining whether two face images belong to the same identity or not. For unrestricted faces in the wild, this is a very challenging task. Besides significant degradation due to images that have large variations in pose, illumination, expression, aging, and occlusions, it also suffers from large-scale ever-expanding data needed to perform one-to-many recognition task. In this paper, we propose a face verification method by learning face similarities using a Convolutional Neural Networks (ConvNet). Instead of extracting features from each face image separately, our ConvNet model jointly extracts relational visual features from two face images in comparison. We train four hybrid ConvNet models to learn how to distinguish similarities between the face pair of four different face portions and join them at top-layer classifier level. We use binary-class classifier at top-layer level to identify the similarity of face pairs which includes a conventional Multi-Layer Perceptron (MLP), Support Vector Machines (SVM), Native Bayes, and another ConvNet. There are 3 face pairing configurations discussed in this paper. Results from experiments using Labeled face in the Wild (LFW) and CelebA datasets indicate that our hybrid ConvNet increases the face verification accuracy by as much as 27% when compared to individual ConvNet approach. We also found that Lateral face pair configuration yields the best LFW test accuracy on a very strict test protocol without any face alignment using MLP as top-layer classifier at 87.89%, which on-par with the state-of-the-arts. We showed that our approach is more flexible in terms of inferencing the learned models on out-of-sample data by testing LFW and CelebA on either model.</span>

Zero-Shot Deep Learning for Media Mining: Person Spotting and Face Clustering in Video Big Data

The analysis of frame sequences in talk show videos, which is necessary for media mining and television production, requires significant manual efforts and is a very time-consuming process. Given the vast amount of unlabeled face frames from talk show videos, we address and propose a solution to the problem of recognizing and clustering faces. In this paper, we propose a TV media mining system that is based on a deep convolutional neural network approach, which has been trained with a triplet loss minimization method. The main function of the proposed system is the indexing and clustering of video data for achieving an effective media production analysis of individuals in talk show videos and rapidly identifying a specific individual in video data in real-time processing. Our system uses several face datasets from Labeled Faces in the Wild (LFW), which is a collection of unlabeled web face images, as well as YouTube Faces and talk show faces datasets. In the recognition (person spotting) task, our system achieves an F-measure of 0.996 for the collection of unlabeled web face images dataset and an F-measure of 0.972 for the talk show faces dataset. In the clustering task, our system achieves an F-measure of 0.764 and 0.935 for the YouTube Faces database and the LFW dataset, respectively, while achieving an F-measure of 0.832 for the talk show faces dataset, an improvement of 5.4%, 6.5%, and 8.2% over the previous methods.

Eye localization based on weight binarization cascade convolution neural network

Eye localization is a key step in the field of face recognition and analysis, which is the premise and breakthrough of drowsiness estimation and auxiliary driving. An eye localization method based on Weight Binarization Cascade Convolution Neural Network (WBCCNN) is proposed, in which the WBCCNN is composed of four levels and the weight is constrained by binarization. It predicts eye positions from coarse-to-fine to improve the performance of eye localization, and binary network not only helps to reduce the storage size of the model, but also speeds up the operation. Experiments on eye localization are performed using Labeled Faces in the Wild (LFW), BioID, and Labeled Face Parts in the Wild (LFPW) Databases, from which the results show that the average detection errors of left eye and right eye by our method are 0.66% and 0.71% on LFW, respectively. The operation speed of binary network is approximately as twice as that of non-binary. In addition, our method requires less storage capacity, which maintains higher performance on BioID and LFPW, compared to some state-of-the-art works.

Deep Convolutional Neural Network-Based Approaches for Face Recognition

Face recognition (FR) is defined as the process through which people are identified using facial images. This technology is applied broadly in biometrics, security information, accessing controlled areas, keeping of the law by different enforcement bodies, smart cards, and surveillance technology. The facial recognition system is built using two steps. The first step is a process through which the facial features are picked up or extracted, and the second step is pattern classification. Deep learning, specifically the convolutional neural network (CNN), has recently made commendable progress in FR technology. This paper investigates the performance of the pre-trained CNN with multi-class support vector machine (SVM) classifier and the performance of transfer learning using the AlexNet model to perform classification. The study considers CNN architecture, which has so far recorded the best outcome in the ImageNet Large-Scale Visual Recognition Challenge (ILSVRC) in the past years, more specifically, AlexNet and ResNet-50. In order to determine performance optimization of the CNN algorithm, recognition accuracy was used as a determinant. Improved classification rates were seen in the comprehensive experiments that were completed on the various datasets of ORL, GTAV face, Georgia Tech face, labelled faces in the wild (LFW), frontalized labeled faces in the wild (F_LFW), YouTube face, and FEI faces. The result showed that our model achieved a higher accuracy compared to most of the state-of-the-art models. An accuracy range of 94% to 100% for models with all databases was obtained. Also, this was obtained with an improvement in recognition accuracy up to 39%.