Labeled Faces In The Wild Research Articles

Fine-grained face verification: FGLFW database, baselines, and human-DCMN partnership

As performance on some aspects of the Labeled Faces in the Wild (LFW) benchmark approaches 100% accuracy, there is an intense debate on whether unconstrained face verification problem has already been solved. In this paper, we study a new face verification problem that assumes the imposter would deliberately seek a people with similarly-looking face to invade the biometric system. To simulate this deliberate imposture attack, we first construct a Fine-Grained LFW (FGLFW) database, which deliberately selects 3000 similarly-looking face pairs within original image folders by human crowdsourcing to replace the negative pairs of LFW. Our controlled human survey reports 99.85% accuracy on LFW, but only 92.03% accuracy on FGLFW. As the algorithm baselines, we evaluate several state-of-the-art metric learning, face descriptors, and deep learning methods on the new FGLFW database, and their accuracy drops about 10–20% compared to the corresponding LFW performance. To address this challenge, we develop a Deep Convolutional Maxout Network (DCMN) which aim to tolerate the multi-modal intra-personal variations and distinguish fine-grained localized inter-personal facial details. The experimental results suggest that the proposed DCMN method significantly outperforms current techniques such as Deepface, DeepID2, and VGG-Face. Fusion of the scores of our proposed DCMN to that of human operators notably boost the verification accuracy from 92–96%, suggesting that human-algorithm partnerships are promising to detect the similarly-looking deliberate impostors.

Face Occlusion Detection Using Deep Convolutional Neural Networks

With the rise of crimes associated with Automated Teller Machines (ATMs), security reinforcement by surveillance techniques has been a hot topic on the security agenda. As a result, cameras are frequently installed with ATMs, so as to capture the facial images of users. The main objective is to support follow-up criminal investigations in the event of an incident. However, in the case of miss-use, the user’s face is often occluded. Therefore, face occlusion detection has become very important to prevent crimes connected with ATM usage. Traditional approaches to solving the problem typically comprise a succession of steps: localization, segmentation, feature extraction and recognition. This paper proposes an end-to-end facial occlusion detection framework, which is robust and effective by combining region proposal algorithm and Convolutional Neural Networks (CNN). The framework utilizes a coarse-to-fine strategy, which consists of two CNNs. The first CNN detects the head element within an upper body image while the second distinguishes which facial part is occluded from the head image. In comparison with previous approaches, the usage of CNN is optimal from a system point of view as the design is based on the end-to-end principle and the model operates directly on image pixels. For evaluation purposes, a face occlusion database consisting of over 50[Formula: see text]000 images, with annotated facial parts, was used. Experimental results revealed that the proposed framework is very effective. Using the bespoke face occlusion dataset, Aleix and Robert (AR) face dataset and the Labeled Face in the Wild (LFW) database, we achieved over 85.61%, 97.58% and 100% accuracies for head detection when the Intersection over Union-section (IoU) is larger than 0.5, and 94.55%, 98.58% and 95.41% accuracies for occlusion discrimination, respectively.

Ensemble of texture descriptors and classifiers for face recognition

Presented in this paper is a novel system for face recognition that works well in the wild and that is based on ensembles of descriptors that utilize different preprocessing techniques. The power of our proposed approach is demonstrated on two datasets: the FERET dataset and the Labeled Faces in the Wild (LFW) dataset. In the FERET datasets, where the aim is identification, we use the angle distance. In the LFW dataset, where the aim is to verify a given match, we use the Support Vector Machine and Similarity Metric Learning. Our proposed system performs well on both datasets, obtaining, to the best of our knowledge, one of the highest performance rates published in the literature on the FERET datasets. Particularly noteworthy is the fact that these good results on both datasets are obtained without using additional training patterns. The MATLAB source of our best ensemble approach will be freely available at https://www.dei.unipd.it/node/2357.

Face Recognition Performance Improvement Using Derivative of Accumulated Absolute Difference Based on Probabilistic Histogram

In the paper we propose a face verifying algorithm for face recognition that can identify two face mismatch pairs in cases of incorrect decisions. The computational approach taken in this system is performed by the derivative of accumulated absolute difference between two faces unseen before. Unlike the traditional multi-dimensional distance measurement, the proposed algorithm also considers an increasing trend of accumulated absolute difference in respect to the Gaussian components. A Gaussian mixture model of bag-of-feature from training faces is also widely applicable to several biometric systems. Evaluation of the proposed algorithm is done on unconstrained environments using Labeled Face in the Wild (LFW) datasets. Experiments show that the proposed algorithm outperforms all conventional face recognition algorithms with advantage of about 4.92% over direct-bag-of-features and 18.05% over principal component analysis-based and is also appropriate for identification task of the face recognition systems. Furthermore, some particular advantages of our approach are that it can be applied to other verification systems.

Face Retrieval Based On Local Binary Pattern and Its Variants: A Comprehensive Study

Face retrieval (FR) is one of the specific fields in content-based image retrieval (CBIR). Its aim is to search relevant faces in large database based on the contents of the images rather than the metadata. It has many applications in important areas such as face searching, forensics, and identification… In this paper, we experimentally evaluate Face Retrieval based on Local Binary Pattern (LBP) and its variants: Rotation Invariant Local Binary Pattern (RILBP) and Pyramid of Local Binary Pattern (PLBP). We also use a grid LBP based operator, which divides an image into 6×7 sub-regions then concentrates LBP feature vector from each of them into a spatially enhanced feature histogram. These features were firstly tested on three fontal face datasets: The Database of Faces (TDF), Caltech Faces 1999 (CF1999) and the combination of The Database of faces and Caltech Faces 1999 (CF). Good result on these dataset has encouraged us to conduct tests on Labeled Faces in the Wild (LFW), where the images were taken from real-world condition. Mean average precision (MAP) was used for measuring the performance of the system. We carry out the experiments in two main stages indexing and searching with the use of k-fold cross-validation. We further boost the system by using Locality Sensitive Hashing (LSH). Furthermore, we also evaluate the impact of LSH on the searching stage. The experimental results have shown that LSH is effective for face searching as well as LBP is robust feature in fontal face retrieval

Multialgorithmic Frameworks for Human Face Recognition

This paper presents a critical evaluation of multialgorithmic face recognition systems for human authentication in unconstrained environment. We propose different frameworks of multialgorithmic face recognition system combining holistic and texture methods. Our aim is to combine the uncorrelated methods of the face recognition that supplement each other and to produce a comprehensive representation of the biometric cue to achieve optimum recognition performance. The multialgorithmic frameworks are designed to combine different face recognition methods such as (i) Eigenfaces and local binary pattern (LBP), (ii) Fisherfaces and LBP, (iii) Eigenfaces and augmented local binary pattern (A-LBP), and (iv) Fisherfaces and A-LBP. The matching scores of these multialgorithmic frameworks are processed using different normalization techniques whereas their performance is evaluated using different fusion strategies. The robustness of proposed multialgorithmic frameworks of face recognition system is tested on publicly available databases, for example, AT & T (ORL) and Labeled Faces in the Wild (LFW). The experimental results show a significant improvement in recognition accuracies of the proposed frameworks of face recognition system in comparison to their individual methods. In particular, the performance of the multialgorithmic frameworks combining face recognition methods with the devised face recognition method such as A-LBP improves significantly.

Minimalistic CNN-based ensemble model for gender prediction from face images

Despite being extensively studied in the literature, the problem of gender recognition from face images remains difficult when dealing with unconstrained images in a cross-dataset protocol. In this work, we propose a convolutional neural network ensemble model to improve the state-of-the-art accuracy of gender recognition from face images on one of the most challenging face image datasets today, LFW (Labeled Faces in the Wild). We find that convolutional neural networks need significantly less training data to obtain the state-of-the-art performance than previously proposed methods. Furthermore, our ensemble model is deliberately designed in a way that both its memory requirements and running time are minimized. This allows us to envision a potential usage of the constructed model in embedded devices or in a cloud platform for an intensive use on massive image databases.

A STATE OF THE ART COMPARISON OF DATABASES FOR FACIAL OCCLUSION

Face recognition continues to be one of the most popular research areas of image processing and computer vision. There are various face databases available to researchers for face detection and recognition. These databases are customized for a particular need of one algorithm. They are range in size, scope, and purpose. Few of these databases from the literature contain face occlusions in several positions of the faces to enable real world applications. In this paper, we present four different occlusion face databases. These are Aleix-Robert (AR), Bosphorus, Labeled Faces in the Wild (LFW), and University of Milano Bicocca Database (UMB) face databases. At each section, the key features of the database are presented with the recording conditions, though not all of them are discussed at the same level of details. Detailed comparisons of the databases were made based on controlled and uncontrolled databases, 2D and 3D databases and also their uniqueness. Comparison was also made with other databases out of the categorization mentioned. The databases are useful for performing a rigorous benchmarking of face detection and recognition algorithms.

Bayesian face recognition using 2D Gaussian-Hermite moments

This paper presents a statistical face recognition algorithm by expressing face images in terms of orthogonal two-dimensional Gaussian-Hermite moments (2D-GHMs). Motivation for developing 2D-GHM-based recognition algorithm includes the ability of these moments to capture higher-order hidden nonlinear 2D structures within images and the invariance of certain linear combinations of moments to common geometric distortions in images. The key contribution of this paper is that features of 2D faces are represented in terms of a statistically selected set of 2D discriminative GHMs (DGHMs) as opposed to commonly chosen heuristic set of first few order moments only. In particular, the intraclass correlation coefficient for the entire set of moments of the training images are used to select only a desired set of moments that maximize the discrimination among available classes. The naive Bayes classifier that yields optimal performance in many statistical applications is used for identification due to the simplicity of its implementation for handling huge size face database. Experiments are conducted to evaluate the performance of the proposed recognition algorithm on exhaustive databases such as the AT&T, Face Recognition Grand Challenge (FRGC), Face Recognition Technology (FERET), Labeled Faces in the Wild (LFW), and YouTube that possess face images or videos with significant variations in terms of appearance, occlusion, expression, pose, resolution, and illumination both in the constrained and unconstrained environments. In the constrained condition, comparisons with the well-established 2D-principal component analysis, 2D-linear discriminant analysis, and 2D-canonical correlation analysis methods as well as orthogonal 2D-Krawtchouk moment-based method reveal the superior performance of the proposed method in terms of recognition accuracy for varying numbers of training and probe images. The proposed DGHM features also show superior recognition or verification performance on the standard protocols of the unconstrained face databases when comparing with the commonly referred descriptors such as the local binary pattern or scale-invariant feature transform.

PCANet: A Simple Deep Learning Baseline for Image Classification?

In this paper, we propose a very simple deep learning network for image classification that is based on very basic data processing components: 1) cascaded principal component analysis (PCA); 2) binary hashing; and 3) blockwise histograms. In the proposed architecture, the PCA is employed to learn multistage filter banks. This is followed by simple binary hashing and block histograms for indexing and pooling. This architecture is thus called the PCA network (PCANet) and can be extremely easily and efficiently designed and learned. For comparison and to provide a better understanding, we also introduce and study two simple variations of PCANet: 1) RandNet and 2) LDANet. They share the same topology as PCANet, but their cascaded filters are either randomly selected or learned from linear discriminant analysis. We have extensively tested these basic networks on many benchmark visual data sets for different tasks, including Labeled Faces in the Wild (LFW) for face verification; the MultiPIE, Extended Yale B, AR, Facial Recognition Technology (FERET) data sets for face recognition; and MNIST for hand-written digit recognition. Surprisingly, for all tasks, such a seemingly naive PCANet model is on par with the state-of-the-art features either prefixed, highly hand-crafted, or carefully learned [by deep neural networks (DNNs)]. Even more surprisingly, the model sets new records for many classification tasks on the Extended Yale B, AR, and FERET data sets and on MNIST variations. Additional experiments on other public data sets also demonstrate the potential of PCANet to serve as a simple but highly competitive baseline for texture classification and object recognition.

Learning a non‐linear combination of Mahalanobis distances using statistical inference for similarity measure

In this study, the authors learn a similarity measure that discriminates between inter‐class and intra‐class samples based on a statistical inference perspective. A non‐linear combination of Mahalanobis is proposed to reflect the properties of a likelihood ratio test. Since an object's appearance is influenced by the identity of the object and variations in the capturing process, the authors represent the feature vector, which is the difference between two samples in the differences space, as a sample that is drawn from a mixture of many distributions. This mixture consists of the identities distribution and other distributions of the variations in the capturing process, in case of dissimilar samples. However, in the case of similar samples, the mixture consists of the variations in the capturing process distributions only. Using this representation, the proposed similarity measure accurately discriminates between inter‐class and intra‐class samples. To highlight the good performance of the proposed similarity measure, it is tested on different computer vision applications: face verification and person re‐identification. To illustrate how the proposed learning method can easily be used on large scale datasets, experiments are conducted on different challenging datasets: labelled faces in the wild (LFW), public figures face database, ETHZ and VIPeR. Moreover, in these experiments, the authors evaluate different stages, for example, features detector, descriptor type and descriptor dimension, which constitute the face verification pipeline. The experimental results confirm that the learning method outperforms the state‐of‐the‐art.

A Gender Recognition Approach with an Embedded Preprocessing

Gender recognition from facial images has become an empirical aspect in present world. It is one of the main problems of computer vision and researches have been conducting on it. Though several techniques have been proposed, most of the techniques focused on facial images in controlled situation. But the problem arises when the classification is performed in uncontrolled conditions like high rate of noise, lack of illumination, etc. To overcome these problems, we propose a new gender recognition framework which first preprocess and enhances the input images using Adaptive Gama Correction with Weighting Distribution. We used Labeled Faces in the Wild (LFW) database for our experimental purpose which contains real life images of uncontrolled condition. For measuring the performance of our proposed method, we have used confusion matrix, precision, recall, F-measure, True Positive Rate (TPR), and False Positive Rate (FPR). In every case, our proposed framework performs superior over other existing state-of-the-art techniques.

Real-World and Rapid Face Recognition Toward Pose and Expression Variations via Feature Library Matrix

In this paper, a novel method for face recognition under pose and expression variations is proposed from only a single image in the gallery. A 3D probabilistic facial expression recognition generic elastic model is proposed to reconstruct a 3D model from real-world human face using only a single 2D frontal image with/without facial expressions. Then, a feature library matrix (FLM) is generated for each subject in the gallery from all face poses by rotating the 3D reconstructed models and extracting features in the rotated face pose. Therefore, each FLM is subsequently rendered for each subject in the gallery based on triplet angles of face poses. In addition, before matching the FLM, an initial estimate of triplet angles is obtained from the face pose in probe images using an automatic head pose estimation approach. Then, an array of the FLM is selected for each subject based on the estimated triplet angles. Finally, the selected arrays from FLMs are compared with extracted features from the probe image by iterative scoring classification using the support vector machine. Convincing results are acquired to handle pose and expression changes on the Bosphorus, Face Recognition Technology (FERET), Carnegie Mellon University-Pose, Illumination, and Expression (CMU-PIE), and Labeled Faces in the Wild (LFW) face databases compared with several state-of-the-art methods in pose-invariant face recognition. The proposed method not only demonstrates an excellent performance by obtaining high accuracy on all four databases but also outperforms other approaches realistically.

Similarity metric learning for face verification using sigmoid decision function

In this paper, we consider the face verification problem, which is to determine whether two face images belong to the same subject or not. Although many research efforts have been focused on this problem, it still remains a challenging problem due to large intra-personal variations in imaging conditions, such as illumination, pose, expression, and occlusion. Our proposed method is based on the idea that we would like the similarity between positive pairs larger than negative pairs, and obtain a similarity estimation of two images. We construct our decision function by incorporating bilinear similarity and Mahalanobis distance to the sigmoid function. The constructed decision function makes our method discriminative for inter-personal differences and invariant to intra-personal variations such as pose/lighting/expression. What is more, our formulated objective function is convex, which guarantees global minimum. Our method belongs to nonlinear metric which is more robust to handle heterogeneous data than linear metric. We evaluate our proposed verification method on the challenging labeled faces in the wild (LFW) database. Experimental results demonstrate that the proposed method outperforms state-of-the-art methods such as Joint Bayesian under the unrestricted setting of LFW.

SOFT BIOMETRICS: GENDER RECOGNITION FROM UNCONSTRAINED FACE IMAGES USING LOCAL FEATURE DESCRIPTOR

Gender recognition from unconstrained face images is a challenging task due to the high degree of misalignment, pose, expression, and illumination variation. In previous works, the recognition of gender from unconstrained face images is approached by utilizing image alignment, exploiting multiple samples per individual to improve the learning ability of the classifier, or learning gender based on prior knowledge about pose and demographic distributions of the dataset. However, image alignment increases the complexity and time of computation, while the use of multiple samples or having prior knowledge about data distribution is unrealistic in practical applications. This paper presents an approach for gender recognition from unconstrained face images. Our technique exploits the robustness of local feature descriptor to photometric variations to extract the shape description of the 2D face image using a single sample image per individual. The results obtained from experiments on Labeled Faces in the Wild (LFW) dataset describe the effectiveness of the proposed method. The essence of this study is to investigate the most suitable functions and parameter settings for recognizing gender from unconstrained face images.

Surpassing Human-Level Face Verification Performance on LFW with GaussianFace

Face verification remains a challenging problem in very complex conditions with large variations such as pose, illumination, expression, and occlusions. This problemis exacerbated when we rely unrealistically on a singletraining data source, which is often insufficient to coverthe intrinsically complex face variations. This paperproposes a principled multi-task learning approachbased on Discriminative Gaussian Process Latent VariableModel (DGPLVM), named GaussianFace, for faceverification. In contrast to relying unrealistically on asingle training data source, our model exploits additional data from multiple source-domains to improve the generalization performance of face verification inan unknown target-domain. Importantly, our model can adapt automatically to complex data distributions, and therefore can well capture complex face variations inherent in multiple sources. To enhance discriminative power, we introduced a more efficient equivalent form of Kernel Fisher Discriminant Analysis to DGPLVM.To speed up the process of inference and prediction, we exploited the low rank approximation method. Extensive experiments demonstrated the effectiveness of the proposed model in learning from diverse data sources and generalizing to unseen domains. Specifically, the accuracy of our algorithm achieved an impressive accuracyrate of 98.52% on the well-known and challenging Labeled Faces in the Wild (LFW) benchmark. For the first time, the human-level performance in face verification (97.53%) on LFW is surpassed.

Learning to classify gender from four million images

The application of learning algorithms to big datasets has been identified for a long time as an effective way to attack important tasks in pattern recognition, but the generation of large annotated datasets has a significant cost. We present a simple and effective method to generate a classifier of face images, by training a linear classification algorithm on a massive dataset entirely assembled and labelled by automated means. In doing so, we perform the largest experiment on face gender recognition so far published, reporting the highest performance yet. Four million images and more than 60,000 features are used to train online classifiers. By using an ensemble of linear classifiers, we achieve an accuracy of 96.86% on the most challenging public database, labelled faces in the wild (LFW), 2.05% higher than the previous best result on the same dataset (Shan, 2012). This result is relevant both for the machine learning community, addressing the role of large datasets, and the computer vision community, providing a way to make high quality face gender classifiers. Furthermore, we propose a general way to generate and exploit massive data without human annotation. Finally, we demonstrate a simple and effective adaptation of the Pegasos that makes it more robust.

SOFT BIOMETRICS: GENDER RECOGNITION FROM UNCONSTRAINED FACE IMAGES USING LOCAL FEATURE DESCRIPTOR

Gender recognition from unconstrained face images is a challenging task due to the high degree of misalignment, pose, expression, and illumination variation. In previous works, the recognition of gender from unconstrained face images is approached by utilizing image alignment, exploiting multiple samples per individual to improve the learning ability of the classifier, or learning gender based on prior knowledge about pose and demographic distributions of the dataset. However, image alignment increases the complexity and time of computation, while the use of multiple samples or having prior knowledge about data distribution is unrealistic in practical applications. This paper presents an approach for gender recognition from unconstrained face images. Our technique exploits the robustness of local feature descriptor to photometric variations to extract the shape description of the 2D face image using a single sample image per individual. The results obtained from experiments on Labeled Faces in the Wild (LFW) dataset describe the effectiveness of the proposed method. The essence of this study is to investigate the most suitable functions and parameter settings for recognizing gender from unconstrained face images.

Identifying Individuals Using Multimodal Face Recognition Techniques

A biometric system which primarily based on the cues of unimodal biometric for individual identification is not always meet the desired results. The concept of multimodal biometrics for human Identification is an emerging trend. In this paper, we present state-of-the-art novel multimodal biometric system, for face recognition, which combines the similarity scores of the unimodal modalities such as appearance based and texture based techniques of face recognition, to cater the decisive results at the level of matching score. Formally, it includes the fusion of unimodal techniques to devise the multimodal models in four possible combinations such as (a) Eigenfaces and local binary pattern (LBP), (b) Fisherfaces and LBP, (c) organics’ and augmented local binary pattern (A-LBP), and (d) Fisherfaces and A-LBP. The performance of the multimodal face recognition systems is tested on the publicly available face databases such as the AT & T-ORL and the Labeled Faces in the Wild (LFW) using a new Bray Curtis dissimilarity metric. The experimental results show a significant improvement in the performance of recognition accuracies of multimodal face recognition techniques.

Reconstruction-Based Metric Learning for Unconstrained Face Verification

In this paper, we propose a reconstruction-based metric learning method to learn a discriminative distance metric for unconstrained face verification. Unlike conventional metric learning methods, which only consider the label information of training samples and ignore the reconstruction residual information in the learning procedure, we apply a reconstruction criterion to learn a discriminative distance metric. For each training example, the distance metric is learned by enforcing a margin between the interclass sparse reconstruction residual and interclass sparse reconstruction residual, so that the reconstruction residual of training samples can be effectively exploited to compute the between-class and within-class variations. To better use multiple features for distance metric learning, we propose a reconstruction-based multimetric learning method to collaboratively learn multiple distance metrics, one for each feature descriptor, to remove uncorrelated information for recognition. We evaluate our proposed methods on the Labelled Faces in the Wild (LFW) and YouTube face data sets and our experimental results clearly show the superiority of our methods over both previous metric learning methods and several state-of-the-art unconstrained face verification methods.