Face Database Research Articles

An LDOP approach for face identification under unconstrained scenarios

ABSTRACT In unconstrained environments, it encounters a number of challenges when considering handcrafted features for face recognition, including changes in pose, illumination and facial expression and plastic surgery variations, look-alike faces and selfie images. As majority of the published works on local descriptors are based on the relationship between the centre pixel and neighbourhood pixels at different radial widths, by including the relationship between different neighbourhood pixels of a target pixel with it can facilitate the decoupling of the discriminatory features further. To exploit such relationships and to mitigate the challenges, the proposed study reports a novel local descriptor called Local Directional Octa Pattern (LDOP) that makes use of derivative operation and three well-known metrics, namely, histogram intersection, K-nearest neighbour and Euclidean distance for face identification. Rigorous experiments have been conducted on six benchmark face databases, namely, Extended Yale Face B, Labeled Faces in the Wild, Plastic Surgery, Look-alike, UMDAA-02-FD and ORL, confirming the superiority of the proposed local descriptor over state-of-the-art descriptors with accuracies of 97.37%, 49.09%, 75.47%, 31.2% and 93.31% at Rank–10 and 100% at Rank-5 determined on the databases, respectively.

Face Recognition System for Real Life Applications using Augmentation Techniques

A facial recognition system may recognise a person by comparing his face to the ones in a database of faces taken from a digital image or video frame. It is primarily used to verify the identity of users by analysing facial features from an image. According to several studies, facial recognition technology has made enormous strides with the growth of science and technology, but there is still opportunity for its improvement in practical use. Some issues like rotation, occlusion, and meta learning approach can be handled for improved model accuracy by using 3D technologies like image augmentation to supplement 2D images. Today's facial recognition technology can be used for more than only identifying people; it can also be used for security purposes in residential, business, and commercial applications. Due to its convenience, face recognition technology is widely employed in the financial sector as well. This research proposes face recognition for real life applications using Siamese network and Deep Neural Network, Facenet.

Local image descriptor developed from Fechner’s law

Human perception of visual stimulus and its physical characteristics have a nonlinear-logarithmic relation as stated in Fechner’s law, which is a psychophysical law of perception (i.e., subjective sensation). Inspired by Fechner’s law, we use the salient differences among pixels in images as the physical characteristics to mimic the human pattern perception and propose a Fechner local image descriptor (FLID) for face image representation. FLID is a nonlinear descriptor, so the represented image is more hierarchical than the general linear one. In addition, a threshold method is used to divide the features into four different intervals, which can effectively reduce the effects of noise and illumination. However, considering that FLID only extracts the local features of single-scale blocks, we incorporate multi-scale block into FLID and propose a multi-scale block Fechner local image descriptor (MB-FLID) to extract more global structural features. In the experimental part, we verified the performance of FLID and MB-FLID on the Face Recognition Technology (FERET), extended Yale-B, Olivetti Research Laboratories, Yale, and AR face databases, and compared them with a host of other local feature descriptors. The experimental results show that the proposed FLID and MB-FLID outperform the compared descriptors.

Facial feature embedded CycleGAN for VIS–NIR translation

Visible and near-infrared (VIS–NIR) heterogeneous face recognition remains a challenging task due to distinctions between spectral components of two modalities and insufficiently pairwise VIS–NIR data. Inspired by the cycle-consistent generative adversarial network (CycleGAN), this paper proposes a facial feature embedded CycleGAN to translate between VIS and NIR face images, aiming to enable the distributions of translated (fake) images to be similar as those of true images. To learn the particular feature of NIR or VIS domain while preserving common facial representation between VIS and NIR domains, a facial feature extractor (FFE), tailored specifically for extracting effective feature from face images, is embedded in the generator of original CycleGAN. For implementing the FFE, we use the MobileFaceNet which is pre-trained on a VIS face database. The domain-invariant feature learning is enhanced by proposing a new pixel consistency loss. Additionally, we establish a new WHU VIS–NIR database including varies in face rotation and expressions to enrich the insufficient training data. Moreover, experiments on the well-known Oulu-CASIA NIR–VIS database and our WHU VIS–NIR database validate the potential benefit of the proposed FFE-based CycleGAN (FFE-CycleGAN). In particular, we achieve 96.5% accuracy on Oulu-CASIA and 98.9% accuracy on WHU VIS–NIR.

FRMDB: Face Recognition Using Multiple Points of View.

Although face recognition technology is currently integrated into industrial applications, it has open challenges, such as verification and identification from arbitrary poses. Specifically, there is a lack of research about face recognition in surveillance videos using, as reference images, mugshots taken from multiple Points of View (POVs) in addition to the frontal picture and the right profile traditionally collected by national police forces. To start filling this gap and tackling the scarcity of databases devoted to the study of this problem, we present the Face Recognition from Mugshots Database (FRMDB). It includes 28 mugshots and 5 surveillance videos taken from different angles for 39 distinct subjects. The FRMDB is intended to analyze the impact of using mugshots taken from multiple points of view on face recognition on the frames of the surveillance videos. To validate the FRMDB and provide a first benchmark on it, we ran accuracy tests using two CNNs, namely VGG16 and ResNet50, pre-trained on the VGGFace and VGGFace2 datasets for the extraction of face image features. We compared the results to those obtained from a dataset from the related literature, the Surveillance Cameras Face Database (SCFace). In addition to showing the features of the proposed database, the results highlight that the subset of mugshots composed of the frontal picture and the right profile scores the lowest accuracy result among those tested. Therefore, additional research is suggested to understand the ideal number of mugshots for face recognition on frames from surveillance videos.

Cancelable biometric schemes for Euclidean metric and Cosine metric

The handy biometric data is a double-edged sword, paving the way of the prosperity of biometric authentication systems but bringing the personal privacy concern. To alleviate the concern, various biometric template protection schemes are proposed to protect the biometric template from information leakage. The preponderance of existing proposals is based on Hamming metric, which ignores the fact that predominantly deployed biometric recognition systems (e.g. face, voice, gait) generate real-valued templates, more applicable to Euclidean metric and Cosine metric. Moreover, since the emergence of similarity-based attacks, those schemes are not secure under a stolen-token setting. In this paper, we propose a succinct biometric template protection scheme to address such a challenge. The proposed scheme is designed for Euclidean metric and Cosine metric instead of Hamming distance. Mainly, the succinct biometric template protection scheme consists of distance-preserving, one-way, and obfuscation modules. To be specific, we adopt location sensitive hash function to realize the distance-preserving and one-way properties simultaneously and use the modulo operation to implement many-to-one mapping. We also thoroughly analyze the proposed scheme in three aspects: irreversibility, unlinkability and revocability. Moreover, comprehensive experiments are conducted on publicly known face databases. All the results show the effectiveness of the proposed scheme.

Discriminant Feature Extraction by Generalized Difference Subspace.

In this paper, we reveal the discriminant capacity of orthogonal data projection onto the generalized difference subspace (GDS), both theoretically and experimentally. In our previous work, we demonstrated that the GDS projection works as a quasi-orthogonalization of class subspaces, which is an effective feature extraction for subspace based classifiers. Here, we further show that GDS projection also works as a discriminant feature extraction through a similar mechanism to the Fisher discriminant analysis (FDA). A direct proof of the connection between GDS projection and FDA is difficult due to the significant difference in their formulations. To circumvent the complication, we first introduce geometrical Fisher discriminant analysis (gFDA) based on a simplified Fisher criterion. It is derived from a heuristic yet practically plausible assumption: the direction of the sample mean vector of a class is largely aligned to the first principal component vector of the class, given that the principal component analysis (PCA) is applied without data centering. gFDA works stably even under few samples, bypassing the small sample size (SSS) problem of FDA. We then prove that gFDA is equivalent to GDS projection with a small correction term. This equivalence ensures GDS projection to inherit the discriminant ability from FDA via gFDA. Furthermore, we discuss two useful extensions of these methods, 1) a nonlinear extension by kernel trick, 2) a combination with CNN features. The equivalence and the effectiveness of the extensions have been verified through extensive experiments on the extended Yale B+, CMU face database, ALOI, ETH80, MNIST, and CIFAR10, mainly focusing on image recognition under small samples.

Face Structure, Beauty, and Race: A Study of Population Databases Using Computer Modeling.

Understanding the differences in facial shapes in individuals from different races is relevant across several fields, from cosmetic and reconstructive medicine to anthropometric studies. To determine whether there are features shared by the faces of an aesthetic female face database and if they correlate to their racial demographics using novel computer modeling. The database was formed using the "top 100 most beautiful women" lists released by "For Him Magazine" for the last 15 years. Principal component analysis (PCA) of 158 parameters was carried out to check for clustering or racial correlation with these clusters. PCA is a machine-learning tool used to reduce the number of variables in a large data set, allowing for easier analysis of the data while retaining as much information as possible from the original data set. A review of the literature on craniofacial anthropometric differences across ethnicities was also undertaken to complement the computer data. Two thousand eight hundred and seventy aesthetic faces formed the database in the same racial proportion as 10,000 faces from the general population as a baseline. PCA clustering illustrated grouping by latent space parameters for facial dimensions but showed no correlation with racial demographics. There was a commonality of facial features within the aesthetic cohort, which differed from the general population. Fourteen papers were included in the review which contained 8142 individuals. Aesthetic female faces have commonalities in facial features regardless of racial demographic, and the dimensions of these features vary from the baseline population. There may even be a common human aesthetic proportion that transcends racial boundaries, but this is yet to be elucidated.

Collaborative representation with hole filling techniques for Kinect RGB-D face recognition

We present in this paper, three different filtering techniques based on kernel function to fill missing information in depth map images obtained from low resolution sensor such as Kinect to improve the performance accuracy of RGB-D face recognition systems. We propose in this study, an RGB-D face recognition scheme that combines depth map and colour image using wavelet average fusion followed by collaborative representation classifier (CRC) for comparison of reference and probe images. We present the evaluation results based on our GU-RGBD face database and IIIT-D face database to present the significance of our three different filters employed in hole filling. Further, our investigation presents an extensive experimental analysis using eight different feature extraction techniques independently across three different filters to demonstrate the potential of our proposed approach. The proposed approach of hole filling, improves the performance accuracy of RGB-D face recognition as compared to without employing filtering operations.

NaijaFaceVoice: A Large-Scale Deep Learning Model and Database of Nigerian Faces and Voices

The fusion of two or more traits in multimodal biometrics generally improves recognition accuracy. The question is, by how much? Large-scale databases are better suited for training deep learning models for better generalization and accuracy. Therefore, a large-scale multimodal database is beneficial. However, publicly available large-scale multimodal databases are scarce, especially for faces and voices. Again, because a face image is 2-D while a voice is 1-D, there is the challenge of the best way to fuse both. Therefore, improvements owing to fusion have hitherto yielded marginal improvements. This study proposes a semi-automated curation algorithm for the extraction of the faces and voices of target individuals in videos to create a large-scale face-voice database. The curation technique involves observing the positions at the time of the occurrence of the target subject’s faces and voices in videos. These positions are supplied to a MATLAB2017b script that detects the faces in the observed regions, crops, resizes, auto-labels, and writes them to the disk. A second MATLAB2017b script, extracts the audio content within the observed regions, auto-labels, and writes the voice segments to the disk. The created database named NaijaFaceVoice consists of 2,656 subjects with over 2 million faces and 195 hours of utterances. The database was employed to develop a large-scale recognition system that leveraged Convolutional Neural Networks. Robust fusion methods incorporating the proposed Spectrogram-Voting concept significantly improved performance achieving a record equal error rate of 0.0003519%, an improvement by a factor of over 450.

Smart Attendance Automation System

The smart attendance marking system, which is based on face recognition technology, is a modern solution that may aid in the automation of the attendance management process in a variety of organizations, including schools, colleges, and corporations. Using facial recognition technology, the research team hopes to create an efficient system for tracking and recording attendance. The system will be developed by gathering and analysing data in order to construct a database of faces that will be used to train deep learning algorithms to recognise faces effectively. Compared to traditional attendance tracking systems, the smart attendance marking system based on face recognition technology has various advantages. For starters, it saves time and lowers manual labour because attendance is automatically tracked. Second, it enhances precision, lowering the possibility of errors. Third, it delivers real-time attendance updates, making it simple for instructors or administration to monitor attendance and take appropriate action. The aim of the work is expected to improve attendance management in various organisations by optimising attendance monitoring methods while decreasing manual effort and errors. Because the system may be connected with existing attendance management systems or utilised as a standalone solution, it is a versatile and adaptable option for a variety of organisations. The paper’s effort is to design, develop, and implement an efficient system that can track and record attendance accurately using facial recognition technology, giving benefits such as time savings, accuracy, real-time updates, and better security.

Leveraging ResNet and label distribution in advanced intelligent systems for facial expression recognition.

With the development of AI (Artificial Intelligence), facial expression recognition (FER) is a hot topic in computer vision tasks. Many existing works employ a single label for FER. Therefore, the label distribution problem has not been considered for FER. In addition, some discriminative features can not be captured well. To overcome these problems, we propose a novel framework, ResFace, for FER. It has the following modules: 1) a local feature extraction module in which ResNet-18 and ResNet-50 are used to extract the local features for the following feature aggregation; 2) a channel feature aggregation module, in which a channel-spatial feature aggregation method is adopted to learn the high-level features for FER; 3) a compact feature aggregation module, in which several convolutional operations are used to learn the label distributions to interact with the softmax layer. Extensive experiments conducted on the FER+ and Real-world Affective Faces databases demonstrate that the proposed approach obtains comparable performances: 89.87% and 88.38%, respectively.

Study of input images parameters for improving person identification information technology

This article describes the research of the algorithm based on local-textural descriptors, which is the basis of the information technology of person identification, with the aim of increasing its efficiency and reducing the probability of identification errors. An analysis of previous studies has shown that the performance of a face recognition and identification algorithm can vary significantly after applying it to images from different datasets. In addition, the issue of using local texture descriptors in the tasks of face recognition and identification on images of different quality has not been sufficiently studied in modern research. Therefore, the main goal of this work is to study the parameters of face images that will be used for identification process with the appliance of the algorithm based on such local-textural methods as histograms of oriented gradients and local binary patterns in one-dimensional space. During the experiments, it was established that the algorithm identification accuracy rate can be decreased by 35-70% when it is applied to image samples taken in unconstrained conditions. In order to investigate the possibility of reducing the variation of the results, experiments were conducted on the same images with the transformation of their properties, such as format and resolution. During the research, experiments were carried out with several of the most common databases of face images, as a result of which the efficiency of the algorithm reached the level of accuracy of identification of 95% on images taken under controlled conditions and unified within one database. As a result, it was found that in some cases, the conversion of the image format submitted to the input of the algorithm can increase the accuracy of the algorithm’s identification by 5%. Also, the efficiency of the algorithm is affected by the resolution of the input images, in particular, due to the application of the algorithm to transformed images from the Database of Faces database, the accuracy of identification increased by 5-10%, FERET — decreased by 5-35%, SCface — decreased by 10-30%, CFP — increased by 5-10%, Tinyface — increased by 25-35%, LFW — increased by 15%, AgeDB — increased by 5%. The highest identification accuracy rate of the algorithm, which is 95%, was obtained after its application to images taken under constrained conditions and unified within the same database.

ChildPredictor: A Child Face Prediction Framework With Disentangled Learning

The appearances of children are inherited from their parents, which makes it feasible to predict them. Predicting realistic children's faces may help settle many social problems, such as age-invariant face recognition, kinship verification, and missing child identification. It can be regarded as an image-to-image translation task. Existing approaches usually assume domain information in the image-to-image translation can be interpreted by "style", i.e., the separation of image content and style. However, such separation is improper for the child face prediction, because the facial contours between children and parents are not the same. To address this issue, we propose a new disentangled learning strategy for children's face prediction. We assume that children's faces are determined by genetic factors (compact family features, e.g., face contour), external factors (facial attributes irrelevant to prediction, such as moustaches and glasses), and variety factors (individual properties for each child). On this basis, we formulate predictions as a mapping from parents' genetic factors to children's genetic factors, and disentangle them from external and variety factors. In order to obtain accurate genetic factors and perform the mapping, we propose a ChildPredictor framework. It transfers human faces to genetic factors by encoders and back by generators. Then, it learns the relationship between the genetic factors of parents and children through a mapping function. To ensure the generated faces are realistic, we collect a large Family Face Database to train ChildPredictor and evaluate it on the FF-Database validation set. Experimental results demonstrate that ChildPredictor is superior to other well-known image-to-image translation methods in predicting realistic and diverse child faces. Implementation codes can be found at https://github.com/zhaoyuzhi/ChildPredictor.

A Novel Lightweight Deep Convolutional Neural Network Model for Human Emotions Recognition in Diverse Environments

A picture is worth a thousand words to convey information during daily life communication. Recognizing a person’s emotions from facial expressions (FEs) has become a prevalent research field in the past few decades. Deep learning (DL) models, mainly deep convolutional neural networks (DCNNs), are trending in the last few years to perform recognition/classification tasks. Several prior DCNNs have resulted in good recognition accuracy for facial emotion recognition (FER) systems. Still, there is a need for an effective as well as efficient FER system that can recognize the FEs irrespective of the illumination conditions, subjects’ gender, age range, geographical locations, race, etc. In this work, we have presented a novel lightweight DCNN model that can recognize the FEs in the aforementioned conditions. We have made it lightweight by optimal selection of its hidden layers, which ultimately resulted in the reduced number of floating point operations (FLOPs). We have embedded the data augmentation step into the model’s training phase to enhance its generalization ability. An early stopping criterion is also introduced to prevent the model’s overfitting. We have trained and evaluated the performance of the proposed model on widely used benchmarks for FEs databases. We have selected five diverse databases (two collected in the lab, one based on stylized cartoon characters, and two collections in an unconstrained realistic environment), including CK+, Karolinska Directed Emotional Faces (KDEF), Facial Expression Research Group (FERG), Facial Expression Recognition‐2013 (FER‐2013), and Real‐World Affective Faces Database (RaF‐DB). The recognition accuracy of 99.98%, 99.25%, 88.17%, 84.09%, 69.87%, and 69.16% is achieved for the FERG, CK+, KDEF, RaF‐DB, FER‐2013 human faces (FER‐2013H), and FER‐2013 complete (FER‐2013C) database, respectively. Our model outperforms the number of state‐of‐the‐art FER approaches in terms of recognition accuracy and FLOPs.

Retracted] Key Technologies of Face Sensor Recognition Entry System for New Energy Vehicles Based on Particle Swarm Neural Network

In order to solve the problem of high dimensionality and low recognition rate caused by complex calculation in face recognition, the author proposes a face recognition algorithm based on weighted DWT and DCT based on particle swarm neural network applied to new energy vehicles. The algorithm first decomposes the face image with wavelet transform, removes the influence of the diagonal component, the weighted low‐frequency and high‐frequency discrete cosine transform coefficients are extracted as feature vectors, and finally, the particle swarm optimization BP neural network is used for classification and identification. Experimental results show that when the wavelet weights take a0 = 0.9, a1 = 0.05, and a2 = 0.05, the recognition rate reaches the highest. Regardless of whether the low‐frequency component continues to increase or decrease, and the high‐frequency component continues to decrease or increase, the recognition rate will decrease. When the eigenvector dimension is around 60, the recognition rate difference between the weighted wavelet algorithm and the general low‐frequency wavelet algorithm reaches the maximum. The recognition rate of the proposed algorithm is much higher than the other two traditional algorithms. Conclusion. The effectiveness and feasibility of the algorithm are verified on the ORL face database.

Face Templates Encryption Technique Based on Random Projection and Deep Learning

Cancellable biometrics is the solution for the trade-off between two concepts: Biometrics for Security and Security for Biometrics. The cancelable template is stored in the authentication system’s database rather than the original biometric data. In case of the database is compromised, it is easy for the template to be canceled and regenerated from the same biometric data. Recoverability of the cancelable template comes from the diversity of the cancelable transformation parameters (cancelable key). Therefore, the cancelable key must be secret to be used in the system authentication process as a second authentication factor in conjunction with the biometric data. The main contribution of this paper is to tackle the risks of stolen/lost/shared cancelable keys by using biometric trait (in different feature domains) as the only authentication factor, in addition to achieving good performance with high security. The standard Generative Adversarial Network (GAN) is proposed as an encryption tool that needs the cancelable key during the training phase, and the testing phase depends only on the biometric trait. Additionally, random projection transformation is employed to increase the proposed system’s security and performance. The proposed transformation system is tested using the standard ORL face database, and the experiments are done by applying different features domains. Moreover, a security analysis for the proposed transformation system is presented.

CAN THE COMBINATION OF FACIAL FEATURES ENHANCE THE PERFORMANCE OF FACE RECOGNITION?

The field of computer vision and pattern recognition has shown great interest in facial recognition due to its wide range of applications. These applications span across historical and genealogical research, forensic science, searching for missing family members, analyzing social media, automatically managing and annotating image databases, and identifying kinship relationships. This research paper aims to address the challenges associated with facial recognition by introducing two innovative approaches: Fusion-based Classifier Combination (FCC) and Sequential CNN Deep learning-based face recognition (S-CNN). In the first part of the study, we assess the effectiveness of three techniques: Local Binary Patterns (LBP), Histogram of Oriented Gradients (HOG), and a hand-crafted learned technique called Compact Binary Facial Descriptors (CBFD). To overcome these challenges, we employ a classification step that utilizes a novel multi-classifier combination model. In the second part, we propose a novel method where we extract high-level features from multiple image regions treated as sequential data using ensemble of Convolutional Neural Networks (CNNs). These features are then fed into a Deep Neural Network (DNN) for facial recognition. The experimental results obtained from well-known face databases, including Labeled Faces in the Wild (LFW) and ORL, highlight the competitive performance of both the proposed multi-classifier combination model and the S-CNN deep learning model when compared to state-of-the-art methods .

Development of Algorithm for Person Re-Identification Using Extended Openface Method

Deep learning has risen in popularity as a face recognition technology in recent years. Facenet, a deep convolutional neural network (DCNN) developed by Google, recognizes faces with 128 bytes per face. It also claims to have achieved 99.96% on the reputed Labelled Faces in the Wild (LFW) dataset. However, the accuracy and validation rate of Facenet drops down eventually, there is a gradual decrease in the resolution of the images. This research paper aims at developing a new facial recognition system that can produce a higher accuracy rate and validation rate on low-resolution face images. The proposed system Extended Openface performs facial recognition by using three different features i) facial landmark ii) head pose iii) eye gaze. It extracts facial landmark detection using Scattered Gated Expert Network Constrained Local Model (SGEN-CLM). It also detects the head pose and eye gaze using Enhanced Constrained Local Neural field (ECLNF). Extended openface employs a simple Support Vector Machine (SVM) for training and testing the face images. The system’s performance is assessed on low-resolution datasets like LFW, Indian Movie Face Database (IMFDB). The results demonstrated that Extended Openface has a better accuracy rate (12%) and validation rate (22%) than Facenet on low-resolution images.

ARTIFICIAL INTELLIGENCE FACE RECOGNITION: SECURITY SYSTEM

Face recognition has recently been a popular study topic as a result of the growing demand for security. Security systems, identity verification, access control, surveillance systems, and social networks are just a few of the applications that can be expanded from facial recognition. However, as passwords and fingerprint scanners are supplanted by improved computer technology and more precise algorithms, facial recognition is becoming increasingly popular. Facial recognition technology is the simplest and most coherent of all the techniques. Face recognition does not necessitate an individual’s active participation. Face recognition systems are theoretically capable of minimizing risk and eventually avoiding future attacks in places where identity identification is necessary, such as airports and border crossings. If there are criminals on the loose, the same justification can be made as with surveillance devices. Surveillance cameras with face recognition capabilities will aid in the identification of these individuals. Alternatively, these surveillance systems can be used to track down the whereabouts of missing persons, although this is contingent on the use of powerful facial recognition algorithms and a fully built facial database. Face detection, facial feature extraction (image normalization), face identification, and results are the major operations to be conducted for face recognition.