Face Recognition Performance Research Articles

More learning with less labeling for face recognition

In this paper, we propose an improved face recognition framework where the training is started with a small set of human annotated face images and then new images are incorporated into the training set with minimum human annotation effort. In order to minimize the human annotation effort for new images, the proposed framework combines three different strategies, namely self-paced learning (SPL), active learning (AL), and minimum sparse reconstruction (MSR). As in the recently proposed ASPL framework [1], SPL is used for automatic annotation of easy images, for which the classifiers are highly confident and AL is used to request the help of an expert for annotating difficult or low-confidence images. In this work, we propose to use MSR to subsample the low-confidence images based on diversity using minimum sparse reconstruction in order to further reduce the number of images that require human annotation. Thus, the proposed framework provides an improvement over the recently proposed ASPL framework [1] by employing MSR for eliminating “similar” images from the set selected by AL for human annotation. Experimental results on two large-scale datasets, namely CASIA-WebFace-Sub and CACD show that the proposed method called ASPL-MSR can achieve similar face recognition performance by using significantly less expert-annotated data as compared to the state-of-the-art. In particular, ASPL-MSR requires manual annotation of only 36.10% and 54.10% of the data in CACD and CASIA-WebFace-Sub datasets, respectively, to achieve the same face recognition performance as the case when the whole training data is used with ground truth labels. The experimental results indicate that the number of manually annotated samples have been reduced by nearly 4% and 2% on the two datasets as compared to ASPL [1].

Toward High-quality Face-Mask Occluded Restoration

Face-mask occluded restoration aims at restoring the masked region of a human face, which has attracted increasing attention in the context of the COVID-19 pandemic. One major challenge of this task is the large visual variance of masks in the real world. To solve it we first construct a large-scale Face-mask Occluded Restoration (FMOR) dataset, which contains 5,500 unmasked images and 5,500 face-mask occluded images with various illuminations, and involves 1,100 subjects of different races, face orientations, and mask types. Moreover, we propose a Face-Mask Occluded Detection and Restoration (FMODR) framework, which can detect face-mask regions with large visual variations and restore them to realistic human faces. In particular, our FMODR contains a self-adaptive contextual attention module specifically designed for this task, which is able to exploit the contextual information and correlations of adjacent pixels for achieving high realism of the restored faces, which are however often neglected in existing contextual attention models. Our framework achieves state-of-the-art results of face restoration on three datasets, including CelebA, AR, and our FMOR datasets. Moreover, experimental results on AR and FMOR datasets demonstrate that our framework can significantly improve masked face recognition and verification performance.

Improvement of face recognition performance using a new hybrid subspace classifier

Multiple classification systems play an important role in increasing recognition performance, especially when using heterogeneous classifiers that effectively improve performance. In this study, a new hybrid classifier was designed using heterogeneous Fisherface and discriminative common vector approach (DCVA) subspace recognition methods, which gave successful results in face recognition. While the classification process of DCVA is based on the common properties of signals belonging to the classes, the classification process of Fisherface is based on the different properties of signals. To create a hybrid classifier, called the Hybrid DCVA-Fisherface, the classifiers' decision rules were combined using the Minimum Proportional Score Algorithm and Recognition Update Algorithm. In addition to the proposed subspace classifiers, convolutional neural networks, Transform learning-Alexnet, Alexnet + SVM, and Alexnet + KNN were used for classification. Studies were conducted using the ORL, YALE, Extended YALE B and Face Research Lab London Set (FRLL). To better examine the efficiency of the algorithms, tests were also carried out by downsampling the images. When the experimental results were analysed, the proposed hybrid classifier gave higher recognition rates than all classifiers for ORL, YALE, and Extended YALE B. However, deep learning methods generally achieved better recognition performance than subspace classifiers for the FRLL database, which has more classes than other databases.

Investigating the Impact of Yaw Pose Variation on Facial Recognition Performance

Facial recognition systems often struggle with detecting faces in poses that deviate from the frontal view. Therefore, this paper investigates the impact of variations in yaw poses on the accuracy of facial recognition systems and presents a robust approach optimized to detect faces with pose variations ranging from 0◦ to ±90◦ . The proposed system integrates MTCNN, FaceNet, and SVC, and is trained and evaluated on the Taiwan dataset, which includes face images with diverse yaw poses. The training dataset consists of 89 subjects, with approximately 70 images per subject, and the testing dataset consists of 49 subjects, each with approximately 5 images. Our system achieved a training accuracy of 99.174% and a test accuracy of 96.970%, demonstrating its efficiency in detecting faces with pose variations. These findings suggest that the proposed approach can be a valuable tool in improving facial recognition accuracy in real-world scenarios.

Beyond Frontal Face Recognition

Face recognition is one of the most researched subjects in computer vision. The attention it receives is due to the complexity of the problem. Face recognition models have to deal with a wide variety of intraclass variations such as pose variations, facial expressions, the effect of aging, and natural occlusion due to different illumination. This challenge is often referred to as pose-illumination-expression. Our brain performs face recognition efficiently because we process it holistically. Since achieving human-level accuracy in face recognition is the ultimate goal, we should ascertain whether a computational model mimicking this approach would better tackle this problem. In this research, we developed a computational learning model that closely mimics the way the human visual cortex performs face recognition. It decomposes the recognition task into two specialized sub-tasks. A generator performs the holistic step, followed by a classifier for the recognition step, together referred to as the "holistic model." To deal with the pose variations problem, we introduced the use of calculated distance features known as configural information, which correlate the frontal face with profile face. We compared the holistic model against two baseline models and the current state-of-the-art (or classical models). The experimental results show the holistic model outperforming the current state-of-the-art for the Multi-PIE dataset, by 2.14% and performed as expected for the Labeled Face in the Wild dataset. The ability of the holistic model to recognize a face in any orientation with high accuracy will have a tremendous impact on biometric authentication with liveness detection.

Two-Stream Prototype Learning Network for Few-Shot Face Recognition Under Occlusions

Few-shot face recognition under occlusion (FSFRO) aims to recognize novel subjects given only a few, probably occluded face images, and it is challenging and common in real-world scenarios. Unknown occlusions may deteriorate the class prototypes, while an occluded image in the support set may be critical for recognition if the query image is occluded. This motivates us to propose a novel Two-stream Prototype Learning Network (TSPLN) for FSFR under occlusions by simultaneously considering the quality of support images and their relevance to the query i mage. Specifically, we design a two-stream architecture, which mainly consists of a support-centered stream and query-centered stream, to learn the optimal class prototypes. The former stream is to reduce the negative impact of occluded images on the prototype. This is achieved by exploring the similarities between different images in the support set. In the query-centered stream, we exploit the relevance between the query and support set based on feature alignment (FA). We conduct extensive experiments on two popular datasets: CASIA-WebFace and RMFRD. The experimental results show that our proposed method achieves the state-of-the-art performance for occluded face recognition in the few-shot setting.

Joint Holistic and Masked Face Recognition

With the widespread use of face masks due to the COVID-19 pandemic, accurate masked face recognition has become more crucial than ever. While several studies have investigated masked face recognition using convolutional neural networks (CNNs), there is a paucity of research exploring the use of plain Vision Transformers (ViTs) for this task. Unlike ViT models used in image classification, object detection, and semantic segmentation, the model trained by modern face recognition losses struggles to converge when trained from scratch. To this end, this paper initializes the model parameters via a proxy task of patch reconstruction and observes that the ViT backbone exhibits improved training stability with satisfactory performance for face recognition. Beyond the training stability, two strategies based on prompts are proposed to integrate holistic and masked face recognition in a single framework, namely FaceT. Along with popular holistic face recognition benchmarks, several open-sourced masked face recognition benchmarks are collected for evaluation. Our extensive experiments demonstrate that the proposed FaceT performs on par or better than state-of-the-art CNNs on both holistic and masked face recognition benchmarks. Codes will be made available at https://github.com/zyainfal/Joint-Holistic-and-Masked-Face-Recognition.

Student Attendance Monitoring System using Facial Recognition

In this age of rapidly evolving technologies and trends, performing routine tasks like Attendance Recording should no longer follow the manual or semi-manual tiresome methods. Taking students’ attendance records in the classroom during a lecture period is now a common practice in the University systems especially in most private institutions in Nigeria; a case study employed is Babcock University, Ilisan Remo, Ogun State, Nigeria. This helps to account for where a student is per time, increases student’s learning focus and helps the institution to make the right decision for instance whether to allow a student to take the final exam or not. Attendance marking using conventional methods such as calling students’ names one by one or having them to write it could be quite tedious and time wasting. It becomes more difficult to manage when the class size is large. Another challenge includes the possibility of capturing a proxy attendance such as students writing for their absent colleagues. This study proposes to solve this problem using a biometric information processing known as the facial recognition system. Its application is easier with working range larger than others such as fingerprint, iris scanning and signature. Many algorithms and techniques have been developed to improve facial recognition performance, but the proposed model employs Cascade Classifier which breaks the problem of detecting faces into multiple stages. For each stage, the algorithm performs a very rough and quick test, and if this current stage passes, it does a slightly more detailed test. It eventually detects a face if all stages are passed. The implementation tools include Python, HTML, MySQL, PyCharm, XAMP server as the local host server and a web browser to register and display results. As a result, this development makes attendance monitoring simple, efficient, and time saving. Keywords: Attendance, Monitoring, Biometric, Facial Recognition, Cascade Classifier.

Pose-Invariant Face Recognition via Facial Landmark Based Ensemble Learning

In recent years, pose-invariant face recognition has been mainly approached from a holistic insight. DCNNs (ArcFace, Elastic Face, FaceNet) are used to compute a face image embedding, which is used later to perform face recognition. This paper presents a novel approach to pose-invariant face recognition through the use of ensemble learning and local feature descriptors. The proposed method trains a base learner for each person’s face recognition ensemble system, based on feature vectors (SIFT, GMM, LBP) extracted from image regions surrounding specific facial landmarks. Three different classification models (SVM, Naive Bayes, GMM) are exclusively used as base learners, and the training procedure for each of these models is detailed. The proposed methodology includes a novel face pose descriptor referred to as the Face Angle Vector (FAV) which is utilized by a head pose classification model to determine the pose class of a face image. This model works together with a Base Learner Selection (BLS) block, to determine a set of facial landmarks to extract local feature descriptors, and uses them as the input to their corresponding base learners. Experimental results show a better performance over state-of-the-art methods using the CMU-PIE as the testing dataset, and face poses within ±90°.

Smart and Sustainable Surveillance System

Safety and security are major concerns in the modern day. People and organizations can employ security mechanisms to safeguard their property for their homes or commercial enterprises. Present security systems involve the utilization of Assorted Sensors in cameras for video surveillance. This paper aims at providing one such idea to ensure the protection and security of one’s property. This technique performs Face Recognition as an authentication procedure when a new face is detected by a snapshot. We propose to present a sensible, smart and sustainable Closed - Circuit Television (CCTV) camera with intrusion detection using the LBPH-Local binary pattern histogram algorithm, SIM-Structural Similarity Index Measure, Haar Cascade Classifier, and TKinter. By utilizing intrusion detection, CCTV cameras record real-time videos and process the video at the time of recording to search out the unwanted people arriving within the surveillance area. Our GUI has different buttons supported with features. Adding DL support would create broad scope in this paper such as with DL we would be able to add up much more functionality. We can have future enhancements on this paper such as creating Portable CCTV, Deadly weapon detection, Accident Detection, Fire Detection.

Preprocessing Challenges for Real World Affect Recognition

Real world human affect recognition requires immediate attention which is a significant aspect of humancomputer interaction. Audio-visual modalities can make a significant contribution by providing rich contextual information. Preprocessing is an important step in which the relevant information is extracted. It has a crucial impact on prominent feature extraction and further processing. The main aim is to highlight the challenges in preprocessing real world data. The research focuses on experimental testing and comparative analysis for preprocessing using OpenCV, Single Shot MultiBox Detector (SSD), DLib, Multi-Task Cascaded Convolutional Neural Networks (MTCNN), and RetinaFace detectors. The comparative analysis shows that MTCNN and RetinaFace give better performance in real world data. The performance of facial affect recognition using a pre-trained CNN model is analysed with a lab-controlled dataset CK+ and a representative wild dataset AFEW. This comparative analysis demonstrates the impact of preprocessing issues on feature engineering framework in real world affect recognition.

Hometown context and childhood activities predict face recognition performance

Hometown context and childhood activities predict face recognition performance

Blur and Motion Blur Influence on Recognition Performance of Color Face

Face recognition is an existing and one of the most prominent biometrics techniques, including the processing of images. It is widely used in many applications. The performance of such systems is directly due to face image quality. Since blur and motion blur are common imagery problems, this paper explores the influence of such disturbances on color face recognition performance. The research described in this paper compares the performance of the face recognition algorithm based on the Haar features and Local Binary Patterns Histograms when it uses color face images of good quality, images with added Gaussian blur and motion blur, as well as enhanced images.

PLFace: Progressive Learning for Face Recognition with Mask Bias

The outbreak of the COVID-19 coronavirus epidemic has promoted the development of masked face recognition (MFR). Nevertheless, the performance of regular face recognition is severely compromised when the MFR accuracy is blindly pursued. More facts indicate that MFR should be regarded as a mask bias of face recognition rather than an independent task. To mitigate mask bias, we propose a novel Progressive Learning Loss (PLFace) that achieves a progressive training strategy for deep face recognition to learn balanced performance for masked/mask-free faces recognition based on margin losses. Particularly, our PLFace adaptively adjusts the relative importance of masked and mask-free samples during different training stages. In the early stage of training, PLFace mainly learns the feature representations of mask-free samples. At this time, the regular sample embeddings shrink to the corresponding prototype, which represents the center of each class while being stored in the last linear layer. In the later stage of training, PLFace converges on mask-free samples and further focuses on masked samples until the masked sample embeddings are also gathered in the center of the class. The entire training process emphasizes the paradigm that normal samples shrink first and masked samples gather afterward. Extensive experimental results on popular regular and masked face benchmarks demonstrate that our proposed PLFace can effectively eliminate mask bias in face recognition. Compared to state-of-the-art competitors, PLFace significantly improves the accuracy of MFR while maintaining the performance of normal face recognition.

IHEM Loss: Intra-Class Hard Example Mining Loss for Robust Face Recognition

Recently, angular margin-based methods have become the mainstream approach for unconstrained face recognition with remarkable success. However, robust face recognition still remains a challenge, as the face is subject to variations in pose, age, expression, occlusion, and illumination, especially in unconstrained scenarios. Since the training dataset are always collected in unconstrained scenarios, it is inevitable that there’re significant number of hard examples in the training process. In this paper, we design a hard example selection function to effectively identify hard examples in the training procedure with the supervision of angular margin-based losses. Furthermore, a novel Intra-class Hard Example Mining (IHEM) loss function is proposed, which penalizes the cosine distance between the hard examples and their class centers to enhance the discriminative power of face representations. To ensure high performance for face recognition, we combine the supervision of angular margin-based loss and IHEM loss for model training. Specifically, during the training procedure, the angular margin-based loss guarantees the power of feature discrimination for face recognition, while the IHEM loss further encourages the intra-class compactness of hard example. Extensive results demonstrate the superiority of our approach.

Masked face recognition with convolutional visual self-attention network

With the global outbreak of COVID-19, wearing face masks has been actively introduced as an effective public measure to reduce the risk of virus infection. This measure leads to the failure of face recognition in many cases. Therefore, it is very necessary to improve the recognition performance of masked face recognition (MFR). Inspired by the successful application of self-attention in computer vision, we propose a Convolutional Visual Self-Attention Network (CVSAN), which uses self-attention to augment the convolution operator. Specifically, this is achieved by connecting a convolutional feature map, which enforces local features, to a self-attention feature map that is capable of modeling long-range dependencies. Since there is currently no publicly available large-scale masked face data, we generate a Masked VGGFace2 dataset based on the face detection algorithm to train the CVSAN model. Experiments show that the CVSAN algorithm significantly improves the performance of MFR compared to other algorithms.

Social contact, own-group recognition bias and visual attention to faces.

The own-group recognition bias (OGRB) has been consistently linked to social contact in theoretical accounts. Indeed, social contact is assumed by most authors to underlie the perceptual expertise of out-groups' faces. However, little is known empirically about how it might impact face-processing strategies. We tested the proposition that social interaction would improve the face recognition performance of another group by modulating visual strategies for different face areas. In Experiment 1, we studied visual processes using an eye tracker during a person's first live encounter with a particular member of their own group (European) or an outgroup (African) to explore how increasing familiarity during a first interaction influences face-processing strategies. In Experiments 2 and 3, we explored the effect of simulated intergroup contact on face recognition accuracy, while simultaneously studying the impact of contact on visual attention strategies that occur during recognition (Experiment 2) and encoding (Experiment 3). The results showed a strong OGRB and a difference in visual processes based on the ethnic group of the targets. Although a single interaction is not sufficient to reduce the OGRB, familiarization during a live interaction (Experiment 1) and virtual social contact (Experiment 2) had an impact on the visual strategies employed.

Sistem Pencacah Sampah Berbasis Computer Vision Menggunakan Metode Eigenface

Indonesia is one of the largest countries in Asia, with a very dense population. According to data from The World Bank, the human population indicator in Indonesia in 2019 increased by 270,625.57 people. It shows that population density in Indonesia is related to world problems related to household waste. The household sector contributes as the top waste producer in Indonesia. Adipura data in 2019 stated that at least household waste contributed 36%. This figure is higher than the heap of waste generated from the traditional market, which reaches 24%. The accumulation of waste that occurs without any sorting of waste makes the waste more difficult to decompose and difficult to recycle. Apart from that, the current landfill causes air pollution. Therefore, it is necessary to increase public awareness regarding waste sorting and processing to overcome these problems. We propose to make a device that can help sort organic and inorganic waste with Artificial Intelligence technology based on Computer Vision using the Eigenface method and the Internet of Things. Eigenface is a method that has the working principle of using an XML file to perform face recognition. 1. Implementation of a Computer Vision-Based Garbage Counting System Using the Eigenface Method can run well, where when the system detects an organic object, the door of the counting machine can be opened, and vice versa if it detects inorganic, the machine door is closed. Accuracy results for organic are 70% and for inorganic 75%. It is due to the lack of variation in the dataset and changes in the object's physical condition.

Robots engage face-processing less strongly than humans.

Robot faces often differ from human faces in terms of their facial features (e.g., lack of eyebrows) and spatial relationships between these features (e.g., disproportionately large eyes), which can influence the degree to which social brain [i.e., Fusiform Face Area (FFA), Superior Temporal Sulcus (STS); Haxby et al., 2000] areas process them as social individuals that can be discriminated from other agents in terms of their perceptual features and person attributes. Of interest in this work is whether robot stimuli are processed in a less social manner than human stimuli. If true, this could undermine human-robot interactions (HRIs) because human partners could potentially fail to perceive robots as individual agents with unique features and capabilities-a phenomenon known as outgroup homogeneity-potentially leading to miscalibration of trust and errors in allocation of task responsibilities. In this experiment, we use the face inversion paradigm (as a proxy for neural activation in social brain areas) to examine whether face processing differs between human and robot face stimuli: if robot faces are perceived as less face-like than human-faces, the difference in recognition performance for faces presented upright compared to upside down (i.e., inversion effect) should be less pronounced for robot faces than human faces. The results demonstrate a reduced face inversion effect with robot vs. human faces, supporting the hypothesis that robot faces are processed in a less face-like manner. This suggests that roboticists should attend carefully to the design of robot faces and evaluate them based on their ability to engage face-typical processes. Specific design recommendations on how to accomplish this goal are provided in the discussion.

Bi-level optimization-based projective non-negative matrix factorization

Non-negative matrix factorization (NMF) is a powerful tool for image data analysis and has been utilized in a range of applications involving data classification, clustering, and image processing. Nevertheless, NMF and most of its variants are unsupervised learning algorithms and do not take account of the projection property of the basis image matrix. These flaws will have a negative impact on their classification performance. To resolve the mentioned drawbacks of NMF-based methods, this paper establishes a novel projective non-negative matrix factorization (PNMF) framework using the class-label information and deals with the model via a bi-level quadratic optimization (BLO) strategy. The convergence of the proposed BLO-PNMF algorithm is theoretically analyzed and proved in detail, and also verified by experiments. Our approach and the related state-of-the-art methods are evaluated and compared on three face databases. The experimental results indicate that the BLO-PNMF method can extract more localized features for image data representation and achieve superior performance in face recognition.