Face Datasets Research Articles

Self-Information Forgery Mining for Face Forgery Detection

Abstract In the face of rapid advances in face forgery technology, effective detection methods have become crucial to maintain the authenticity of digital media. Deep learning technology has provided new strategies for recognizing and preventing face forgery in recent years. In this study, a new face forgery detection technique is proposed by utilizing self-information theory, which improves the accuracy and robustness of detection by mining forgery traces, especially in diverse forgery scenarios. The study extracts face features through an improved high-resolution network HRNet and optimizes identity information extraction by combining facial reenactment techniques to detect forged faces efficiently. Experiments have been conducted on several mainstream forged face datasets, and the method presented in this paper can effectively improve the detection performance with an average accuracy of 74.75% on C40 recompressed images. Comparison experiments show that this research method’s frame-level and video-level detection accuracy on the Celeb-DF dataset are 0.9846 and 0.9985, respectively, which are higher than those of existing techniques. Cross-library tests validate the method’s generalization performance, and the AUC metric remains at 0.7305 even in low-quality video environments, which shows good resistance to environmental interference. This study proposes a self-information forgery mining technique that enhances forgery detection accuracy while demonstrating superior generalization ability.

Texture-guided transfer learning for low-quality face recognition.

Although many advanced works have achieved significant progress for face recognition with deep learning and large-scale face datasets, low-quality face recognition remains a challenging problem in real-word applications, especially for unconstrained surveillance scenes. We propose a texture-guided (TG) transfer learning approach under the knowledge distillation scheme to improve low-quality face recognition performance. Unlike existing methods in which distillation loss is built on forward propagation; e.g., the output logits and intermediate features, in this study, the backward propagation gradient texture is used. More specifically, the gradient texture of low-quality images is forced to be aligned to that of its high-quality counterpart to reduce the feature discrepancy between the high-and low-quality images. Moreover, attention is introduced to derive a soft-attention (SA) version of transfer learning, termed as SA-TG, to focus on informative regions. Experiments on the benchmark low-quality face DB's TinyFace and QMUL-SurFace confirmed the superiority of the proposed method, especially more than 6.3% Rank1 accuracy improvement is achieved on TinyFace.

Automated facial characterization and image retrieval by convolutional neural networks.

Developing efficient methods to infer relations among different faces consisting of numerous expressions or on the same face at different times (e.g., disease progression) is an open issue in imaging related research. In this study, we present a novel method for facial feature extraction, characterization, and identification based on classical computer vision coupled with deep learning and, more specifically, convolutional neural networks. We describe the hybrid face characterization system named FRetrAIval (FRAI), which is a hybrid of the GoogleNet and the AlexNet Neural Network (NN) models. Images analyzed by the FRAI network are preprocessed by computer vision techniques such as the oriented gradient-based algorithm that can extract only the face region from any kind of picture. The Aligned Face dataset (AFD) was used to train and test the FRAI solution for extracting image features. The Labeled Faces in the Wild (LFW) holdout dataset has been used for external validation. Overall, in comparison to previous techniques, our methodology has shown much better results on k-Nearest Neighbors (KNN) by yielding the maximum precision, recall, F1, and F2 score values (92.00, 92.66, 92.33, and 92.52%, respectively) for AFD and (95.00% for each variable) for LFW dataset, which were used as training and testing datasets. The FRAI model may be potentially used in healthcare and criminology as well as many other applications where it is important to quickly identify face features such as fingerprint for a specific identification target.

Facial Recognition Databases: Recent Developments and Review of Methods

Facial recognition is one of the most important biometrics that many researchers are increasingly studying, as it is used in various applications such as surveillance, security, law enforcement, information, person identification, smart cards, access control, etc. There is a fundamental relationship between the development of facial recognition algorithms and the possibility of the existence of databases of different faces that influence the appearance of the face in a constrained manner. Standard datasets of images of appropriate size for a subject should be accessible to the public to compare the performance and assessment of identification or verification of a facial recognition system. This paper aims to present a review of the most popular 2D unmasked and masked face datasets available in the current century that are accessible for free download or can be certified with an acceptable effort, where these databases are suitable for training and testing 2D face recognition approaches. Also, this review discussed the evaluation metrics for face recognition and their two types of tasks (identification and verification).

Development of the RIKEN database for dynamic facial expressions with multiple angles

The development of facial expressions with sensing information is progressing in multidisciplinary fields, such as psychology, affective computing, and cognitive science. Previous facial datasets have not simultaneously dealt with multiple theoretical views of emotion, individualized context, or multi-angle/depth information. We developed a new facial database (RIKEN facial expression database) that includes multiple theoretical views of emotions and expressers’ individualized events with multi-angle and depth information. The RIKEN facial expression database contains recordings of 48 Japanese participants captured using ten Kinect cameras at 25 events. This study identified several valence-related facial patterns and found them consistent with previous research investigating the coherence between facial movements and internal states. This database represents an advancement in developing a new sensing system, conducting psychological experiments, and understanding the complexity of emotional events.

The cross-race effect in automatic facial expression recognition violates measurement invariance.

Emotion has been a subject undergoing intensive research in psychology and cognitive neuroscience over several decades. Recently, more and more studies of emotion have adopted automatic rather than manual methods of facial emotion recognition to analyze images or videos of human faces. Compared to manual methods, these computer-vision-based, automatic methods can help objectively and rapidly analyze a large amount of data. These automatic methods have also been validated and believed to be accurate in their judgments. However, these automatic methods often rely on statistical learning models (e.g., deep neural networks), which are intrinsically inductive and thus suffer from problems of induction. Specifically, the models that were trained primarily on Western faces may not generalize well to accurately judge Eastern faces, which can then jeopardize the measurement invariance of emotions in cross-cultural studies. To demonstrate such a possibility, the present study carries out a cross-racial validation of two popular facial emotion recognition systems-FaceReader and DeepFace-using two Western and two Eastern face datasets. Although both systems could achieve overall high accuracies in the judgments of emotion category on the Western datasets, they performed relatively poorly on the Eastern datasets, especially in recognition of negative emotions. While these results caution the use of these automatic methods of emotion recognition on non-Western faces, the results also suggest that the measurements of happiness outputted by these automatic methods are accurate and invariant across races and hence can still be utilized for cross-cultural studies of positive psychology.

A Chinese Face Dataset with Dynamic Expressions and Diverse Ages Synthesized by Deep Learning

Facial stimuli have gained increasing popularity in research. However, the existing Chinese facial datasets primarily consist of static facial expressions and lack variations in terms of facial aging. Additionally, these datasets are limited to stimuli from a small number of individuals, in that it is difficult and time-consuming to recruit a diverse range of volunteers across different age groups to capture their facial expressions. In this paper, a deep-learning based face editing approach, StyleGAN, is used to synthesize a Chinese face dataset, namely SZU-EmoDage, where faces with different expressions and ages are synthesized. Leverage on the interpolations of latent vectors, continuously dynamic expressions with different intensities, are also available. Participants assessed emotional categories and dimensions (valence, arousal and dominance) of the synthesized faces. The results show that the face database has good reliability and validity, and can be used in relevant psychological experiments. The availability of SZU-EmoDage opens up avenues for further research in psychology and related fields, allowing for a deeper understanding of facial perception.

Joint Segmentation and Identification Feature Learning for Occlusion Face Recognition.

The existing occlusion face recognition algorithms almost tend to pay more attention to the visible facial components. However, these models are limited because they heavily rely on existing face segmentation approaches to locate occlusions, which is extremely sensitive to the performance of mask learning. To tackle this issue, we propose a joint segmentation and identification feature learning framework for end-to-end occlusion face recognition. More particularly, unlike employing an external face segmentation model to locate the occlusion, we design an occlusion prediction module supervised by known mask labels to be aware of the mask. It shares underlying convolutional feature maps with the identification network and can be collaboratively optimized with each other. Furthermore, we propose a novel channel refinement network to cast the predicted single-channel occlusion mask into a multi-channel mask matrix with each channel owing a distinct mask map. Occlusion-free feature maps are then generated by projecting multi-channel mask probability maps onto original feature maps. Thus, it can suppress the representation of occlusion elements in both the spatial and channel dimensions under the guidance of the mask matrix. Moreover, in order to avoid misleading aggressively predicted mask maps and meanwhile actively exploit usable occlusion-robust features, we aggregate the original and occlusion-free feature maps to distill the final candidate embeddings by our proposed feature purification module. Lastly, to alleviate the scarcity of real-world occlusion face recognition datasets, we build large-scale synthetic occlusion face datasets, totaling up to 980193 face images of 10574 subjects for the training dataset and 36721 face images of 6817 subjects for the testing dataset, respectively. Extensive experimental results on the synthetic and real-world occlusion face datasets show that our approach significantly outperforms the state-of-the-art in both 1:1 face verification and 1:N face identification.

FaceScape: 3D Facial Dataset and Benchmark for Single-View 3D Face Reconstruction.

In this paper, we present a large-scale detailed 3D face dataset, FaceScape, and the corresponding benchmark to evaluate single-view facial 3D reconstruction. By training on FaceScape data, a novel algorithm is proposed to predict elaborate riggable 3D face models from a single image input. FaceScape dataset releases 16,940 textured 3D faces, captured from 847 subjects and each with 20 specific expressions. The 3D models contain the pore-level facial geometry that is also processed to be topologically uniform. These fine 3D facial models can be represented as a 3D morphable model for coarse shapes and displacement maps for detailed geometry. Taking advantage of the large-scale and high-accuracy dataset, a novel algorithm is further proposed to learn the expression-specific dynamic details using a deep neural network. The learned relationship serves as the foundation of our 3D face prediction system from a single image input. Different from most previous methods, our predicted 3D models are riggable with highly detailed geometry under different expressions. We also use FaceScape data to generate the in-the-wild and in-the-lab benchmark to evaluate recent methods of single-view face reconstruction. The accuracy is reported and analyzed on the dimensions of camera pose and focal length, which provides a faithful and comprehensive evaluation and reveals new challenges. The unprecedented dataset, benchmark, and code have been released to the public for research purpose.

Automatic Facial Palsy Detection—From Mathematical Modeling to Deep Learning

Automated solutions for medical diagnosis based on computer vision form an emerging field of science aiming to enhance diagnosis and early disease detection. The detection and quantification of facial asymmetries enable facial palsy evaluation. In this work, a detailed review of the quantification of facial palsy takes place, covering all methods ranging from traditional manual mathematical modeling to automated computer vision-based methods. Moreover, facial palsy quantification is defined in terms of facial asymmetry indices calculation for different image modalities. The aim is to introduce readers to the concept of mathematical modeling approaches for facial palsy detection and evaluation and present the process of the development of this separate application field over time. Facial landmark extraction, facial datasets, and palsy grading systems are included in this research. As a general conclusion, machine learning methods for the evaluation of facial palsy lead to limited performance due to the use of handcrafted features, combined with the scarcity of the available datasets. Deep learning methods allow the automatic learning of discriminative deep facial features, leading to comparatively higher performance accuracies. Datasets limitations, proposed solutions, and future research directions in the field are also presented.

RBGAN: Realistic-generation and balanced-utility GAN for face de-identification

The rise of computer vision technologies and surveillance devices has led to numerous studies on vision, but it also raises privacy concerns, especially regarding facial identity information. Protecting the privacy of face identity information has prompted the development of face de-identification, which faces two main challenges. The first challenge is the realism of the generated images, while the second challenge is finding the proper balance between privacy protection and preserving data utility. To address these challenges, we propose a face de-identification model called Realistic-Generation and Balanced-Utility GAN (RBGAN). RBGAN includes a disentangled module and a symmetric-consistency-guided generation module, which aim to preserve multiple attributes while generating realistic de-identified images. We design an attribute-specialized encoder to obtain the disentangled representation of identity and attribute to balance privacy protection and data utility preservation. To generate realistic de-identified images, we incorporate symmetric consistency and face reconstruction into the face de-identification process using the Generative Adversarial Network (GAN). The disentangled module can extract richer representation, and the symmetric-consistency-guided generation module can construct more facial details. As a result, our proposed RBGAN can generate de-identified face images that closely resemble real faces and are difficult to detect by visual systems. Experiments conducted on benchmark face datasets demonstrate the effectiveness of our model in generating realistic images while preserving multiple attributes.

A Novel Approach for Human Face Extraction and Detection using SAE-AFB-RFCN Framework

Research into Facial Recognition Technology (FRT), which uses a person's face to identify them, has become a hot topic among scientists. Face recognition relies heavily on feature extraction and classifiers. Occlusion, illuminations, and a complicated background provide the most difficult problems for face recognition systems to overcome. With the advent of Artificial Intelligence and Deep Learning techniques now it became easy to identify different features of an Image and to detect a face. In this paper, Stacked Auto Encoder (SAE), Artificial Feeding Bird (AFB), Region Based Fully Convolutional Network (RFCN), novel approach is developed for human face feature extraction and detection. Initially, the dataset is normalized using rescaling method. Then the Stacked Auto Encoder with Artificial Feeding Birds (SAE-AFB) optimization algorithm is used for facial feature extraction and Region based Fully Convolutional Networks (R-FCN) algorithm is used for detection and classification. The WIDER Face dataset is used for training and testing. Experimental results demonstrate that the proposed SAE-AFB-RFCN framework outperforms the existing algorithms in terms of accuracy, precision, recall and F1-score.

An ensemble of CNNs with self-attention mechanism for DeepFake video detection

The availability of large-scale facial datasets with the rapid progress of deep learning techniques, such as Generative Adversarial Networks, has enabled anyone to create realistic fake videos. These fake videos can potentially become harmful when used for fake news, hoaxes, and identity fraud. We propose a deep learning bagging ensemble classifier to detect manipulated faces in videos. The proposed bagging classifier uses the convolution and self-attention network (CoAtNet) model as a base learner. CoAtNet model is vertically stacking depthwise convolution layers and self-attention layers in such a way that generalization, capacity, and efficiency are improved. Depthwise convolution captures local features from faces extracted from video then pass these features to the attention layers to extract global information and efficiently capture long-range dependencies of spatial details. Each learner is trained on a different subset randomly taken of training data with a replacement then models’ predictions are combined to classify the video either as real or fake. We also use CutMix data augmentation on the extracted faces to enhance the generalization and localization performance of the base learner model. Our experimental results show that our proposed method achieves higher efficiency compared to state-of-the-art methods with AUC values of 99.70%, 97.49%, 98.90%, and 87.62% on the different manipulation techniques of the FaceForensics++ dataset (DeepFakes (DF), Face2Face (F2F), FaceSwap (FS), and NeuralTextures (NT)), respectively, and 99.74% on the Celeb-DF dataset.

Exploring cross-cultural and gender differences in facial expressions: a skin tone analysis using RGB Values

Facial expressions serve as crucial indicators of an individual's psychological state, playing a pivotal role in face-to-face communication. This research focuses on advancing collaboration between machines and humans by undertaking a thorough investigation into facial expressions. Specifically, we delve into the analysis of emotional variations related to changes in skin tone across different genders and cultural backgrounds (Black and white). The research methodology is structured across three phases. In Phase I, image data is acquired and meticulously processed from the Chicago face dataset, resulting in 12,402 augmented images across five classes (Normal case, Benign case, Adenocarcinoma, Squamous-cell-carcinoma, Large-cell-carcinoma). Phase II involves the identification of Regions of Interest (ROI) and the extraction of RGB values as features from these ROIs. Various methods, including those proposed by Kovac, Swift, and Saleh, are employed for precise skin identification. The final phase, Phase III, centers on the in-depth analysis of emotions and presents the research findings. Statistical techniques, such as Descriptive statistics, independent sample T-tests for gender and cross-cultural comparisons, and two-way ANOVA, are applied to RED, BLUE, and GREEN pixel values as response variables, with gender and emotions as explanatory variables. The rejection of null hypotheses prompts a Post Hoc test to discern significant pairs of means. The results indicate that both cross-cultural backgrounds and gender significantly influence pixel colors, underscoring the impact of different localities on pixel coloration. Across various expressions, our results exhibit a minimal 0.05% error rate in all classifications. Notably, the study reveals that green pixel color does not exhibit a significant difference between Anger and Neutral emotions, suggesting a near-identical appearance for green pixels in these emotional states. These findings contribute to a nuanced understanding of the intricate relationship between facial expressions, gender, and cultural backgrounds, providing valuable insights for future research in human–machine interaction and emotion recognition.

Survey Analysis on Secured user Authentication through Biometric Recognition

Secured user authentication is the process of verifying the user authenticity. Biometric authentication is the human identification system employed to match the biometric characteristics of user for verifying the authenticity. Biometric identifiers are exclusive, making it harder to hack accounts using them. Common types of biometrics comprise the fingerprint scanning verifies authentication based on a user's fingerprints Face recognition and voice recognition are employed in real-time application for improving the security level in different application scenarios. Face recognition is a method of identifying or verifying the individual identity using their face expression. Voice recognition is the ability of machine to receive and interpret the dictation to understand. Many researchers carried out their research on different face and voice recognition methods. But, recognition accuracy was not improved with minimum time consumption by existing biometric recognition method. In this research, different recognition methods are reviewed using biometric recognition method for user authentication. The recognition methods are efficiently on human faces dataset with respect to performance metrics like recognition accuracy, error rate, and recognition time.

Two phase representation discriminant projection for face recognition

This paper proposes a novel approach called two-phase representation discriminant projection (TPRDP) for facial recognition. It uses two-phase representation learning to obtain a robust representation for each training sample. In the first phase, the method identifies a set of closely related samples to the training sample based on representation-based distances. In the second phase, TPRDP uses these samples to linearly encode the training sample, effectively enhancing its representation capability. After that, TPRDP constructs two graphs to capture relationships between samples. By leveraging these graphs, TPRDP seeks an optimized transformation matrix that enhances both within-class compactness and between-class separability. Through the projection of samples into a subspace using this matrix, TPRDP effectively improves the discrimination between different classes while preserving compactness within each class. Comprehensive experiments on various face datasets, including AR, CMU PIE, and ExtYaleB, confirm the effectiveness of our method.

Can machine recognize a long-missed old friend? A test to the FaceNet face recognition algorithm

Facial recognition receives heated discussion in recent years. Given the increasing demand for portable and easy-to-use face recognition programs, several coding libraries have been developed that includes the algorithm for identifying and classifying faces. One example of such libraries, the “face_recognition” library developed by Adam Geitgey allows fast execution for face recognition tasks and requires only 1 photo as training input. It adopts the Convolutional Neural Network (CNN) from OpenFace, which is built based on the FaceNet algorithm. Although the study claims that the algorithm has a high base on tests from common face dataset, it is not clear whether the algorithm can identify people consistently when they are in their different ages. In this paper, a face dataset is constructed by collecting publicly available photos from celebrities that can be traced to a certain date of taken. These photos are grouped by 5 pieces per person with each photo taken roughly 4 years apart from the previous one. Then, experiments are conducted to test the effectiveness of face_recognition library on identifying the person from these groups of photos. The results report an over 75% success rate for the algorithm on image identification tasks, indicating a good ability for the face encoding to resist the gap of time.

Enhancing face recognition with latent space data augmentation and facial posture reconstruction

The small amount of training data for many state-of-the-art deep learning-based Face Recognition (FR) systems causes a marked deterioration in their performance. Although a considerable amount of research has addressed this issue by inventing new data augmentation techniques, using either input space transformations or Generative Adversarial Networks (GAN) for feature space augmentations, these techniques have yet to satisfy expectations. In this paper, we propose an approach named Face Representation Augmentation (FRA) for augmenting face datasets. To the best of our knowledge, FRA is the first method that shifts its focus towards manipulating the face embeddings generated by any face representation learning algorithm to create new embeddings representing the same identity and facial emotion but with an altered posture. Extensive experiments conducted in this study convince us of the efficacy of the methodology and its power to provide noiseless, completely new facial representations to improve the training procedure of any FR algorithm. Therefore, FRA can help the recent state-of-the-art FR methods by providing more data for training FR systems. The proposed method, using experiments conducted on the Karolinska Directed Emotional Faces (KDEF) dataset, improves the identity classification accuracies by 9.52 %, 10.04 %, and 16.60 %, in comparison with the base models of MagFace, ArcFace, and CosFace, respectively.

Система верификации личности по изображению лица в защищенном режиме на основе искусственных нейронных сетей

The aim of the work is to develop a system for verifying subjects by face based on a neural network model that is executed in a protected mode. Protected mode means that the identity verification system is highly resistant to destructive influences, such as competitive attacks, and allows storing and processing biometric data without compromising it. The system is based on a biometrics to code converter trained according to GOST R 52633.5, which allows you to associate the subject’s facial biometrics image with its cryptographic key or long password, which can later be used for authentication, and deep convolutional neural networks. For face detection in the image, the MTCNN artificial neural network architecture was used, and several neural network architectures were tested for feature extraction: InceptionResnet, Facenet512, VGG-Face and OpenFace. The best results were shown by the InceptionResnet neural network. When evaluating the effectiveness and testing the reliability of the proposed system on a special dataset of faces collected under different lighting conditions in a room, it was possible to achieve a relatively low value of equal probability of errors of the first and second kind (EER = 0.0146 with a key length of 278 bits), which confirms the effectiveness of the considered approach to building face verification systems.

A SURVEY OF FACIAL GENDER CLASSIFICATION

: Gender is the most critical demographic attribute of human beings. Gender classification from facial images is widely used in computer vision for surveillance, a straightforward task for humans, and a challenging task for machines. This paper provides a study of facial gender classification in the field of computer vision. We highlight the challenges faced during the image capturing process due to factors such as illumination, angle, occlusion, and expressions. We also review various feature extraction approaches used by previous researchers to perform the facial gender classification task. This paper also compares the performances of previous methods on various face datasets to perform gender classification. We observed from the previous studies that good performance had been achieved for a dataset of facial images taken in a controlled environment. However, much more work needs to be done to increase the accuracy and robustness of facial gender classification, especially in uncontrolled environments.