Face Datasets Research Articles

Система верификации личности по изображению лица в защищенном режиме на основе искусственных нейронных сетей

The aim of the work is to develop a system for verifying subjects by face based on a neural network model that is executed in a protected mode. Protected mode means that the identity verification system is highly resistant to destructive influences, such as competitive attacks, and allows storing and processing biometric data without compromising it. The system is based on a biometrics to code converter trained according to GOST R 52633.5, which allows you to associate the subject’s facial biometrics image with its cryptographic key or long password, which can later be used for authentication, and deep convolutional neural networks. For face detection in the image, the MTCNN artificial neural network architecture was used, and several neural network architectures were tested for feature extraction: InceptionResnet, Facenet512, VGG-Face and OpenFace. The best results were shown by the InceptionResnet neural network. When evaluating the effectiveness and testing the reliability of the proposed system on a special dataset of faces collected under different lighting conditions in a room, it was possible to achieve a relatively low value of equal probability of errors of the first and second kind (EER = 0.0146 with a key length of 278 bits), which confirms the effectiveness of the considered approach to building face verification systems.

A SURVEY OF FACIAL GENDER CLASSIFICATION

: Gender is the most critical demographic attribute of human beings. Gender classification from facial images is widely used in computer vision for surveillance, a straightforward task for humans, and a challenging task for machines. This paper provides a study of facial gender classification in the field of computer vision. We highlight the challenges faced during the image capturing process due to factors such as illumination, angle, occlusion, and expressions. We also review various feature extraction approaches used by previous researchers to perform the facial gender classification task. This paper also compares the performances of previous methods on various face datasets to perform gender classification. We observed from the previous studies that good performance had been achieved for a dataset of facial images taken in a controlled environment. However, much more work needs to be done to increase the accuracy and robustness of facial gender classification, especially in uncontrolled environments.

Research on the development strategy of art industry based on RNN model

Abstract In this paper, we firstly propose an art style classification system based on the RNN model to study the development of the art industry, and continuously fine-tune this white noise image according to the long and short-term memory network and gated recurrent unit network until it makes this white noise image similar to the artwork image style, and the ink painting image feature extraction is mainly studied from the recurrent neural network based ink style feature extraction and the overall nested edge based The two aspects of brushstroke feature extraction based on overall nested edge detection are studied. Then, based on the art industry and big data, the ink painting style classification system is constructed, and the effectiveness of the system is verified by model art classification effect analysis. The results show that the RNN performs well on the face dataset, where the accuracy of multiple classifications up to 10 categories is above 85%. This indicates that the RNN model in this paper can maintain good performance in the ink painting art classification task.

Privacy-preserving model for biometric-based authentication and Key Derivation Function

Bio-cryptosystems often save the biometric template for authentication and generally employ randomly generated keys to encrypt and sign data. This method raises privacy protection concerns. Furthermore, for a system secured by the usage of a cryptographic key, losing the key often has disastrous consequences. To overcome the privacy issues, and allow the secure recovery of lost keys, we design a Key Derivation Function to extract a key from biometric data: a new method – based on clustering algorithms – detects consistent and discriminative features from biometric characteristics to create a code. Then, HMAC-SHA256 (as specified by the National Institute of Standards and Technology) generates a standard key from the code. To reproduce the code at future times (that also serves for authentication), the Key Derivation Function stores helper data with the guarantee of privacy. Indeed, with a private face dataset, the probability of generating the code with only the helper data is less than 2−300, and less than 2−246 for a subset of the YouTube Face database. Moreover, on the private database and the tested users from the YouTube Face database, our system has a false acceptance rate of 0%. It corrects up to 40.3% of noise levels on the private database and has good management of the inter-user variability.

Automated anomalous child repetitive head movement identification through transformer networks.

The increasing prevalence of behavioral disorders in children is of growing concern within the medical community. Recognising the significance of early identification and intervention for atypical behaviors, there is a consensus on their pivotal role in improving outcomes. Due to inadequate facilities and a shortage of medical professionals with specialized expertise, traditional diagnostic methods have been unable to effectively address the rising incidence of behavioral disorders. Hence, there is a need to develop automated approaches for the diagnosis of behavioral disorders in children, to overcome the challenges with traditional methods. The purpose of this study is to develop an automated model capable of analyzing videos to differentiate between typical and atypical repetitive head movements in. To address problems resulting from the limited availability of child datasets, various learning methods are employed to mitigate these issues. In this work, we present a fusion of transformer networks, and Non-deterministic Finite Automata (NFA) techniques, which classify repetitive head movements of a child as typical or atypical based on an analysis of gender, age, and type of repetitive head movement, along with count, duration, and frequency of each repetitive head movement. Experimentation was carried out with different transfer learning methods to enhance the performance of the model. The experimental results on five datasets: NIR face dataset, Bosphorus 3D face dataset, ASD dataset, SSBD dataset, and the Head Movements in the Wild dataset, indicate that our proposed model has outperformed many state-of-the-art frameworks when distinguishing typical and atypical repetitive head movements in children.

FlexComb: A Facial Landmark-Based Model for Expression Combination Generation

Facial expressions are a crucial but challenging aspect of animating in-game characters. They provide vital nonverbal communication cues, but given the high complexity and variability of human faces, the task of capturing the natural diversity and affective complexity of human faces can be a labour-intensive process for animators. This motivates the need for more accurate, realistic and lightweight methods for generating emotional expressions for in-game characters. In this work, we introduce FlexComb, a Facial Landmark-based Expression Combination model, designed to generate a real-time space of realistic facial expression combinations. FlexComb leverages the highly varied CelebV-HQ dataset containing emotions in the wild, and a transformer-based architecture. The central component of the FlexComb system is an emotion recognition model that is trained on the facial dataset, and used to generate a larger dataset of tagged faces. The resulting system generates in-game facial expressions by sampling from this tagged dataset, including expressions that combine emotions in specified amounts. This allows in-game characters to take on variety of realistic facial expressions for a single emotion, which addresses this primary challenge of facial emotion modeling. FlexComb shows potential for expressive facial emotion simulation with applications that include animation, video game development, virtual reality, and human-computer interaction.

The Florence multi-resolution 3D facial expression dataset

In the literature, several 3D face datasets have been collected, aiming at advancing the field of 3D face analysis from different perspectives. Data collection generally follows specific research needs, and the existing 3D face datasets all have different characteristics that are tailored for investigating different tasks, encompassing face recognition, facial expressions and emotions analysis, 3D face reconstruction. However, the majority of these datasets are either collected with high-resolution scanners, or consumer level devices, such as the Kinect, the latter being motivated by the burdensome and costly process of collecting high-quality scans. Differently from 2D imagery, the difference in resolution in 3D data represents a non negligible problem that is under-investigated, and still prevents the successful development of methods that can work in real scenarios. In this paper, we propose a new 3D face dataset, named “Florence Multi-Resolution 3D Facial Expression” (Florence 3DMRE), which aims at bridging the gap between high- and low-resolution 3D face datasets. Its peculiarity consists in (1) including high-resolution (HR) models obtained with a HR scanner, and paired samples collected with a Kinect sensor, (2) LR and HR scans are synchronized and capture extreme and asymmetric facial deformations as used in facial rehabilitation exercises. In total, our dataset consists of 14 subjects, each performing 19 complex and asymmetric expressions. For each of them, we collected a high-resolution scan, and an RGB-D sequence. Finally, to highlight the value of the dataset and the challenges it introduces, we use the collected data to perform baseline experiments for cross-resolution 3D face recognition and reconstruction. The dataset is released for research purposes only, and complies to GDPR for data treatment. The dataset can be found at this link.

3D Face Recognition: Two Decades of Progress and Prospects

Three-dimensional (3D) face recognition has been extensively investigated in the last two decades due to its wide range of applications in many areas, such as security and forensics. Numerous methods have been proposed to deal with the challenges faced by 3D face recognition, such as facial expressions, pose variations, and occlusions. These methods have achieved superior performances on several small-scale datasets, including FRGC v2.0, Bosphorus, BU-3DFE, and Gavab. However, deep learning–based 3D face recognition methods are still in their infancy due to the lack of large-scale 3D face datasets. To stimulate future research in this area, we present a comprehensive review of the progress achieved by both traditional and deep learning–based 3D face recognition methods in the last two decades. Comparative results on several publicly available datasets under different challenges of facial expressions, pose variations, and occlusions are also presented.

Face deblurring based on regularized structure and enhanced texture information

Image deblurring is an essential problem in computer vision. Due to highly structured and special facial components (e.g. eyes), most general image deblurring methods and face deblurring methods failed to yield more explicit structure and facial details, resulting in too smooth, uncoordinated and distorted face structure. Considering the unique face texture and sufficient facial details, we present an effective face deblurring network by exploiting more regularized structure and enhanced texture information (RSETNet). We first incorporate the face parsing network with fine-tuning to obtain more accurate face structure, and we present the feature adaptive denormalization (FAD) to regularize the facial structure as a condition of auxiliary to generate more harmonious and undistorted face structure. Meanwhile, to improve the generated facial texture information, we propose a new Laplace depth-wise separable convolution (LDConv) and multi-patch discriminator. Compared with existing methods, our face deblurring method could restore face structure more accurately and with more facial details. Experiments on two public face datasets have demonstrated the effectiveness of our proposed methods in terms of qualitative and quantitative indicators.

LARNeXt: End-to-End Lie Algebra Residual Network for Face Recognition.

Face recognition has always been courted in computer vision and is especially amenable to situations with significant variations between frontal and profile faces. Traditional techniques make great strides either by synthesizing frontal faces from sizable datasets or by empirical pose invariant learning. In this paper, we propose a completely integrated embedded end-to-end Lie algebra residual architecture (LARNeXt) to achieve pose robust face recognition. First, we explore how the face rotation in the 3D space affects the deep feature generation process of convolutional neural networks (CNNs), and prove that face rotation in the image space is equivalent to an additive residual component in the feature space of CNNs, which is determined solely by the rotation. Second, on the basis of this theoretical finding, we further design three critical subnets to leverage a soft regression subnet with novel multi-fusion attention feature aggregation for efficient pose estimation, a residual subnet for decoding rotation information from input face images, and a gating subnet to learn rotation magnitude for controlling the strength of the residual component that contributes to the feature learning process. Finally, we conduct a large number of ablation experiments, and our quantitative and visualization results both corroborate the credibility of our theory and corresponding network designs. Our comprehensive experimental evaluations on frontal-profile face datasets, general unconstrained face recognition datasets, and industrial-grade tasks demonstrate that our method consistently outperforms the state-of-the-art ones.

PIDViT: Pose-Invariant Distilled Vision Transformer for Facial Expression Recognition in the Wild

Many Facial expression recognition methods have achieved great success, but they only considered front facial images or facial images close to the front. Besides, unlike in-the-laboratory datasets, the facial images in the real world (or in the wild) are without lighting and pose control, so that it is a big challenge to recognize these facial expressions. In this paper, the PIDViT (i.e., Pose-Invariant Distilled Vision Transformer) using the teacher-student model for the probability distributions of facial expressions of frontal and multi-pose faces was proposed and solved the pose variance and occlusion issues on expression recognition. First, the multi-pose face dataset FairFace-3D was built from the original FairFace and then used to train pose-invariance on the PIDViT. The PIDViT was trained in two stages; stage 1 is to train the PIDViT to achieve the consistency of facial expressions between frontal faces and multi-pose faces, and stage 2 is to use the student model pre-trained in stage 1 and train facial expressions further on target datasets. Finally, comprehensive experiments were conducted on three in the wild facial expression datasets, and the results validates the generalization of the PIDViT and its superiority over most state-of-the-art models.

The Effects of Head Pose and Face Roundness on Age Progression in Children Faces

Background: This paper analyzes the geometric changes in human face during childhood to estimate the related age; cranial changes are used as age-progression features within childhood stage (0-12) years. Infant face is close to the circular shape turning to an ellipse shape over age progression face oval is determined and drawn using face landmarks, were it’s robust against opening mouth, thinness, fatness or face occlusion by hair.
 Materials and Methods: The experiments depending on two types of face dataset. First one is the standard FG-NET dataset, which was provided with face landmark points numbered from 1 to 68 [14, 15]. Besides, an Internet-based collected data set of (3010) face images extracted from Daily Photo Project.
 Results: Drawn face ellipse provided set of measures that significantly modeled changes in forehead size and face roundness. Studied age period was between birth and 12 years old. Exaggerated head size at birth provides round face with big forehead, which starts shrinking as age progresses natural face.
 Conclusion: Face ellipse provided efficient measures and distances to represent face changes along childhood. Comparing with published researches in young-face age estimation, proposed face ellipse recorded encouraging results.

Face Mask Detection Model Using Convolutional Neural Network

In current times, after the rapid expansion and spread of the COVID-19 outbreak globally, people have experienced severe disruption to their daily lives. One idea to manage the out-break is to enforce people wear a face mask in public places. Therefore, automated and efficient face detection methods are essential for such enforcement. In this paper, a face mask detection model for images has been presented which classifies the images as “with mask” and “without mask”. The model is trained and evaluated using the three datasets Real-World Masked Face Dataset (RMFD), Simulated Masked Face Dataset (SMFD), and Labeled Faces in the Wild (LFW), and attained a performance accuracy rate of 99.72% for first dataset, and 100% for the second and third datasets. This work can be utilized as a digitized scanning tool in schools, hospitals, banks, and airports, and many other public or commercial locations.

Open-Source 3D Morphing Software for Facial Plastic Surgery and Facial Landmark Detection Research and Open Access Face Data Set Based on Deep Learning (Artificial Intelligence) Generated Synthetic 3D Models.

Background: The scarcity of 3D facial models presents a significant hurdle for researchers and educators. Gathering such data demands substantial resources. Objective: To introduce an open-source 3D morphing software to generate 3D facial data sets for research and to provide a large sample data set that is based on synthetically generated 3D models. Methods: Software is developed to morph 3D facial models in bulk by altering landmark locations. Twenty synthetic 3D facial models are generated utilizing deep learning tools and 28 landmarks located on each. The measurements of synthetic models are confirmed to be realistic by comparing them with facial statistics. Several facial deformities and types are simulated at various magnitudes on 3D models to generate a large data set. Results: An open-source software and an open-access data set of 980 3D facial models, each with 28 landmark locations, are provided. Since the data set is based on synthetically generated 3D models, no institutional review board approval is required. Conclusion: The 3D morphing software and the large 3D data set are expected to benefit researchers and educators in the field of facial surgery and facial landmark detection.

CLERA: A Unified Model for Joint Cognitive Load and Eye Region Analysis in the Wild

Non-intrusive, real-time analysis of the dynamics of the eye region allows us to monitor humans’ visual attention allocation and estimate their mental state during the performance of real-world tasks, which can potentially benefit a wide range of human-computer interaction (HCI) applications. While commercial eye-tracking devices have been frequently employed, the difficulty of customizing these devices places unnecessary constraints on the exploration of more efficient, end-to-end models of eye dynamics. In this work, we propose CLERA, a unified model for Cognitive Load and Eye Region Analysis, which achieves precise keypoint detection and spatiotemporal tracking in a joint-learning framework. Our method demonstrates significant efficiency and outperforms prior work on tasks including cognitive load estimation, eye landmark detection, and blink estimation. We also introduce a large-scale dataset of 30 k human faces with joint pupil, eye-openness, and landmark annotation, which aims at supporting future HCI research on human factors and eye-related analysis.

A Two-Stage Emotion Generation Model Combining CGAN and pix2pix

Computer vision has made significant advancements in emotional design. Designers can now utilize computer vision to create emotionally captivating designs that deeply resonate with people. This article aims at enhancing emotional design selection by separating appearance and color. A two-stage emotional design method is proposed, which yields significantly better results compared to classical single-stage methods.. In the Radboud face dataset (RaFD), facial expressions primarily rely on appearance, while color plays a relatively smaller role. Therefore, the two-stage model presented in this article can focus on shape design. By utilizing the SSIM image quality evaluation index, our model demonstrates a 31.63% improvement in generation performance compared to the CGAN model. Additionally, the PSNR image quality evaluation index shows a 10.78% enhancement in generation performance. The proposed model achieves superior design results and introduces various design elements.This article exhibits certain improvements in design effectiveness and scalability compared to conventional models.

Analysis of the prospect of the development of university innovation and entrepreneurship courses based on the chain-type intelligent education model

Abstract Based on the chained intelligent education model, this paper provides practical training and practice activities for the development of university innovation and entrepreneurship courses in colleges and universities by constructing a university innovation and entrepreneurship learning platform to achieve resource sharing of multi-professional experimental bases, thus ensuring that the development of innovation and entrepreneurship courses in all majors in colleges and universities is satisfied. Then the learning platform system is designed and implemented, and the system technical support includes classroom key frame extraction based on deep neural network, S-T graph and Rt-Ch frame feature analysis, and classroom face dataset. Finally, the construction guarantee measures and management suggestions of the university innovation and entrepreneurship platform are proposed. The platform can stimulate students’ entrepreneurial spirit, improve their entrepreneurial ability, expand the knowledge related to innovation and entrepreneurship, and enhance the practical experience. The statistical analysis of the development of the course affecting innovation and entrepreneurship through the survey analysis method yielded a score of 2.17 (standard score of 54.27) for the item of guiding other students in the innovation and entrepreneurship course, which is the lowest score among the 27 questions, indicating that students’ main consciousness has not yet been formed, and their consciousness and ability of active learning and guiding others to learn are to be improved. This study improves students’ innovation spirit, entrepreneurial consciousness and entrepreneurial ability through the transfer of innovative and entrepreneurial knowledge, and it has a guiding reference value for college students to grow into high-quality and innovative talents.

Attention-Guided Network Model for Image-Based Emotion Recognition

Neural networks are increasingly able to outperform traditional machine learning and filtering approaches in classification tasks. However, with the rise in their popularity, many unknowns still exist when it comes to the internal learning processes of the networks in terms of how they make the right decisions for prediction. As a result, in this work, different attention modules integrated into a convolutional neural network coupled with an attention-guided strategy were examined for facial emotion recognition performance. A custom attention block, AGFER, was developed and evaluated against two other well-known modules of squeeze–excitation and convolution block attention modules and compared with the base model architecture. All models were trained and validated using a subset from the OULU-CASIA database. Afterward, cross-database testing was performed using the FACES dataset to assess the generalization capability of the trained models. The results showed that the proposed attention module with the guidance strategy showed better performance than the base architecture while maintaining similar results versus other popular attention modules. The developed AGFER attention-integrated model focused on relevant features for facial emotion recognition, highlighting the efficacy of guiding the model during the integral training process.

Study of image sensors for enhanced face recognition at a distance in the Smart City context

Smart monitoring and surveillance systems have become one of the fundamental areas in the context of security applications in Smart Cities. In particular, video surveillance for Human Activity Recognition (HAR) applied to the recognition of potential offenders and to the detection and prevention of violent acts is a challenging task that is still undergoing. This paper presents a method based on deep learning for face recognition at a distance for security applications. Due to the absence of available datasets on face recognition at a distance, a methodology to generate a reliable dataset that relates the distance of the individuals from the camera, the focal length of the image sensors and the size in pixels of the target face is introduced. To generate the extended dataset, the Georgia Tech Face and Quality Dataset for Distance Faces databases were chosen. Our method is then tested and applied to a set of commercial image sensors for surveillance cameras using this dataset. The system achieves an average accuracy above 99% for several sensors and allows to calculate the maximum distance for a sensor to get the required accuracy in the recognition, which could be crucial in security applications in smart cities.

A Large Benchmark Dataset for Individual Sheep Face Recognition

The mutton sheep breeding industry has transformed significantly in recent years, from traditional grassland free-range farming to a more intelligent approach. As a result, automated sheep face recognition systems have become vital to modern breeding practices and have gradually replaced ear tagging and other manual tracking techniques. Although sheep face datasets have been introduced in previous studies, they have often involved pose or background restrictions (e.g., fixing of the subject’s head, cleaning of the face), which restrict data collection and have limited the size of available sample sets. As a result, a comprehensive benchmark designed exclusively for the evaluation of individual sheep recognition algorithms is lacking. To address this issue, this study developed a large-scale benchmark dataset, Sheepface-107, comprising 5350 images acquired from 107 different subjects. Images were collected from each sheep at multiple angles, including front and back views, in a diverse collection that provides a more comprehensive representation of facial features. In addition to the dataset, an assessment protocol was developed by applying multiple evaluation metrics to the results produced by three different deep learning models: VGG16, GoogLeNet, and ResNet50, which achieved F1-scores of 83.79%, 89.11%, and 93.44%, respectively. A statistical analysis of each algorithm suggested that accuracy and the number of parameters were the most informative metrics for use in evaluating recognition performance.