Face Database Research Articles

Considering How Machine‐Learning Algorithms (Re)produce Social Biases in Generated Faces

ABSTRACTAdvances in computer science–specifically in the development and use of generative machine learning–have provided powerful new tools for psychologists to create synthetic human faces as stimuli, which ultimately provide high‐quality photorealistic face images that have many advantages, including reducing typical ethical and privacy concerns and generating face images from minoritized communities that are typically underrepresented in existing face databases. However, there are a number of ways that using machine learning‐based face generation and manipulation software can introduce bias into the research process, thus threatening the validity of studies. The present article provides a summary of how one class of recently popular algorithms for generating faces–generative adversarial networks (GANs)—works, how we control GANs, and where biases (with a particular focus on racial biases) emerge throughout these processes. We discuss recommendations for mitigating these biases, as well as how these concepts manifest in similar modern text‐to‐image algorithms.

Advancements in Multi-Modal Face Recognition: A State-of-the-Art Review of Security Biometrics

This survey is a state-of-the-art review of recent advancements in face recognition, particularly multi-modal face recognition and its associated applications in the domain of security biometrics and identity verification. In this regard, the paper sheds light on the significance of the Sejong Face database and several other leading databases as equally significant tools in studying some of the most challenging tasks in face recognition: occluded and diverse face identification, cross-modality evaluation, and infrared-visible face detection. It also insists on the high challenge of face identification across scenarios and with different forms of disguise. In addition, it emphasizes the need for an effective security framework to be implemented with other major sectors that deal in finances. The purpose is to develop an exhaustive list of methodologies and innovations based on the Sejong Face Database with the intention of finding new methods and algorithms that guarantee accuracy and reliability. Specifically, the application of deep learning methods comprising convolutional neural networks with cross-modality discriminator networks and unit-class loss demonstrates significant advancements in the development of face recognition systems; consequently, these systems exhibit enhanced security and efficacy. The review article is an important source that points out the current status of the research, as well as future possibilities, regarding biometric authentication and its implications in the digital scenario.

Facial expression morphing: enhancing visual fidelity and preserving facial details in CycleGAN-based expression synthesis

Recent advancements in facial expression synthesis using deep learning, particularly with Cycle-Consistent Adversarial Networks (CycleGAN), have led to impressive results. However, a critical challenge persists: the generated expressions often lack the sharpness and fine details of the original face, such as freckles, moles, or birthmarks. To address this issue, we introduce the Facial Expression Morphing (FEM) algorithm, a novel post-processing method designed to enhance the visual fidelity of CycleGAN-based outputs. The FEM method blends the input image with the generated expression, prioritizing the preservation of crucial facial details. We experimented with our method on the Radboud Faces Database (RafD) and evaluated employing the Fréchet Inception Distance (FID) standard benchmark for image-to-image translation and introducing a new metric, FSD (Facial Similarity Distance), to specifically measure the similarity between translated and real images. Our comprehensive analysis of CycleGAN, UNet Vision Transformer cycle-consistent GAN versions 1 (UVCGANv1) and 2 (UVCGANv2) reveals a substantial enhancement in image clarity and preservation of intricate details. The average FID score of 31.92 achieved by our models represents a remarkable 50% reduction compared to the previous state-of-the-art model’s score of 63.82, showcasing the significant advancements made in this domain. This substantial enhancement in image quality is further supported by our proposed FSD metric, which shows a closer resemblance between FEM-processed images and the original faces.

Hawai‘i face database: a racially and ethnically diverse set of facial stimuli

ABSTRACT Within psychology, face perception processes have been widely studied, examining traits and social categories that a face can communicate. However, much of this research has predominately focused on White faces. We review existing face databases that include racially diverse stimuli and note the lack of representation of Asian subgroups (e.g. East, South, and Southeast Asian), Pacific Islanders, as well as both Multiracial and multiethnic faces (especially with multiple minoritized backgrounds). We provide a new racially diverse set of free, standardized images including 140 unique faces representing eight different groups that vary in ethnicity and race (Asian, East Asian, Southeast Asian, Pacific Islander/Native Hawaiian, Hispanic/Latinx, White, Multiracial, and Multiracial Asian), along with norming data. These images and data are available for access use in research: https://osf.io/fkn7y

Children with facial differences experience deficits in emotion skills.

Children with facial differences, such as repaired cleft lip and palate (CLP), may present with reduced capacity for sensorimotor simulation, particularly in the form of facial mimicry. This study examined whether facial mimicry, emotion recognition, and empathy skills are reduced in children with CLP when compared with sex/age-matched controls. A case-control design was utilized. Forty-five children between the ages of 8 and 12 with CLP, and 45 age/sex-matched controls were recruited. Participants completed a facial mimicry task, and facial movements were tracked and quantified using OpenFace. Participants also completed picture and context-based emotion recognition tasks. Picture-based assessment involved identifying emotions from the Dynamic FACES database. Context-based assessment consisted of identifying how a child might feel in various situations. Finally, participants and their parents completed the Empathy Questionnaire for Children and Adolescents (EmQue-CA). Children with CLP presented with significantly reduced facial mimicry (p = .017), picture-based (p < .001), and context-based emotion recognition scores (p < .001) when compared with controls. Better facial mimicry was associated with better picture-based emotion recognition scores in the control group only (r = .22, p < .01). Children with CLP also had significantly lower child and parent-proxy EmQue-CA scores (p < .001). Greater facial mimicry significantly predicted better parent-proxy EmQue-CA scores (p = .016) but did not predict child scores. Children with CLP presented with reduced facial mimicry, poorer emotion recognition, and empathy skills. These findings have a broader relevance as they suggest children with facial differences may present with reduced facial mimicry and/or deficits in emotion recognition. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Diversity-driven Privacy Protection Masks Against Unauthorized Face Recognition

Face recognition (FR) technologies have enabled many life-enriching applications but have also opened doors for potential misuse. Governments, private companies, or even individuals can scrape the web, collect facial images, and build a face database to fuel the FR system to identify human faces without their consent. This paper introduces PMask to combat such a privacy threat against unauthorized FR. It provides a holistic approach to enable privacy-preserving sharing of facial images. PMask preprocesses the facial image and hides its unique facial signature through iterative optimization with dual goals: (i) minimizing the amount of noise to ensure high image quality and (ii) minimizing the perception loss between the privacy-protected face and the original face to ensure the face is recognizable to be the same person by humans. Extensive experiments are conducted on eight representative FR models to evaluate PMask against unauthorized FR. The results validate that PMask provides much stronger protection, introduces less perceptible changes to facial images, and runs faster than state-of-the-art methods to provide privacy protection with a better user experience.

Validity Study of an Emotional Face-Database in Iranian Community

Objective: To investigate the accuracy of facial emotion recognition in the Iranian community, a face database validated in this community is required. To do this, we conducted a validation study on the Radboud face database. The primary objective of this study was to evaluate the accuracy of recognizing emotions through faces in an Iranian sample and then to choose the pictures with high agreement in terms of detecting emotions. Method: This is a cross sectional study recruiting a total number of 142 males and females aged between 20 and 50 years old (Mean ± SD of age 31.7 ± 7.07). The participants were instructed to detect the type of emotion of each face as well as its valence and arousal. The percentage of participants’ agreement on evaluating each picture was assessed. To evaluate the effect of different variables on participants’ accuracy, one way and repeated measure ANOVA analyses were also used. Results: Emotional faces were recognized by around 84% of the participants. The highest accuracy belongs to happy (Mean ± SD of 98 ± 6.1%) and the lowest one to neutral (75 ± 18.06%) faces. The accuracy for detecting other emotions were as follows: sad (91 ± 8.7%), surprised (87 ± 10.64%), angry (77 ± 15.6%), and fearful (76 ± 15.26%). Additionally, we found no differences between male and female participants in terms of recognizing emotions. Then we selected the pictures with high agreement (above 85 percent) in labeling emotions among the participants. Conclusion: The current study provided a valid emotional face database based on Iranian participants’ responses in terms of recognizing basic emotions. The selected pictures can be used in designing tasks to evaluate emotion recognition ability in clinical and nonclinical populations. It can also be used in designing applications to improve detecting emotion in clinical samples such as individuals with autism spectrum disorder.

Long-term photogrammetric outcomes of midface advancement in Apert syndrome: are we nearing normal?

The aesthetic goals of midface surgery in Apert syndrome are to correct the multi-planar midface deficiency and normalize facial ratios. This study characterizes the long-term photogrammetric outcomes of midface advancement in Apert syndrome. Patients with Apert syndrome who underwent midface distraction from 2000 to 2023 were retrospectively reviewed. Soft tissue measurements were applied to preoperative (T0), short-term postoperative (T1), and long-term postoperative (T2) photographs. Long-term changes were compared between subcranial and transcranial procedures, segmental and non-segmental osteotomies, and individual techniques. Frontal facial dimensions at time T2 were compared to age- and sex-matched normal controls from the NIMH-ChEFS face database. Twenty patients had postoperative follow-up of T1 = 0.6 ± 0.4 and T2 = 4.7 ± 1.1years and were compared to 40 normal controls. From time T0 to T2, middle facial third height increased (26.1 ± 3.0% to 27.8 ± 2.6%, p = 0.026), canthal tilt improved (- 7.6° ± 3.7° to - 3.9° ± 4.4°, p < 0.001), and facial convexity increased (182.9° ± 6.2° to 167.9° ± 8.6°, p < 0.001). From time T1 to T2, facial convexity flattened (159.4° ± 10.1° to 167.9° ± 8.6°, p < 0.001). Compared to controls, patients at time T2 had shorter middle facial third height (27.8 ± 2.6% vs. 32.0 ± 1.9%, p < 0.001) and reverse canthal tilt (- 3.9° ± 4.4° vs. 5.4° ± 2.6°, p < 0.001). A tailored treatment approach to the Apert midface yields varying degrees of resolution of central midfacial shortening, canthal tilt, and facial concavity at 5years postoperatively. A slight reduction in facial convexity over time, likely reflecting pseudorelapse in the setting of sagittal overcorrection, should be anticipated. Greater utilization of segmental osteotomies may bring facial third ratios and canthal tilt closer to normal morphometric values.

Altered emotion perception in insomnia, anxiety, depression, mania, psychotic experiences and schizotypal symptoms: a dataset

This data resource provides evidence concerning the prevalence of perceptual alterations of emotional faces amongst individuals experiencing symptoms of insomnia, anxiety, depression, mania, psychotic experiences, and schizotypal tendencies. More specifically, we explored the categorisation accuracy (whether the displayed emotion was correctly identified), misperception (which emotion an incorrect judgment was perceived to be), intensity (extent of the emotion signal strength) and emotional valence (the extent and direction of perceived affect) of six facial expressions of emotion from the Karolinska Directed Emotional Faces database. Complete data from N = 572 respondents are included. The dataset is available to other researchers and is provided on Figshare. Information concerning the data records, usage notes, code availability and technical validation are presented. Finally, we present demographic and correlational data concerning psychiatric symptoms and alterations in the perception of emotional faces.

Quarantine Order Violators System using Face Recognition (FACID)

Face recognition technology is commonly utilized for user authentication and verification by analysing a digital image of a person's face and matching it against a database of faces for identification purposes. The COVID-19 pandemic has led to mandatory quarantines and the use of quarantine bracelets for some individuals, which can be time-consuming and require a lot of effort. Face recognition technology can help raise awareness about the current situation and alleviate some of the burden associated with quarantine measures. The aim of this research is to concentrate on the face recognition of individuals with a history of quarantine, as a measure to prevent the spread of COVID-19 in educational institutions like universities, colleges, and schools. This research study concentrates on individuals with a history of quarantine orders. The system will employ the Histograms of Oriented Gradients (HOG) algorithm for face detection. Additionally, the system will utilize the Face Landmark Algorithm to compare the 128-d vector with images stored locally. The system will make use of the Helen face collection dataset for its data requirements. The aim of this study is to explore and identify techniques for detecting individuals who violate quarantine measures and issuing notifications through the proposed system. By implementing this system, it could contribute to creating a more secure environment within the educational institution and potentially reduce the spread of the virus.

Versatile Papertronics: Photo-Induced Synapse and Security Applications on Papers.

Paper is a readily available material in nature. Its recyclability, eco-friendliness, portability, flexibility, and affordability make it a favored substrate for researchers seeking cost-effective solutions. Electronic devices based on solution process are fabricated on paper and banknotes using PVK and SnO2 nanoparticles. The devices manufactured on paper substrates exhibit photosynaptic behavior under ultraviolet pulse illumination, stemming from numerous interactions on the surface of the SnO2 nanoparticles. A light-modulated artificial synapse device is realized on a paper at a low voltage bias of -0.01V, with an average recognition rate of 91.7% based on the Yale Face Database. As a security device on a banknote, 400 devices in a 20 × 20 array configuration exhibited random electrical characteristics owing to the local morphology of the SnO2 nanoparticles and differences in the depletion layer width at the SnO2/PVK interface. The security Physically Unclonable Functions (PUF) key based on the current distribution extracted at -1V show unpredictable reproducibility with 50% uniformity, 48.7% inter-Hamming distance, and 50.1% bit-aliasing rates. Moreover, the device maintained its properties for more than 210 days under a curvature radius of 8.75mm and bias and UV irradiation stress conditions.

Are contemporary facial recognition algorithms making human facial comparison performance worse?

Facial recognition plays a vital role in several security and law enforcement workflows, such as passport control and criminal investigations. The identification process typically involves a facial recognition system comparing an image against a large database of faces to return a list of probable matches, called a candidate list, for review. A human then looks at the returned images to determine whether there is a match. Most evaluations of these systems tend to examine the performance of the algorithm or human in isolation, not accounting for the interaction that occurs in operational contexts. To ensure optimal whole system performance, it is important to understand how the output produced by an algorithm can impact human performance. Anecdotal claims have been made by users of facial recognition systems that the images being returned by new algorithms in these systems have become more similar in appearance compared to old algorithms, making their job of determining the presence of a match more difficult. This paper explores whether these claims are true and whether the latest facial recognition algorithms decrease human performance compared to an old algorithm from the same company. We examined the performance of 40 novice participants on 120 face matching trials. Each trial required the participant to compare a face image against a candidate list containing eight possible matches returned by either a new or old algorithm (60 trials of each). Overall, participants were more likely to make errors when presented with a candidate list from a new algorithm. Specifically, they were more likely to misidentify an incorrect identity as a match. Participants were more accurate, confident, and faster on candidate lists from the older algorithm. These findings suggest that new algorithms are generating more plausible matches, making the task of determining a match harder for humans. We propose strategies to potentially improve performance and recommendations for future research.

Facial Expression Recognition Using a Semantic-Based Bottleneck Attention Module

Facial expression recognition (FER) is of great value in affective computing. Currently, many FER models have difficulty distinguishing between primary and secondary features. To this end, we propose a FER model based on the semantic bottleneck attention module (BAM). The improvement is that the input image is processed through two branches. BAM uses spatial attention to construct feature maps that can highlight important feature areas and suppress secondary features. The moving flipped bottleneck convolution (MBConv) module introduces random depth, reducing training time and improving model performance. Random depth enables the model to reduce the risk of overfitting during training by randomly skipping some layers. Global average pooling replaces the fully connected layer and compresses the entire feature map (FM) into numerical values to obtain high-purity image features. Through experiments on Extended Cohn-Kanade, Static Facial Expressions in the Wild, and Real-world Affective Faces Database CK+, SFEW, and RAFDB databases, the accuracy of our model reaches 98.33%, 61.14%, and 89.43% respectively, which is better than the other three advanced comparison methods.

AI-Based Visual Early Warning System

Facial expressions are a universally recognised means of conveying internal emotional states across diverse human cultural and ethnic groups. Recent advances in understanding people’s emotions expressed through verbal and non-verbal communication are particularly noteworthy in the clinical context for the assessment of patients’ health and well-being. Facial expression recognition (FER) plays an important and vital role in health care, providing communication with a patient’s feelings and allowing the assessment and monitoring of mental and physical health conditions. This paper shows that automatic machine learning methods can predict health deterioration accurately and robustly, independent of human subjective assessment. The prior work of this paper is to discover the early signs of deteriorating health that align with the principles of preventive reactions, improving health outcomes and human survival, and promoting overall health and well-being. Therefore, methods are developed to create a facial database mimicking the underlying muscular structure of the face, whose Action Unit motions can then be transferred to human face images, thus displaying animated expressions of interest. Then, building and developing an automatic system based on convolution neural networks (CNN) and long short-term memory (LSTM) to recognise patterns of facial expressions with a focus on patients at risk of deterioration in hospital wards. This research presents state-of-the-art results on generating and modelling synthetic database and automated deterioration prediction through FEs with 99.89% accuracy. The main contributions to knowledge from this paper can be summarized as (1) the generation of visual datasets mimicking real-life samples of facial expressions indicating health deterioration, (2) improvement of the understanding and communication with patients at risk of deterioration through facial expression analysis, and (3) development of a state-of-the-art model to recognize such facial expressions using a ConvLSTM model.

MeshWGAN: Mesh-to-Mesh Wasserstein GAN With Multi-Task Gradient Penalty for 3D Facial Geometric Age Transformation.

As the metaverse develops rapidly, 3D facial age transformation is attracting increasing attention, which may bring many potential benefits to a wide variety of users, e.g., 3D aging figures creation, 3D facial data augmentation and editing. Compared with 2D methods, 3D face aging is an underexplored problem. To fill this gap, we propose a new mesh-to-mesh Wasserstein generative adversarial network (MeshWGAN) with a multi-task gradient penalty to model a continuous bi-directional 3D facial geometric aging process. To the best of our knowledge, this is the first architecture to achieve 3D facial geometric age transformation via real 3D scans. As previous image-to-image translation methods cannot be directly applied to the 3D facial mesh, which is totally different from 2D images, we built a mesh encoder, decoder, and multi-task discriminator to facilitate mesh-to-mesh transformations. To mitigate the lack of 3D datasets containing children's faces, we collected scans from 765 subjects aged 5-17 in combination with existing 3D face databases, which provided a large training dataset. Experiments have shown that our architecture can predict 3D facial aging geometries with better identity preservation and age closeness compared to 3D trivial baselines. We also demonstrated the advantages of our approach via various 3D face-related graphics applications.

Coupled Multimodal Emotional Feature Analysis Based on Broad-Deep Fusion Networks in Human-Robot Interaction.

A coupled multimodal emotional feature analysis (CMEFA) method based on broad-deep fusion networks, which divide multimodal emotion recognition into two layers, is proposed. First, facial emotional features and gesture emotional features are extracted using the broad and deep learning fusion network (BDFN). Considering that the bi-modal emotion is not completely independent of each other, canonical correlation analysis (CCA) is used to analyze and extract the correlation between the emotion features, and a coupling network is established for emotion recognition of the extracted bi-modal features. Both simulation and application experiments are completed. According to the simulation experiments completed on the bimodal face and body gesture database (FABO), the recognition rate of the proposed method has increased by 1.15% compared to that of the support vector machine recursive feature elimination (SVMRFE) (without considering the unbalanced contribution of features). Moreover, by using the proposed method, the multimodal recognition rate is 21.22%, 2.65%, 1.61%, 1.54%, and 0.20% higher than those of the fuzzy deep neural network with sparse autoencoder (FDNNSA), ResNet-101 + GFK, C3D + MCB + DBN, the hierarchical classification fusion strategy (HCFS), and cross-channel convolutional neural network (CCCNN), respectively. In addition, preliminary application experiments are carried out on our developed emotional social robot system, where emotional robot recognizes the emotions of eight volunteers based on their facial expressions and body gestures.

How Ethical Is It to Rely on Artificial Intelligence with Biased Facial Recognition?

This paper examines the ethics and current situation of facial recognition from the perspective of racial and gender equality. Facial recognition is a fairly new image analytics technology. It consists of face detection, which dissociates the face from the background, and facial recognition, which compares unique features of an individual’s face to a database and biometrics of sample faces. The primary ethical concern with this technology is biased teaching. Statistics show that a darker-skinned female is significantly less likely to be detected and recognized with the correct identity than a lighter-skinned male. Many works on this issue, some mentioned and analyzed in this paper, underscore the morally and ethically problematic use of this technology in a wide range of important fields considering that facial recognition technology is still in its infancy, and the consequences of misidentification can be severe.

Facial recognition for disaster victim identification

Mass disaster events can result in high levels of casualties that need to be identified. Whilst disaster victim identification (DVI) relies on primary identifiers of DNA, fingerprints, and dental, these require ante-mortem data that may not exist or be easily obtainable. Facial recognition technology may be able to assist. Automated facial recognition has advanced considerably and access to ante-mortem facial images are readily available. Facial recognition could therefore be used to expedite the DVI process by narrowing down leads before primary identifiers are made available. This research explores the feasibility of using automated facial recognition technology to support DVI. We evaluated the performance of a commercial-off-the-self facial recognition algorithm on post-mortem images (representing images taken after a mass disaster) against ante-mortem images (representing a database that may exist within agencies who hold face databases for identity documents (such as passports or driver's licenses). We explored facial recognition performance for different operational scenarios, with different levels of face image quality, and by cause of death. Our research is the largest facial recognition evaluation of post-mortem and ante-mortem images to date. We demonstrated that facial recognition technology would be valuable for DVI and that the performance varies by image quality and cause of death. We provide recommendations for future research.

People Attribute a Range of Highly-Varied and Socially-Bound Meanings to Naturalistic Sad Facial Expressions

Depictions of sadness are commonplace, and here we aimed to discover and catalogue the complex and nuanced ways that people interpret sad facial expressions. We used a rigorous qualitative methodology to build a thematic framework from 3,243 open-ended responses from 41 people who participated in 2020 and described what they thought sad expressors in 80 images were thinking, feeling, and/or intending to do. Face images were sourced from a novel set of naturalistic expressions (ANU Real Facial Expression Database), as well as a traditional posed expression database (Radboud Faces Database). The resultant framework revealed clear themes around the expressors’ thoughts (e.g., acceptance, contemplation, disbelief), social needs (e.g., social support or withdrawal), social behaviours/intentions (e.g., mock or manipulate), and the precipitating events (e.g., social or romantic conflict). Expressions that were perceived as genuine were more frequently described as thinking deeply, reflecting, or feeling regretful, whereas those perceived as posed were more frequently described as exaggerated, overamplified, or dramatised. Overall, findings highlight that facial expressions — even with high levels of consensus about the emotion category they belong to — are interpreted in nuanced and complex ways that emphasise their role as other-oriented social tools, and convey semantically related emotion categories that share smooth gradients with one another. Our novel thematic framework also provides an important foundation for future work aimed at understanding variation in the social functions of sadness, including exploring potential differences in interpretations across cultural settings.

Scoring facial attractiveness with deep convolutional neural networks: How training on standardized images reduces the bias of facial expressions.

In many medical disciplines, facial attractiveness is part of the diagnosis, yet its scoring might be confounded by facial expressions. The intent was to apply deep convolutional neural networks (CNN) to identify how facial expressions affect facial attractiveness and to explore whether a dedicated training of the CNN is able to reduce the bias of facial expressions. Frontal facial images (n = 840) of 40 female participants (mean age 24.5 years) were taken adapting a neutral facial expression and the six universal facial expressions. Facial attractiveness was computed by means of a face detector, deep convolutional neural networks, standard support vector regression for facial beauty, visual regularized collaborative filtering and a regression technique for handling visual queries without rating history. CNN was first trained on random facial photographs from a dating website and then further trained on the Chicago Face Database (CFD) to increase its suitability to medical conditions. Both algorithms scored every image for attractiveness. Facial expressions affect facial attractiveness scores significantly. Scores from CNN additionally trained on CFD had less variability between the expressions (range 54.3-60.9 compared to range: 32.6-49.5) and less variance within the scores (P ≤ .05), but also caused a shift in the ranking of the expressions' facial attractiveness. Facial expressions confound attractiveness scores. Training on norming images generated scores less susceptible to distortion, but more difficult to interpret. Scoring facial attractiveness based on CNN seems promising, but AI solutions must be developed on CNN trained to recognize facial expressions as distractors.