Facial Attributes Research Articles

A Real Time Employee Attendance Monitoring System using ANN

Face recognition refers to the technology that examines and contrasts a person's face characteristics to recognise or verify their identity. Recently, this technology has drawn a lot of attention due to the potential uses it may have in security, marketing, and law enforcement. Face recognition involves studying a picture or video of a person's face to identify features like the space between their eyes, the contour of their nose, and the curve of their mouth. The person's identity is then established or verified by comparing these characteristics to a database of previously saved pictures. A series of techniques called facial recognition algorithms are used to identify and authenticate persons based on the features of their faces. These algorithms compare a person's facial attributes to those in a database of recognised faces by looking at things like the shape of their face, the distance between their eyes, and other distinctive facial features. There are many different types of face recognition algorithms, including geometric-based algorithms, appearance-based algorithms, and hybrid algorithms that combine both approaches. Geometric-based algorithms employ the geometry of face traits to identify and validate people, while appearance-based algorithms use image processing techniques to compare the patterns and textures of facial features. Recent advances in deep learning have significantly improved the accuracy of facial recognition algorithms. Artificial Neural Network (ANN) has shown to be highly effective and have been used in a range of applications, including mobile devices, security, and surveillance. Face recognition algorithms provide advantages, but there are also moral dilemmas with regard to its application, such as potential biases and privacy difficulties. As technology advances, it is imperative to address these problems and ensure that face recognition algorithms are used ethically and responsibly.

Novel Multi-Feature Fusion Facial Aesthetic Analysis Framework

Machine learning has been used in facial beauty prediction studies. However, the integrity of facial geometric information is not considered in facial aesthetic feature extraction, and the impact of other facial attributes (expression) on aesthetics. We propose a novel multi-feature fusion facial aesthetic analysis framework (NMFA) to overcome this problem. First, we designed a facial shape feature, which is an intuitive, visual quantitative description, based on B-spline. Second, we designed a representative low-dimensional facial structural feature to establish the theoretical basis of the facial structure, based on facial aesthetic structure and expression recognition theory. Next, we designed texture and holistic features based on Gabor and VGG-face network. Finally, we used a multi-feature fusion strategy to fuse them for aesthetic evaluation. Experiments were conducted on four databases. The results revealed that the proposed method realizes the visualization of facial shape features, enriches geometric information, solves the problem of lack of facial geometric information and difficulty to understand, and achieves excellent performance with fewer parameters.

Neutral Face Learning and Progressive Fusion Synthesis Network for NIR-VIS Face Recognition

To meet the strong demand for deploying face recognition systems in low-light scenarios, the Near-InfraRed and VISible (NIR-VIS) face recognition task is receiving increasing attention. However, heterogeneous faces have the characteristics of heterogeneity and non-neutrality. Heterogeneity refers to the fact that the matching images are in different modalities, and non-neutrality means that the matching images are significantly different in pose, expression, lighting, etc. Both situations pose challenges for NIR-VIS face matching. To address this problem, we propose a novel Neutral face Learning and Progressive Fusion synthesis (NLPF) network to disentangle the latent attributes of heterogeneous faces and learn neutral face representations. Our approach naturally integrates Identity-related Neutral face Learning (INL) and Attribute Progressive Fusion (APF) into a joint framework. Firstly, INL eliminates modal variations and residual variations by guiding the network to learn homogeneous neutral face feature representations, which tackles the challenge of heterogeneity and non-neutrality by mapping cross-modal images to a common neutral representation subspace. Besides, APF is presented to perform the disentanglement and reintegration of identity-related features, modality-related features and residual features in a progressive fusion manner, which helps to further purify identity-related features. Comprehensive evaluations are carried out on three mainstream NIR-VIS datasets to verify the robustness and effectiveness of the NLPF model. In particular, NLPF has competitive recognition performance on LAMP-HQ, the most challenging NIR-VIS dataset so far.

Multi-Attention Infused Integrated Facial Attribute Editing Model: Enhancing the Robustness of Facial Attribute Manipulation

Facial attribute editing refers to the task of modifying facial images by altering specific target facial attributes. Existing approaches typically rely on the combination of generative adversarial networks and encoder–decoder architectures to tackle this problem. However, current methods may exhibit limited accuracy when dealing with certain attributes. The primary objective of this research is to enhance facial image modification based on user-specified target facial attributes, such as hair color, beard removal, or gender transformation. During the editing process, it is crucial to selectively modify only the regions relevant to the target attributes while preserving the details of other unrelated facial attributes. This ensures that the editing results appear more natural and realistic. This study introduces a novel approach called MAGAN (Combining GRU Structure and Additive Attention with AGU—Adaptive Gated Units). Moreover, a discriminative attention mechanism is introduced to automatically identify key regions in the input images that are relevant to facial attributes. This mechanism concentrates attention on these regions, enhancing the model’s ability to accurately capture and analyze subtle facial attribute features. The method incorporates external attention within the convolutional layers of the encoder–decoder architecture, facilitating the modeling of linear complexity across image regions and implicitly considering correlations among all data samples. By employing discriminative attention in the discriminator, the model achieves more precise attribute editing. To evaluate the effectiveness of MAGAN, experiments were conducted on the CelebA dataset. The average precision of facial attribute generation in images edited by our model stands at 91.83%. PSNR and SSIM for reconstructed images are 32.52 and 0.957, respectively. In comparison with existing methodologies (AttGAN, STGAN, MUGAN), noteworthy enhancements have been achieved in the domain of facial attribute manipulation.

Nonlocal contrast calculated by the second order visual mechanisms and its significance in identifying facial emotions.

Background: Previously obtained results indicate that faces are / preattentively/ detected in the visual scene very fast, and information on facial expression is rapidly extracted at the lower levels of the visual system. At the same time different facial attributes make different contributions in facial expression recognition. However, it is known, among the preattentive mechanisms there are none that would be selective for certain facial features, such as eyes or mouth. The aim of our study was to identify a candidate for the role of such a mechanism. Our assumption was that the most informative areas of the image are those characterized by spatial heterogeneity, particularly with nonlocal contrast changes. These areas may be identified / in the human visual system/ by the second-order visual / mechanisms/ filters selective to contrast modulations of brightness gradients. Methods: We developed a software program imitating the operation of these / mechanisms/ filters and finding areas of contrast heterogeneity in the image. Using this program, we extracted areas with maximum, minimum and medium contrast modulation amplitudes from the initial face images, then we used these to make three variants of one and the same face. The faces were demonstrated to the observers along with other objects synthesized the same way. The participants had to identify faces and define facial emotional expressions. Results: It was found that the greater is the contrast modulation amplitude of the areas shaping the face, the more precisely the emotion is identified. Conclusions: The results suggest that areas with a greater increase in nonlocal contrast are more informative in facial images, and the second-order visual / mechanisms/ filters can claim the role of /filters/ elements that detect areas of interest, attract visual attention and are windows through which subsequent levels of visual processing receive valuable information.

Deep Learning Model for Gender and Age Detection

Age and gender, two significant facial attributes, wield considerable influence within society. The automation of age and gender recognition holds substantial promise across a spectrum of real-world applications, including customer service, priority voting systems, medical diagnosis, and human-computer interaction. Leveraging deep learning techniques has emerged as a common thread in many research endeavours, yielding noteworthy performance enhancements. The integration of diverse deep learning models, coupled with an assessment of accuracy improvements, paves the way for further exploration. The central objective of this paper is to meticulously scrutinize age and gender recognition across various datasets and deep learning models. The paper expounds upon the advancements achieved in this domain, accentuating the contributions made. It outlines the models and datasets employed and rigorously evaluates

Glitch in the matrix: A large scale benchmark for content driven audio–visual forgery detection and localization

Most deepfake detection methods focus on detecting spatial and/or spatio-temporal changes in facial attributes and are centered around the binary classification task of detecting whether a video is real or fake. This is because available benchmark datasets contain mostly visual-only modifications present in the entirety of the video. However, a sophisticated deepfake may include small segments of audio or audio–visual manipulations that can completely change the meaning of the video content. To addresses this gap, we propose and benchmark a new dataset, Localized Audio Visual DeepFake (LAV-DF), consisting of strategic content-driven audio, visual and audio–visual manipulations. The proposed baseline method, Boundary Aware Temporal Forgery Detection (BA-TFD), is a 3D Convolutional Neural Network-based architecture which effectively captures multimodal manipulations. We further improve (i.e. BA-TFD+) the baseline method by replacing the backbone with a Multiscale Vision Transformer and guide the training process with contrastive, frame classification, boundary matching and multimodal boundary matching loss functions. The quantitative analysis demonstrates the superiority of BA-TFD+ on temporal forgery localization and deepfake detection tasks using several benchmark datasets including our newly proposed dataset. The dataset, models and code are available at https://github.com/ControlNet/LAV-DF.

Mitigating bias in facial analysis systems by incorporating label diversity

Facial analysis models are increasingly applied in real-world applications that significantly impact people’s lives. However, as literature has shown, models that automatically classify facial attributes might exhibit algorithmic discrimination behavior concerning protected groups, potentially negatively impacting individuals and society. It is, therefore, critical to develop techniques that can mitigate unintended biases in facial classifiers. Hence, this work introduces a novel learning method combining subjective human-based labels and objective annotations based on mathematical definitions of facial traits. Specifically, we generate new objective annotations from two large-scale human-annotated datasets, each capturing a different perspective of the analyzed facial trait. We then propose an ensemble learning method, which combines individual models trained on different types of annotations. We provide an in-depth analysis of the annotation procedure as well as the datasets distribution. We next show how much each individual model, each trained on a different set of annotations, contributes to the final ensemble. Finally, we empirically show that, by incorporating label diversity in the model pipeline, our method successfully mitigates unintended biases while maintaining significant accuracy on the downstream tasks. Our method is generic and can be applied to different types of facial tasks.

Reconstructing controllable faces from brain activity with hierarchical multiview representations

Reconstructing visual experience from brain responses measured by functional magnetic resonance imaging (fMRI) is a challenging yet important research topic in brain decoding, especially it has proved more difficult to decode visually similar stimuli, such as faces. Although face attributes are known as the key to face recognition, most existing methods generally ignore how to decode facial attributes more precisely in perceived face reconstruction, which often leads to indistinguishable reconstructed faces. To solve this problem, we propose a novel neural decoding framework called VSPnet (voxel2style2pixel) by establishing hierarchical encoding and decoding networks with disentangled latent representations as media, so that to recover visual stimuli more elaborately. And we design a hierarchical visual encoder (named HVE) to pre-extract features containing both high-level semantic knowledge and low-level visual details from stimuli. The proposed VSPnet consists of two networks: Multi-branch cognitive encoder and style-based image generator. The encoder network is constructed by multiple linear regression branches to map brain signals to the latent space provided by the pre-extracted visual features and obtain representations containing hierarchical information consistent to the corresponding stimuli. We make the generator network inspired by StyleGAN to untangle the complexity of fMRI representations and generate images. And the HVE network is composed of a standard feature pyramid over a ResNet backbone. Extensive experimental results on the latest public datasets have demonstrated the reconstruction accuracy of our proposed method outperforms the state-of-the-art approaches and the identifiability of different reconstructed faces has been greatly improved. In particular, we achieve feature editing for several facial attributes in fMRI domain based on the multiview (i.e., visual stimuli and evoked fMRI) latent representations.

FACE SPOOF DETECTION USING CONVOLUTIONAL NEURAL NETWORK

The paper suggests a technique that uses convolutional neural network (CNN) to identify fraudulent facial manipulation. The proposed method comprises teaching an intricate neural network using a comprehensive compilation of genuine and fake facial images. The structure of CNN includes several layers of convolution and pooling, which enable it to identify distinguishing features in the input images. Following its training, the model is employed to differentiate a presented facial image into either authentic or fraudulent. To determine the efficacy of the proposed technique, a standardized data set for identifying counterfeit or altered facial attributes was used. The suggested method presents various benefits compared to current techniques for detecting face spoofing. To start with, utilizing a deep CNN empowers the model to acquire complex and discerning characteristics from the input images, thus augmenting the precision of the categorization mission. Additionally, the suggested method is effective in terms of computational requirements, enabling its utilization in real-time scenarios. The proposed methodology is able to withstand a range of fraudulent tactics used on facial recognition systems, such as print and replay attacks. The findings from this study aid in the progression of face recognition technology by enhancing the accuracy and dependability of fraud detection systems. These improved systems have practical applications in security measures, biometric identification, and digital criminal investigations. The suggested method could substantially enhance the dependability and safety of facial recognition systems, consequently boosting their functional value and credibility.

JR2Net: Joint Monocular 3D Face Reconstruction and Reenactment

Face reenactment and reconstruction benefit various applications in self-media, VR, etc. Recent face reenactment methods use 2D facial landmarks to implicitly retarget facial expressions and poses from driving videos to source images, while they suffer from pose and expression preservation issues for cross-identity scenarios, i.e., when the source and the driving subjects are different. Current self-supervised face reconstruction methods also demonstrate impressive results. However, these methods do not handle large expressions well, since their training data lacks samples of large expressions, and 2D facial attributes are inaccurate on such samples. To mitigate the above problems, we propose to explore the inner connection between the two tasks, i.e., using face reconstruction to provide sufficient 3D information for reenactment, and synthesizing videos paired with captured face model parameters through face reenactment to enhance the expression module of face reconstruction. In particular, we propose a novel cascade framework named JR2Net for Joint Face Reconstruction and Reenactment, which begins with the training of a coarse reconstruction network, followed by a 3D-aware face reenactment network based on the coarse reconstruction results. In the end, we train an expression tracking network based on our synthesized videos composed by image-face model parameter pairs. Such an expression tracking network can further enhance the coarse face reconstruction. Extensive experiments show that our JR2Net outperforms the state-of-the-art methods on several face reconstruction and reenactment benchmarks.

FlowFace: Semantic Flow-Guided Shape-Aware Face Swapping

In this work, we propose a semantic flow-guided two-stage framework for shape-aware face swapping, namely FlowFace. Unlike most previous methods that focus on transferring the source inner facial features but neglect facial contours, our FlowFace can transfer both of them to a target face, thus leading to more realistic face swapping. Concretely, our FlowFace consists of a face reshaping network and a face swapping network. The face reshaping network addresses the shape outline differences between the source and target faces. It first estimates a semantic flow (i.e. face shape differences) between the source and the target face, and then explicitly warps the target face shape with the estimated semantic flow. After reshaping, the face swapping network generates inner facial features that exhibit the identity of the source face. We employ a pre-trained face masked autoencoder (MAE) to extract facial features from both the source face and the target face. In contrast to previous methods that use identity embedding to preserve identity information, the features extracted by our encoder can better capture facial appearances and identity information. Then, we develop a cross-attention fusion module to adaptively fuse inner facial features from the source face with the target facial attributes, thus leading to better identity preservation. Extensive quantitative and qualitative experiments on in-the-wild faces demonstrate that our FlowFace outperforms the state-of-the-art significantly.

Appearance Satisfaction Among Adolescent Girls in Australia, China, India, and Iran: The Role of Perceived Actual-Ideal Discrepancies in Facial and Bodily Attributes

The current study examined the associations between perceived actual-ideal discrepancies in facial and bodily attributes and adolescent girls’ appearance satisfaction and whether these relationships were moderated by the importance associated with the attributes and/or the country of the participants. A multilingual survey was completed by 900 girls aged 12–18 years old living in Australia, China, India, and Iran. Girls in India and Iran were most satisfied with their appearance followed by girls in China and Australia. Iranian girls had the highest perceived actual-ideal discrepancies in facial and bodily attributes. Chinese and Indian girls perceived their facial attributes to be more important to their overall sense of appearance than their bodily attributes, whereas Australian and Iranian girls valued them equally. Higher perceived actual-ideal facial discrepancies were related to lower appearance satisfaction only for Iranian girls and higher perceived bodily discrepancies were linked to lower appearance satisfaction only for Australian girls. The importance associated with physical attributes and/or the country of participants did not moderate the relationship between perceived discrepancies and appearance satisfaction for facial or bodily attributes. Findings underscore the critical role of cultural nuances in understanding body image among adolescent girls and determinants of appearance satisfaction.

Fast Image Classification Algorithms Based on Sequential Analysis

This work is devoted to fast image recognition techniques based on statistical sequential analysis. We examine the possibility to sequentially process principal components and organize a convolutional neural network with early exits. We pay special attention to sequentially learning a multi-task lightweight neural network model in order to predict several facial attributes (age, gender, and ethnicity) based on preliminary training on the face classification task. We specifically note that the entire above-mentioned model should be fine-tuned in order to deal with the emotion recognition problem. Experimental evaluation on several datasets demonstrates that the proposed approach has high accuracy and very low running time and space complexity compared to known state-of-the-art methods.

A Multi-Input Fusion Model for Privacy and Semantic Preservation in Facial Image Datasets

The widespread application of multimedia technologies such as video surveillance, online meetings, and drones facilitates the acquisition of a large amount of data that may contain facial features, posing significant concerns with regard to privacy. Protecting privacy while preserving the semantic contents of facial images is a challenging but crucial problem. Contemporary techniques for protecting the privacy of images lack the incorporation of the semantic attributes of faces and disregard the protection of dataset privacy. In this paper, we propose the Facial Privacy and Semantic Preservation (FPSP) model that utilizes similar facial feature replacement to achieve identity concealment, while adding semantic evaluation to the loss function to preserve semantic features. The proposed model is versatile and efficient in different task scenarios, preserving image utility while concealing privacy. Our experiments on the CelebA dataset demonstrate that the model achieves a semantic preservation rate of 77% while concealing the identities in facial images in the dataset.

Enhanced Embedded AutoEncoders: An Attribute-Preserving Face De-Identification Framework

Nowadays, face recognition technology has been dramatically improved by the advances in deep learning and big data fields. However, this also poses grand challenges in protecting personal identity information in intelligent applications of the Internet of Things (IoT). Existing methods based on the K-Same algorithm have low effectiveness for protecting personal identity while preserving face attributes. In this paper, we propose an attribute-preserving face de-identification framework called Enhanced Embedded AutoEncoders to address this problem. Our framework consists of three parts: a Privacy Removal Network, a Feature Selection Network and a Privacy Evaluation Network. The main purpose of our framework is to ensure that the Privacy Removal Network is capable of discarding information involving identity privacy and retaining desired face attributes for certain prediction applications. In order to achieve this goal, the design of the Privacy Removal Network is crucial. Specifically, we employ two different autoencoders, one of which is embedded within the other. Extensive experimental results show that our framework outperforms existing methods by an average of 3.42%-26.22% in terms of data utility under comparable face de-identification performance, which indicates that the proposed framework can not only effectively retain face attributes but also protect personal identity well.

Study on emotion recognition bias in different regional groups

Human-machine communication can be substantially enhanced by the inclusion of high-quality real-time recognition of spontaneous human emotional expressions. However, successful recognition of such expressions can be negatively impacted by factors such as sudden variations of lighting, or intentional obfuscation. Reliable recognition can be more substantively impeded due to the observation that the presentation and meaning of emotional expressions can vary significantly based on the culture of the expressor and the environment within which the emotions are expressed. As an example, an emotion recognition model trained on a regionally-specific database collected from North America might fail to recognize standard emotional expressions from another region, such as East Asia. To address the problem of regional and cultural bias in emotion recognition from facial expressions, we propose a meta-model that fuses multiple emotional cues and features. The proposed approach integrates image features, action level units, micro-expressions and macro-expressions into a multi-cues emotion model (MCAM). Each of the facial attributes incorporated into the model represents a specific category: fine-grained content-independent features, facial muscle movements, short-term facial expressions and high-level facial expressions. The results of the proposed meta-classifier (MCAM) approach show that a) the successful classification of regional facial expressions is based on non-sympathetic features b) learning the emotional facial expressions of some regional groups can confound the successful recognition of emotional expressions of other regional groups unless it is done from scratch and c) the identification of certain facial cues and features of the data-sets that serve to preclude the design of the perfect unbiased classifier. As a result of these observations we posit that to learn certain regional emotional expressions, other regional expressions first have to be “forgotten”.

Analysis and Study of Fatigue Driving Detection System based on Multimodal Fusion

Fatigue driving always endangers road traffic safety and is one of the main causes of traffic accidents. How to effectively monitor and determine the fatigue driving state, so as to realize the fatigue early warning, has become a hot spot of scientific research. The traditional detection methods are mostly based on a certain human physiological parameters, through the monitoring of the physiological parameters changes, to complete the determination of the fatigue state. Compared with the traditional detection methods, the fatigue detection method based on computer vision has the advantages of accurate and reliable detection results, detection not relying on wearable devices, and good human-computer interaction experience. However, in the specific algorithm application scenarios, there are often interference from environmental factors such as partial occlusion of the inspected image, low image resolution, complex illumination of the detection environment, and unstable acquisition equipment. In the face of many challenges, how to effectively improve the detection accuracy of fatigue state, and the algorithm has the practical application space, still needs to be explored. The traditional fatigue detection method based on facial information often detects and analyzes a certain subordination. Due to the detection of individual differences and the fuzzy definition of fatigue state, the algorithm has the problems of high recognition error rate and poor robustness. Moreover, the single attribute model separates the potential integration and exclusion relationship between facial attributes. At the same time, the huge model size and complex model structure also limit the actual implementation of the algorithm. In view of these urgent problems, this paper first proposes the fusion model of face detection and head posture analysis based on the multi-task convolutional neural network (MTCNN). The model uses the architecture strategy of multi-task cascade to complete the complete facial posture analysis task while completing the facial detection task. Secondly, based on the lightweight convolutional neural network (SqueezeNet) of the network cascade, the facial key point detection network is designed to obtain the 72-point key point coordinates of the face. For the obtained facial attribute information, extract fatigue features and propose a fatigue determination method of multimodal fusion. Finally, combined with the research results, the construction of the online fatigue detection and early warning system platform was completed.

FACE-Q Satisfaction Ratings Are Higher After Facial Feminization Surgery Than Hormone Replacement Therapy Alone.

Facial feminization surgery (FFS) is associated with improved psychological outcomes in transgender patients. We aimed to evaluate the impact of FFS on patient satisfaction with facial appearance using validated, patient-reported outcome tools. Patients were recruited to complete a FACE-Q survey at least 6 months after their FFS. FACE-Q modules used included "Satisfaction with Facial Appearance Overall," "Facial Attributes" (forehead/eyebrows, nose, cheeks, cheekbone, chin, jawline, and neck), "Decision," "Outcomes," "Psychological Function," and "Social Function." "Satisfaction with Facial Appearance Overall" was compared to FACE-Q outcomes in transgender patients on hormone replacement therapy (HRT), alone. Twenty-three patients completed the survey. Rhinoplasty was the most common procedure (82.6%). Insurance was rated as a major barrier in 50% of patients. "Satisfaction with Facial Appearance Overall" using the FACE-Q was 69.3, with "Psychological Function" rated at 73.7 and "Social Function" rated at 76.1. FFS was associated with a higher overall satisfaction in facial appearance compared to patients on HRT for 3 months (69.3 vs. 48.5, p<0.01) and 12 months, alone (69.3 vs. 54.4, p<0.001). Wait time for FFS of less than a year (β=-20.4, p=0.04) and undergoing FFS at a younger age (β=-1.4, p<0.001) were independently associated with higher satisfaction scores in overall facial appearance. FFS is associated with greater satisfaction with overall facial appearance, especially compared to transgender patients only on HRT. Insurance was the greatest barrier to receiving FFS. Improved access to surgery is needed to yield higher overall satisfaction with facial appearance.

SPL-Net: Spatial-Semantic Patch Learning Network for Facial Attribute Recognition with Limited Labeled Data

SPL-Net: Spatial-Semantic Patch Learning Network for Facial Attribute Recognition with Limited Labeled Data