Facial Attributes Research Articles

Disentangled Lifespan Synthesis via Transformer‐Based Nonlinear Regression

AbstractLifespan face age transformation aims to generate facial images that accurately depict an individual's appearance at different age stages. This task is highly challenging due to the need for reasonable changes in facial features while preserving identity characteristics. Existing methods tend to synthesize unsatisfactory results, such as entangled facial attributes and low identity preservation, especially when dealing with large age gaps. Furthermore, over‐manipulating the style vector may deviate it from the latent space and damage image quality. To address these issues, this paper introduces a novel nonlinear regression model‐Disentangled Lifespan face Aging (DL‐Aging) to achieve high‐quality age transformation images. Specifically, we propose an age modulation encoder to extract age‐related multi‐scale facial features as key and value, and use the reconstructed style vector of the image as the query. The multi‐head cross‐attention in the W+ space is utilized to update the query for aging image reconstruction iteratively. This nonlinear transformation enables the model to learn a more disentangled mode of transformation, which is crucial for alleviating facial attribute entanglement. Additionally, we introduce a W+ space age regularization term to prevent excessive manipulation of the style vector and ensure it remains within the W+ space during transformation, thereby improving generation quality and aging accuracy. Extensive qualitative and quantitative experiments demonstrate that the proposed DL‐Aging outperforms state‐of‐the‐art methods regarding aging accuracy, image quality, attribute disentanglement, and identity preservation, especially for large age gaps.

The neuronal signature of surprised facial expression processing under different attentional focuses: A time-domain and time-frequency study.

Previous studies have shown that high-arousal positive and negative facial expressions influence event-related potential (ERP) and time-frequency responses depending on attentional focuses. However, little is known about how relevant neural responses are influenced by surprised facial expressions, which are also high in arousal but ambiguous in valence. To address the issue, 38 participants were presented with surprised, happy, angry and neutral facial expressions. Attention was manipulated to focus on facial emotional attributes, facial non-emotional attributes, non-facial attributes, or was free to the participants. ERP results showed larger N170 responses to surprised compared to neutral facial expressions when attention focused on facial attributes and to surprised compared to angry and neutral facial expressions when attention focused on facial non-emotional attributes. Time-frequency analyses revealed reduced power of early occipital theta to surprised compared to happy and angry expressions when attention focused on facial emotions and to surprised compared to angry expressions when attention focused on non-facial stimuli. Parietal delta power was smaller for surprised facial expressions than for angry facial expressions when attention focused on facial emotions and for surprised facial expressions than for angry and neutral facial expressions when attention was directed to non-facial stimuli. These findings might suggest that neural responses to surprised facial expressions are modulated by attentional focus.

Interactive Generative Adversarial Networks With High-Frequency Compensation for Facial Attribute Editing

Interactive Generative Adversarial Networks With High-Frequency Compensation for Facial Attribute Editing

When houses wear faces: Reverse correlation applied to architectural design

Reverse correlation (RC) is a data-driven method from social psychology that has been effectively shown to visualize the mental representations that humans hold regarding facial attributes. The method helps to understand what features are relevant in terms of the evaluation of faces, such as dominance or submissiveness. To the best of our knowledge, RC has solely been applied to faces within the area of psychology until this point. However, there are many other areas where it is of interest to understand how humans evaluate and visualize content, one of them being the evaluation of house facades. With this work, we extended the application of RC to architectural design, specifically focusing on the evaluation of house facades with respect to the psychological attributes of facelikeness, invitingness, and likeability. Furthermore, we propose a novel approach to create the base image, by utilizing a generative adversarial network. In an online study with a between-subject design, 121 participants completed the RC task, with 40 to 41 participants assigned to each of the three attributes. The resulting classification images (CIs) from the RC task unveil face-related features for the attribute facelikeness, signifying the potential extension of the RC methodology beyond the established domain of facial analysis to other domains, such as architectural design.

Identity-preserving editing of multiple facial attributes by learning global edit directions and local adjustments

Semantic facial attribute editing using pre-trained Generative Adversarial Networks (GANs) has attracted a great deal of attention and effort from researchers in recent years. Due to the high quality of face images generated by StyleGANs, much work has focused on the StyleGANs’ latent space and the proposed methods for facial image editing. Although these methods have achieved satisfying results for manipulating user-intended attributes, they have not fulfilled the goal of preserving the identity, which is an important challenge. We present ID-Style, a new architecture capable of addressing the problem of identity loss during attribute manipulation. The key components of ID-Style include a Learnable Global Direction (LGD) module, which finds a shared and semi-sparse direction for each attribute, and an Instance-Aware Intensity Predictor (IAIP) network, which finetunes the global direction according to the input instance. Furthermore, we introduce two losses during training to enforce the LGD and IAIP to find semi-sparse semantic directions that preserve the identity of the input instance. Despite reducing the size of the network by roughly 95% as compared to similar state-of-the-art works, ID-Style outperforms baselines by 10% and 7% in identity preserving metric (FRS) and average accuracy of manipulation (mACC), respectively.

Face recognition with occluded face using improve intersection over union of region proposal network on Mask region convolutional neural network

Face recognition entails detecting and identifying facial attributes. Mask region convolutional neural network (R-CNN) method is a prominent approach, while prior research predominantly delved into refining loss functions and perfecting object and face detection, recognizing, and identifying faces using imperfect data remained relatively unexplored. This study focuses on an occluded dataset comprising Indonesian faces, wherein 'occluded' denotes facial data that lacks complete visibility-encompassing instances where objects obscure faces or are partially cropped. This investigation involves a deliberate experiment that tailors the intersection over union (IoU) of the region proposal network (RPN) to suit the nuances of occluded Indonesian faces, thereby augmenting accuracy in recognition and segmentation tasks. The innovation IoU in the strategic utilization of Anchors, which involves the exclusion of anchors falling beyond the image borders to optimize computational efficiency. The outcomes of this research are striking; it showcases a remarkable 14.75%, 10.9%, and 12.97% surge based on mean average precision (mAP), mean average recall (mAR), and F1-Scores compared to the conventional Mask R-CNN approach. Notably, our proposed model elevates the average accuracy by 10% to 15% and decreases running time by 21%, a noteworthy enhancement compared to the preceding model. This progress is substantiated by validation utilizing 300 instances dataset, reinforcing the robustness of our approach.

Age and Gender Detection Using Deep Learning

Abstract: Facial attributes are crucial in numerous applications such as access control and video surveillance, where demographic data like age and gender can be inferred from facial images. Automatic estimation of age and gender enables tailored content de- livery and personalized services. However, ex- tracting effective features from facial images poses a significant challenge. This paper pro- poses employing Convolutional Neural Net- works (CNNs) for automatic age and gender prediction. CNNs have demonstrated ground-breaking success in face recognition and im- age classification tasks. Leveraging pre-trained deep CNNs, this research aims to estimate age and gender accurately from facial images. The methodology involves utilizing convolution layers to produce a robust and compact output, enhancing the efficiency of age and gender detection systems.

The impact of the Seller's facial image on consumer purchase behavior in peer-to-peer accommodation platforms

Consumers' initial impressions of a seller's facial appearance play a crucial role in influencing purchase behavior. However, the underlying mechanisms by which a seller's facial appearance influences consumer purchase behavior remain unexplored. This study adopted a multimethod approach to explore these questions. First, drawing on the profile pictures of 1240 sellers from the Airbnb peer-to-peer accommodation platform, we investigated the impact of two principal facial attributes (facial attractiveness and facial expressions) on the formation of consumers' initial impressions and their subsequent influence on purchase decisions. Second, we explored the underlying mechanisms through two online experiments. Our findings indicate a U-shaped relationship between sellers' facial attractiveness and consumer purchase behavior. Additionally, we found that facial expressions such as happy, surprise, and sad, influence consumer purchase behavior positively in comparison to neutral expressions. Furthermore, our study demonstrates that the purchase price acts as a moderator of the U-shaped effect of facial attractiveness on consumer purchase behavior, while text sentiment intensity serves as a negative moderator of the impact of facial expressions of consumer purchase behavior. Finally, the study elucidates the influence mechanism of facial attractiveness and facial expressions on consumers' purchasing decisions. These insights provide novel perspectives on the role of sellers' facial attributes and offer valuable references for devising effective marketing strategies and optimizing platform operational services related to the design of seller facial imagery.

YOLO-FaceNet Fusion: Innovative System for Facial Recognition and Feature Extraction

Facial verification is vital in domains like Heathcare, Identity verification, etc. study presents a sophisticated facial verifying system utilizing state-of-the-art technologies: ArcFace is utilized for the edification of the model, while FaceNet is employed for the elicitation of facial characteristics. Initially, YOLO is deployed to discern and extricate visages from the input effigies. Subsequently, FaceNet is utilized to derive high-caliber facial attributes. In the final stage, ArcFace is employed for the model's edification, thus augmenting its robustness and precision. The apparatus gauges facial resemblance employing the Euclidean distance metric and is appraised through measures such as recall, accuracy, and precision. This approach yields a dependable and accurate facial verification system, thereby enhancing operational efficiency in practical applications.. Keywords – You Only Look Once (YOLO), ArcFace, Convolutional Neural Network (CNN), FaceNet, Machine Learning, Deep Learning, Euclidean Distance.

Digital Planning to Enhance Diagnosis and Precision in Correcting Excessive Gingival Display in the Presence of Asymmetrical Maxillary Position: A Case Report.

This case report addresses the treatment of excessive gingival display (EGD) in the context of maxillary asymmetry, zenith irregularities, and occlusal plane inclination. Digital planning was pivotal in formulating a precise treatment strategy by incorporating facial photographs, digital models, and cone-beam computed tomography data. Parameters, including occlusal plane inclination, teeth position, and lip-to-gingival margin relationships, were considered to ensure treatment alignment with the patient's facial characteristics. Notably, during the planning phase, it was evident that the conventional approach using the cementoenamel junction as the apical limit for incisions would result in asymmetry. Consequently, the gingival margin position was defined in accordance with facial and lip features. The chosen treatment, flapless crown lengthening, was tailored to the patient's thin phenotype and guided by measurements derived from digital planning. Postsurgery, the patient experienced a swift and painless recovery. A harmonious smile with a stable gingival margin position was achieved at the 1-year follow-up, seamlessly complementing the patient's facial attributes. This case underscores the importance of personalized EGD treatment and the value of digital planning in enhancing diagnostic accuracy and precise treatment planning, ultimately facilitating optimal treatment strategies.

Criminal Investigation Tracker with Suspect Prediction

This initiative seeks to craft an inclusive and intuitive web-based platform tailored for criminal identification via image and video surveillance, leveraging cutting-edge facial recognition technology. The system integrates a registration portal for inputting data and images of known criminals, utilizing OpenCV and advanced facial recognition algorithms to securely analyze and store their facial attributes. In addition to allowing users to upload images for analysis, the system offers immediate feedback on potential matches with registered criminals. Moreover, the video surveillance module extends this capability to short videos, employing video analytics to identify faces within the footage. The platform ensures real-time feedback for successful identifications and provides an advanced feature enabling users to download details of identified criminals in an Excel format. By amalgamating state-of-the-art technology with an intuitive interface, this project endeavors to bolster law enforcement endeavors by furnishing an efficient and precise tool for criminal identification and tracking. Its objectives encompass developing a robust system for identifying and tracking criminals through advanced facial recognition algorithms and OpenCV technology, designing a user-friendly web interface for seamless navigation across various modules, and establishing a secure and efficient registration section for compiling comprehensive databases of facial features.

The relevance and accuracy of an AI algorithm-based descriptor on 23 facial attributes in a diverse female US population.

The response of AI in situations that mimic real life scenarios is poorly explored in populations of high diversity. To assess the accuracy and validate the relevance of an automated, algorithm-based analysis geared toward facial attributes devoted to the adornment routines of women. In a cross-sectional study, two diversified groups presenting similar distributions such as age, ancestry, skin phototype, and geographical location was created from the selfie images of 1041 female in a US population. 521 images were analyzed as part of a new training dataset aimed to improve the original algorithm and 520 were aimed to validate the performance of the AI. From a total 23 facial attributes (16 continuous and 7 categorical), all images were analyzed by 24 make-up experts and by the automated descriptor tool. For all facial attributes, the new and the original automated tool both surpassed the grading of the experts on a diverse population of women. For the 16 continuous attributes, the gradings obtained by the new system strongly correlated with the assessment made by make-up experts (r ≥ 0.80; p<0.0001) and supported by a low error rate. For the seven categorical attributes, the overall accuracy of the AI-facial descriptor was improved via enrichment of the training dataset. However, some weaker performance in spotting specific facial attributes were noted. In conclusion, the AI-automatic facial descriptor tool was deemed accurate for analysis of facial attributes for diverse women although some skin complexion, eye color, and hair features required some further finetuning.

FrseGAN: Free‐style editable facial makeup transfer based on GAN combined with transformer

AbstractMakeup in real life varies widely and is personalized, presenting a key challenge in makeup transfer. Most previous makeup transfer techniques divide the face into distinct regions for color transfer, frequently neglecting details like eyeshadow and facial contours. Given the successful advancements of Transformers in various visual tasks, we believe that this technology holds large potential in addressing pose, expression, and occlusion differences. To explore this, we propose novel pipeline which combines well‐designed Convolutional Neural Network with Transformer to leverage the advantages of both networks for high‐quality facial makeup transfer. This enables hierarchical extraction of both local and global facial features, facilitating the encoding of facial attributes into pyramid feature maps. Furthermore, a Low‐Frequency Information Fusion Module is proposed to address the problem of large pose and expression variations which exist between the source and reference faces by extracting makeup features from the reference and adapting them to the source. Experiments demonstrate that our method produces makeup faces that are visually more detailed and realistic, yielding superior results.

Talk-to-Edit: Fine-Grained 2D and 3D Facial Editing via Dialog.

Facial editing is to manipulate the facial attributes of a given face image. Nowadays, with the development of generative models, users can easily generate 2D and 3D facial images with high fidelity and 3D-aware consistency. However, existing works are incapable of delivering a continuous and fine-grained editing mode (e.g., editing a slightly smiling face to a big laughing one) with natural interactions with users. In this work, we propose Talk-to-Edit, an interactive facial editing framework that performs fine-grained attribute manipulation through dialog between the user and the system. Our key insight is to model a continual "semantic field" in the GAN latent space. 1) Unlike previous works that regard the editing as traversing straight lines in the latent space, here the fine-grained editing is formulated as finding a curving trajectory that respects fine-grained attribute landscape on the semantic field. 2) The curvature at each step is location-specific and determined by the input image as well as the users' language requests. 3) To engage the users in a meaningful dialog, our system generates language feedback by considering both the user request and the current state of the semantic field. We demonstrate the effectiveness of our proposed framework on both 2D and 3D-aware generative models. We term the semantic field for the 3D-aware models as "tri-plane" flow, as it corresponds to the changes not only in the color space but also in the density space. We also contribute CelebA-Dialog, a visual-language facial editing dataset to facilitate large-scale study. Specifically, each image has manually annotated fine-grained attribute annotations as well as template-based textual descriptions in natural language. Extensive quantitative and qualitative experiments demonstrate the superiority of our framework in terms of 1) the smoothness of fine-grained editing, 2) the identity/attribute preservation, and 3) the visual photorealism and dialog fluency. Notably, the user study validates that our overall system is consistently favored by around 80% of the participants.

Integrated Facial Attribute Analysis: A Comprehensive Approach for Face Mask Detection, Age Prediction and Gender Classification

Integrated Facial Attribute Analysis: A Comprehensive Approach for Face Mask Detection, Age Prediction and Gender Classification

Access Control Using Facial Recognition

The field of smart home technology has grown by leaps and bounds over the last few decades. This development has made several new products available that can add convenience and security to facility. People became aware of smart home devices with the introduction of the smart thermostat, but these thermostats were only the beginning to smart homes. Smart devices helps people manage their sched-ules, grocery lists, home lighting and even the home security. These devices work together to make life a bit easier as well as safer. One group of such smart home security devices that are gaining popularity are smart door locks. Smart door locks are also seeing new innovative features being added recently where some companies are adding the capability of facial recognition to the locks. With nothing but the face, it’s now possible for a door lock can be operated. As far back as 2016, manufacturers started talk-ing about smart locks with face recognition. The innovation has continued to grow as several main-stream tech devices with facial recognition like the phones and handheld devices with facial recogni-tion unlock features. In smart door locks, the device uses facial recognition, leveraging advanced processing and analytics to identify a person’s facial attributes at the door. KEY WORDS Access control, facial recognition, door locks, security.

Prediction of occupant thermal state via infrared thermography and explainable AI

Accurate and real-time assessment of occupant thermal comfort can provide a solid foundation for efficient air conditioning operations. Existing studies already show the feasibility of using contactless technologies for thermal comfort prediction assisted by machine learning algorithms. However, the lack of transparency in machine learning often weakens user trust. This study performs explainable AI analysis to explore the potential of infrared imaging in thermal comfort evaluation. Specifically, an investigation was carried out in a climatic chamber, and infrared cameras were used to collect facial temperature data. Five popular ensemble tree models were employed to construct prediction models, and explainable AI analysis was performed using SHAP (SHapley Additive exPlanations) theory. Results show that combining additional facial information can significantly improve the overall model performance, and certain facial attributes present high contributions based on SHAP values. Combining facial features with explainable AI provides a convincing basis for thermal comfort assessment. The high SHAP values of facial features can also contribute to finding selective occupants with low neutral voting rates, providing evidence for customized cooling or heating from building systems.

Securing synthetic faces: A GAN-blockchain approach to privacy-enhanced facial recognition

In recent years, facial recognition technology has become increasingly integrated into society, making privacy protection crucial. Previous techniques offered minimal secrecy safeguards through simple obscuration methods. This paper addresses the strict privacy requirements of face image data by developing a novel framework that synergistically integrates Generative Adversarial Networks (GANs), clustering algorithms, and Blockchain technology. The methodology proposes a cutting-edge Privacy-Preserving Self-Attention GAN (PPSA-GAN) to generate realistic synthetic facial imagery. An integrated mini-batch K-means clustering algorithm anonymizes these images into distinct groupings, maximizing privacy preservation. Blockchain integration complements the system by fortifying trust through decentralized ledgers for transparent yet secure data storage and auditing. Rigorous benchmarking on the CelebA dataset confirms the PPSA-GAN architecture’s state-of-the-art performance, attaining an impressive Inception Score of 13.99 and a Fréchet Inception Distance of 35.50. The mini-batch clustering forms 125 distinct clusters, effectively anonymizing facial attributes within the synthetic images. Blockchain integration further bolsters privacy assurances via tamper-proof historical records, showcasing precision, recall, F1-score, and accuracy values of 0.948, 0.938, 0.943, and 0.947, respectively. This multifunctional framework represents a novel contribution, fostering an ethical technological ecosystem that balances progress and privacy. Prospective deployment horizons encompass identity verification, surveillance infrastructure, and augmentation of medical image repositories, seeding an enlightening future for facial recognition domains.

Controllable 3D Face Generation with Conditional Style Code Diffusion

Generating photorealistic 3D faces from given conditions is a challenging task. Existing methods often rely on time-consuming one-by-one optimization approaches, which are not efficient for modeling the same distribution content, e.g., faces. Additionally, an ideal controllable 3D face generation model should consider both facial attributes and expressions. Thus we propose a novel approach called TEx-Face(TExt &amp; Expression-to-Face) that addresses these challenges by dividing the task into three components, i.e., 3D GAN Inversion, Conditional Style Code Diffusion, and 3D Face Decoding. For 3D GAN inversion, we introduce two methods, which aim to enhance the representation of style codes and alleviate 3D inconsistencies. Furthermore, we design a style code denoiser to incorporate multiple conditions into the style code and propose a data augmentation strategy to address the issue of insufficient paired visual-language data. Extensive experiments conducted on FFHQ, CelebA-HQ, and CelebA-Dialog demonstrate the promising performance of our TEx-Face in achieving the efficient and controllable generation of photorealistic 3D faces. The code will be publicly available.

PrefAce: Face-Centric Pretraining with Self-Structure Aware Distillation

Video-based facial analysis is important for autonomous agents to understand human expressions and sentiments. However, limited labeled data is available to learn effective facial representations. This paper proposes a novel self-supervised face-centric pretraining framework, called PrefAce, which learns transferable video facial representation without labels. The self-supervised learning is performed with an effective landmark-guided global-local tube distillation. Meanwhile, a novel instance-wise update FaceFeat Cache is built to enforce more discriminative and diverse representations for downstream tasks. Extensive experiments demonstrate that the proposed framework learns universal instance-aware facial representations with fine-grained landmark details from videos. The point is that it can transfer across various facial analysis tasks, e.g., Facial Attribute Recognition (FAR), Facial Expression Recognition (FER), DeepFake Detection (DFD), and Lip Synchronization (LS). Our framework also outperforms the state-of-the-art on various downstream tasks, even in low data regimes. Code is available at https://github.com/siyuan-h/PrefAce.