Facial Attributes Research Articles

Do inpainting yourself: Generative facial inpainting guided by exemplars

We present EXE-GAN, a novel exemplar-guided facial inpainting framework using generative adversarial networks. Our approach not only preserves the quality of the input facial image but also completes the image with exemplar-like facial attributes. We achieve this by simultaneously leveraging the global style of the input image, the stochastic style generated from the random latent code, and the exemplar style of the exemplar image. We introduce a novel attribute similarity metric to encourage the networks to learn the style of facial attributes from the exemplar in a self-supervised way. To guarantee the natural transition across the boundaries of inpainted regions, we introduce a novel spatial variant gradient backpropagation technique to adjust the loss gradients based on the spatial location. We extensively evaluate EXE-GAN on public CelebA-HQ and FFHQ datasets with practical applications, which demonstrates the superior visual quality of facial inpainting. The source code is available at https://github.com/LonglongaaaGo/EXE-GAN.

Ambiguous facial expression detection for Autism Screening using enhanced YOLOv7-tiny model

Autism spectrum disorder is a developmental condition that affects the social and behavioral abilities of growing children. Early detection of autism spectrum disorder can help children to improve their cognitive abilities and quality of life. The research in the area of autism spectrum disorder reports that it can be detected from cognitive tests and physical activities of children. The present research reports on the detection of autism spectrum disorder from the facial attributes of children. Children with autism spectrum disorder show ambiguous facial expressions which are different from the facial attributes of normal children. To detect autism spectrum disorder from facial images, this work presents an improvised variant of the YOLOv7-tiny model. The presented model is developed by integrating a pyramid of dilated convolutional layers in the feature extraction network of the YOLOv7-tiny model. Further, its recognition abilities are enhanced by incorporating an additional YOLO detection head. The developed model can detect faces with the presence of autism features by drawing bounding boxes and confidence scores. The entire work has been carried out on a self-annotated autism face dataset. The developed model achieved a mAP value of 79.56% which was better than the baseline YOLOv7-tiny and state-of-the-art YOLOv8 Small model.

A Review of Facial Generation Technology Research

Abstract. Face generation, as a cutting-edge generative technology, has made significant strides in various image synthesis tasks, including facial inpainting, text-to-image translation, and video-based facial animation. Generating realistic and diverse human faces is a critical task in computer vision, with a wide array of real-world applications. Given the remarkable success of face generation models, there has been a growing interest in leveraging advanced techniques to further improve the quality, diversity, and control of generated faces. This paper will provide a comprehensive overview of the state-of-the-art face generation techniques. Specifically, the paper reviews the key approaches in this field, including Generative Adversarial Networks (GANs), Vector Quantized methods, and the more recent Diffusion Models, each of which has contributed significantly to the advancement of face generation. The paper discusses how these models have evolved to handle the complexity of face generation, from capturing subtle facial details to enabling fine-grained control over facial attributes. The paper also explores the major applications of face generation technologies, with a particular emphasis on their use in entertainment, virtual reality, and security. Finally, the paper identifies promising directions for future research in face generation, such as improving the interpretability of models, addressing ethical concerns, and enhancing the ability to generate faces that are both highly realistic and diverse.

Disentangled Lifespan Synthesis via Transformer‐Based Nonlinear Regression

AbstractLifespan face age transformation aims to generate facial images that accurately depict an individual's appearance at different age stages. This task is highly challenging due to the need for reasonable changes in facial features while preserving identity characteristics. Existing methods tend to synthesize unsatisfactory results, such as entangled facial attributes and low identity preservation, especially when dealing with large age gaps. Furthermore, over‐manipulating the style vector may deviate it from the latent space and damage image quality. To address these issues, this paper introduces a novel nonlinear regression model‐Disentangled Lifespan face Aging (DL‐Aging) to achieve high‐quality age transformation images. Specifically, we propose an age modulation encoder to extract age‐related multi‐scale facial features as key and value, and use the reconstructed style vector of the image as the query. The multi‐head cross‐attention in the W+ space is utilized to update the query for aging image reconstruction iteratively. This nonlinear transformation enables the model to learn a more disentangled mode of transformation, which is crucial for alleviating facial attribute entanglement. Additionally, we introduce a W+ space age regularization term to prevent excessive manipulation of the style vector and ensure it remains within the W+ space during transformation, thereby improving generation quality and aging accuracy. Extensive qualitative and quantitative experiments demonstrate that the proposed DL‐Aging outperforms state‐of‐the‐art methods regarding aging accuracy, image quality, attribute disentanglement, and identity preservation, especially for large age gaps.

The neuronal signature of surprised facial expression processing under different attentional focuses: A time-domain and time-frequency study.

Previous studies have shown that high-arousal positive and negative facial expressions influence event-related potential (ERP) and time-frequency responses depending on attentional focuses. However, little is known about how relevant neural responses are influenced by surprised facial expressions, which are also high in arousal but ambiguous in valence. To address the issue, 38 participants were presented with surprised, happy, angry and neutral facial expressions. Attention was manipulated to focus on facial emotional attributes, facial non-emotional attributes, non-facial attributes, or was free to the participants. ERP results showed larger N170 responses to surprised compared to neutral facial expressions when attention focused on facial attributes and to surprised compared to angry and neutral facial expressions when attention focused on facial non-emotional attributes. Time-frequency analyses revealed reduced power of early occipital theta to surprised compared to happy and angry expressions when attention focused on facial emotions and to surprised compared to angry expressions when attention focused on non-facial stimuli. Parietal delta power was smaller for surprised facial expressions than for angry facial expressions when attention focused on facial emotions and for surprised facial expressions than for angry and neutral facial expressions when attention was directed to non-facial stimuli. These findings might suggest that neural responses to surprised facial expressions are modulated by attentional focus.

Interactive Generative Adversarial Networks With High-Frequency Compensation for Facial Attribute Editing

Interactive Generative Adversarial Networks With High-Frequency Compensation for Facial Attribute Editing

When houses wear faces: Reverse correlation applied to architectural design

Reverse correlation (RC) is a data-driven method from social psychology that has been effectively shown to visualize the mental representations that humans hold regarding facial attributes. The method helps to understand what features are relevant in terms of the evaluation of faces, such as dominance or submissiveness. To the best of our knowledge, RC has solely been applied to faces within the area of psychology until this point. However, there are many other areas where it is of interest to understand how humans evaluate and visualize content, one of them being the evaluation of house facades. With this work, we extended the application of RC to architectural design, specifically focusing on the evaluation of house facades with respect to the psychological attributes of facelikeness, invitingness, and likeability. Furthermore, we propose a novel approach to create the base image, by utilizing a generative adversarial network. In an online study with a between-subject design, 121 participants completed the RC task, with 40 to 41 participants assigned to each of the three attributes. The resulting classification images (CIs) from the RC task unveil face-related features for the attribute facelikeness, signifying the potential extension of the RC methodology beyond the established domain of facial analysis to other domains, such as architectural design.

Safeguarding Identities with GAN-based Face Anonymization

Effective anonymous facial registration techniques are critical to address privacy concerns arising from facial recognition technology. This study presents an intelligent anonymity platform that incorporates blockchain with advanced privacy and uses a CIAGAN-powered approach. This solution addresses the immediate need for privacy in facial recognition technology. The proposed system uses advanced techniques to anonymously generate highly realistic and effective facial images. The widespread use of facial recognition systems places greater emphasis on privacy concerns, emphasizing the need for strong enrollment mechanisms. The proposed system uses CIAGAN to address this challenge and generate facial images while preserving important attributes. Blockchain storage ensures that data integrity and security are maintained. The process begins with detailed image preprocessing steps to improve data quality and eliminate unwanted noise. CIAGAN can generate anonymous face images with important facial attributes to complicate the recognition of specific objects. A dataset of 202,599 facial images was used. Performance metrics such as PSNR and SSIM indicate image quality and uniformity. The PSNR obtained was 35.0516, indicating a unique image anonymization process.

Identity-preserving editing of multiple facial attributes by learning global edit directions and local adjustments

Semantic facial attribute editing using pre-trained Generative Adversarial Networks (GANs) has attracted a great deal of attention and effort from researchers in recent years. Due to the high quality of face images generated by StyleGANs, much work has focused on the StyleGANs’ latent space and the proposed methods for facial image editing. Although these methods have achieved satisfying results for manipulating user-intended attributes, they have not fulfilled the goal of preserving the identity, which is an important challenge. We present ID-Style, a new architecture capable of addressing the problem of identity loss during attribute manipulation. The key components of ID-Style include a Learnable Global Direction (LGD) module, which finds a shared and semi-sparse direction for each attribute, and an Instance-Aware Intensity Predictor (IAIP) network, which finetunes the global direction according to the input instance. Furthermore, we introduce two losses during training to enforce the LGD and IAIP to find semi-sparse semantic directions that preserve the identity of the input instance. Despite reducing the size of the network by roughly 95% as compared to similar state-of-the-art works, ID-Style outperforms baselines by 10% and 7% in identity preserving metric (FRS) and average accuracy of manipulation (mACC), respectively.

Face recognition with occluded face using improve intersection over union of region proposal network on Mask region convolutional neural network

Face recognition entails detecting and identifying facial attributes. Mask region convolutional neural network (R-CNN) method is a prominent approach, while prior research predominantly delved into refining loss functions and perfecting object and face detection, recognizing, and identifying faces using imperfect data remained relatively unexplored. This study focuses on an occluded dataset comprising Indonesian faces, wherein 'occluded' denotes facial data that lacks complete visibility-encompassing instances where objects obscure faces or are partially cropped. This investigation involves a deliberate experiment that tailors the intersection over union (IoU) of the region proposal network (RPN) to suit the nuances of occluded Indonesian faces, thereby augmenting accuracy in recognition and segmentation tasks. The innovation IoU in the strategic utilization of Anchors, which involves the exclusion of anchors falling beyond the image borders to optimize computational efficiency. The outcomes of this research are striking; it showcases a remarkable 14.75%, 10.9%, and 12.97% surge based on mean average precision (mAP), mean average recall (mAR), and F1-Scores compared to the conventional Mask R-CNN approach. Notably, our proposed model elevates the average accuracy by 10% to 15% and decreases running time by 21%, a noteworthy enhancement compared to the preceding model. This progress is substantiated by validation utilizing 300 instances dataset, reinforcing the robustness of our approach.

Age and Gender Detection Using Deep Learning

Abstract: Facial attributes are crucial in numerous applications such as access control and video surveillance, where demographic data like age and gender can be inferred from facial images. Automatic estimation of age and gender enables tailored content de- livery and personalized services. However, ex- tracting effective features from facial images poses a significant challenge. This paper pro- poses employing Convolutional Neural Net- works (CNNs) for automatic age and gender prediction. CNNs have demonstrated ground-breaking success in face recognition and im- age classification tasks. Leveraging pre-trained deep CNNs, this research aims to estimate age and gender accurately from facial images. The methodology involves utilizing convolution layers to produce a robust and compact output, enhancing the efficiency of age and gender detection systems.

The impact of the Seller's facial image on consumer purchase behavior in peer-to-peer accommodation platforms

Consumers' initial impressions of a seller's facial appearance play a crucial role in influencing purchase behavior. However, the underlying mechanisms by which a seller's facial appearance influences consumer purchase behavior remain unexplored. This study adopted a multimethod approach to explore these questions. First, drawing on the profile pictures of 1240 sellers from the Airbnb peer-to-peer accommodation platform, we investigated the impact of two principal facial attributes (facial attractiveness and facial expressions) on the formation of consumers' initial impressions and their subsequent influence on purchase decisions. Second, we explored the underlying mechanisms through two online experiments. Our findings indicate a U-shaped relationship between sellers' facial attractiveness and consumer purchase behavior. Additionally, we found that facial expressions such as happy, surprise, and sad, influence consumer purchase behavior positively in comparison to neutral expressions. Furthermore, our study demonstrates that the purchase price acts as a moderator of the U-shaped effect of facial attractiveness on consumer purchase behavior, while text sentiment intensity serves as a negative moderator of the impact of facial expressions of consumer purchase behavior. Finally, the study elucidates the influence mechanism of facial attractiveness and facial expressions on consumers' purchasing decisions. These insights provide novel perspectives on the role of sellers' facial attributes and offer valuable references for devising effective marketing strategies and optimizing platform operational services related to the design of seller facial imagery.

YOLO-FaceNet Fusion: Innovative System for Facial Recognition and Feature Extraction

Facial verification is vital in domains like Heathcare, Identity verification, etc. study presents a sophisticated facial verifying system utilizing state-of-the-art technologies: ArcFace is utilized for the edification of the model, while FaceNet is employed for the elicitation of facial characteristics. Initially, YOLO is deployed to discern and extricate visages from the input effigies. Subsequently, FaceNet is utilized to derive high-caliber facial attributes. In the final stage, ArcFace is employed for the model's edification, thus augmenting its robustness and precision. The apparatus gauges facial resemblance employing the Euclidean distance metric and is appraised through measures such as recall, accuracy, and precision. This approach yields a dependable and accurate facial verification system, thereby enhancing operational efficiency in practical applications.. Keywords – You Only Look Once (YOLO), ArcFace, Convolutional Neural Network (CNN), FaceNet, Machine Learning, Deep Learning, Euclidean Distance.

Digital Planning to Enhance Diagnosis and Precision in Correcting Excessive Gingival Display in the Presence of Asymmetrical Maxillary Position: A Case Report.

This case report addresses the treatment of excessive gingival display (EGD) in the context of maxillary asymmetry, zenith irregularities, and occlusal plane inclination. Digital planning was pivotal in formulating a precise treatment strategy by incorporating facial photographs, digital models, and cone-beam computed tomography data. Parameters, including occlusal plane inclination, teeth position, and lip-to-gingival margin relationships, were considered to ensure treatment alignment with the patient's facial characteristics. Notably, during the planning phase, it was evident that the conventional approach using the cementoenamel junction as the apical limit for incisions would result in asymmetry. Consequently, the gingival margin position was defined in accordance with facial and lip features. The chosen treatment, flapless crown lengthening, was tailored to the patient's thin phenotype and guided by measurements derived from digital planning. Postsurgery, the patient experienced a swift and painless recovery. A harmonious smile with a stable gingival margin position was achieved at the 1-year follow-up, seamlessly complementing the patient's facial attributes. This case underscores the importance of personalized EGD treatment and the value of digital planning in enhancing diagnostic accuracy and precise treatment planning, ultimately facilitating optimal treatment strategies.

Criminal Investigation Tracker with Suspect Prediction

This initiative seeks to craft an inclusive and intuitive web-based platform tailored for criminal identification via image and video surveillance, leveraging cutting-edge facial recognition technology. The system integrates a registration portal for inputting data and images of known criminals, utilizing OpenCV and advanced facial recognition algorithms to securely analyze and store their facial attributes. In addition to allowing users to upload images for analysis, the system offers immediate feedback on potential matches with registered criminals. Moreover, the video surveillance module extends this capability to short videos, employing video analytics to identify faces within the footage. The platform ensures real-time feedback for successful identifications and provides an advanced feature enabling users to download details of identified criminals in an Excel format. By amalgamating state-of-the-art technology with an intuitive interface, this project endeavors to bolster law enforcement endeavors by furnishing an efficient and precise tool for criminal identification and tracking. Its objectives encompass developing a robust system for identifying and tracking criminals through advanced facial recognition algorithms and OpenCV technology, designing a user-friendly web interface for seamless navigation across various modules, and establishing a secure and efficient registration section for compiling comprehensive databases of facial features.

The relevance and accuracy of an AI algorithm-based descriptor on 23 facial attributes in a diverse female US population.

The response of AI in situations that mimic real life scenarios is poorly explored in populations of high diversity. To assess the accuracy and validate the relevance of an automated, algorithm-based analysis geared toward facial attributes devoted to the adornment routines of women. In a cross-sectional study, two diversified groups presenting similar distributions such as age, ancestry, skin phototype, and geographical location was created from the selfie images of 1041 female in a US population. 521 images were analyzed as part of a new training dataset aimed to improve the original algorithm and 520 were aimed to validate the performance of the AI. From a total 23 facial attributes (16 continuous and 7 categorical), all images were analyzed by 24 make-up experts and by the automated descriptor tool. For all facial attributes, the new and the original automated tool both surpassed the grading of the experts on a diverse population of women. For the 16 continuous attributes, the gradings obtained by the new system strongly correlated with the assessment made by make-up experts (r ≥ 0.80; p<0.0001) and supported by a low error rate. For the seven categorical attributes, the overall accuracy of the AI-facial descriptor was improved via enrichment of the training dataset. However, some weaker performance in spotting specific facial attributes were noted. In conclusion, the AI-automatic facial descriptor tool was deemed accurate for analysis of facial attributes for diverse women although some skin complexion, eye color, and hair features required some further finetuning.

FrseGAN: Free‐style editable facial makeup transfer based on GAN combined with transformer

AbstractMakeup in real life varies widely and is personalized, presenting a key challenge in makeup transfer. Most previous makeup transfer techniques divide the face into distinct regions for color transfer, frequently neglecting details like eyeshadow and facial contours. Given the successful advancements of Transformers in various visual tasks, we believe that this technology holds large potential in addressing pose, expression, and occlusion differences. To explore this, we propose novel pipeline which combines well‐designed Convolutional Neural Network with Transformer to leverage the advantages of both networks for high‐quality facial makeup transfer. This enables hierarchical extraction of both local and global facial features, facilitating the encoding of facial attributes into pyramid feature maps. Furthermore, a Low‐Frequency Information Fusion Module is proposed to address the problem of large pose and expression variations which exist between the source and reference faces by extracting makeup features from the reference and adapting them to the source. Experiments demonstrate that our method produces makeup faces that are visually more detailed and realistic, yielding superior results.

Talk-to-Edit: Fine-Grained 2D and 3D Facial Editing via Dialog.

Facial editing is to manipulate the facial attributes of a given face image. Nowadays, with the development of generative models, users can easily generate 2D and 3D facial images with high fidelity and 3D-aware consistency. However, existing works are incapable of delivering a continuous and fine-grained editing mode (e.g., editing a slightly smiling face to a big laughing one) with natural interactions with users. In this work, we propose Talk-to-Edit, an interactive facial editing framework that performs fine-grained attribute manipulation through dialog between the user and the system. Our key insight is to model a continual "semantic field" in the GAN latent space. 1) Unlike previous works that regard the editing as traversing straight lines in the latent space, here the fine-grained editing is formulated as finding a curving trajectory that respects fine-grained attribute landscape on the semantic field. 2) The curvature at each step is location-specific and determined by the input image as well as the users' language requests. 3) To engage the users in a meaningful dialog, our system generates language feedback by considering both the user request and the current state of the semantic field. We demonstrate the effectiveness of our proposed framework on both 2D and 3D-aware generative models. We term the semantic field for the 3D-aware models as "tri-plane" flow, as it corresponds to the changes not only in the color space but also in the density space. We also contribute CelebA-Dialog, a visual-language facial editing dataset to facilitate large-scale study. Specifically, each image has manually annotated fine-grained attribute annotations as well as template-based textual descriptions in natural language. Extensive quantitative and qualitative experiments demonstrate the superiority of our framework in terms of 1) the smoothness of fine-grained editing, 2) the identity/attribute preservation, and 3) the visual photorealism and dialog fluency. Notably, the user study validates that our overall system is consistently favored by around 80% of the participants.

Integrated Facial Attribute Analysis: A Comprehensive Approach for Face Mask Detection, Age Prediction and Gender Classification

Integrated Facial Attribute Analysis: A Comprehensive Approach for Face Mask Detection, Age Prediction and Gender Classification

Access Control Using Facial Recognition

The field of smart home technology has grown by leaps and bounds over the last few decades. This development has made several new products available that can add convenience and security to facility. People became aware of smart home devices with the introduction of the smart thermostat, but these thermostats were only the beginning to smart homes. Smart devices helps people manage their sched-ules, grocery lists, home lighting and even the home security. These devices work together to make life a bit easier as well as safer. One group of such smart home security devices that are gaining popularity are smart door locks. Smart door locks are also seeing new innovative features being added recently where some companies are adding the capability of facial recognition to the locks. With nothing but the face, it’s now possible for a door lock can be operated. As far back as 2016, manufacturers started talk-ing about smart locks with face recognition. The innovation has continued to grow as several main-stream tech devices with facial recognition like the phones and handheld devices with facial recogni-tion unlock features. In smart door locks, the device uses facial recognition, leveraging advanced processing and analytics to identify a person’s facial attributes at the door. KEY WORDS Access control, facial recognition, door locks, security.