Facial Appearance Variations Research Articles

Beyond the visible: A survey on cross-spectral face recognition

Cross-spectral face recognition (CFR) refers to recognizing individuals using face images stemming from different spectral bands, such as infrared versus visible. While CFR is inherently more challenging than classical face recognition due to significant variation in facial appearance caused by the modality gap, it is useful in many scenarios including night-vision biometrics and detecting presentation attacks. Recent advances in deep neural networks (DNNs) have resulted in significant improvement in the performance of CFR systems. Given these developments, the contributions of this survey are three-fold. First, we provide an overview of CFR, by formalizing the CFR problem and presenting related applications. Secondly, we discuss the appropriate spectral bands for face recognition and discuss recent CFR methods, placing emphasis on deep neural networks. In particular we describe techniques that have been proposed to extract and compare heterogeneous features emerging from different spectral bands. We also discuss the datasets that have been used for evaluating CFR methods. Finally, we discuss the challenges and future lines of research on this topic.

Age-Invariant Face Recognition Using FaceNet and Multi-task Cascaded Convolutional Networks (MTCNN)

In an era where face recognition technology serves as a cornerstone for security, surveillance, and human-computer interaction, the persistence of age-related variations in facial appearance presents a significant challenge. This research introduces an innovative solution that amalgamates the capabilities of the FaceNet deep learning model with Multi-task Cascaded Convolutional Networks (MTCNN) to achieve precise and robust face recognition across diverse age groups. Leveraging FaceNet's prowess in extracting distinctive features from facial images and mapping them to a high-dimensional feature space for efficient face matching, our system incorporates MTCNN as a pre-processing step to accurately detect and align faces, effectively mitigating age-related changes in facial geometry. Crucially, this novel approach obviates the need for age-specific databases and age group categorization, rendering it a versatile and practical solution for age-invariant face recognition. Rigorous experimentation on benchmark datasets underscores the system's resilience and accuracy across a broad spectrum of age groups, culminating in state-of-the-art results in age-invariant face recognition. The implications of this proposed approach are profound, promising to fortify security measures and enhance user experiences in applications such as access control, personal identification, and customer service, while ensuring dependable and accurate face recognition, irrespective of an individual's age. Key Words: Face recognition, face aging, generative adversarial networks

Coupling RetinaFace and Depth Information to Filter False Positives

Face detection is an important problem in computer vision because it enables a wide range of applications, such as facial recognition and an analysis of human behavior. The problem is challenging because of the large variations in facial appearance across different individuals and lighting and pose conditions. One way to detect faces is to utilize a highly advanced face detection method, such as RetinaFace or YOLOv7, which uses deep learning techniques to achieve high accuracy in various datasets. However, even the best face detectors can produce false positives, which can lead to incorrect or unreliable results. In this paper, we propose a method for reducing false positives in face detection by using information from a depth map. A depth map is a two-dimensional representation of the distance of objects in an image from the camera. By using the depth information, the proposed method is able to better differentiate between true faces and false positives. The method proposed by the authors is tested on a dataset of 549 images, which includes 614 upright frontal faces. The outcomes of the evaluation demonstrate that the method effectively minimizes false positives without compromising the overall detection rate. These findings suggest that incorporating depth information can enhance the accuracy of face detection.

FLEPNet: Feature Level Ensemble Parallel Network for Facial Expression Recognition

With the advent of deep learning, the research on facial expression recognition (FER) has received a lot of interest. Different deep convolutional neural network (DCNN) architectures have been developed for real-time and efficient FER. One of the challenges in FER is obtaining trustworthy features that are strongly associated with facial expression changes. Furthermore, traditional DCNNs for FER problems have two significant issues: insufficient training data, which leads to overfitting, and intra-class facial appearance variations. FLEPNet, a texture-based feature-level ensemble parallel network for FER, is proposed in this study and proved to solve the aforementioned problems. Our parallel network FLEPNet uses multi-scale convolutional and multi-scale residual block-based DCNN as building blocks. First, we consider modified homomorphic filtering to normalize the illumination effectively, which minimizes the intra-class difference. The deep networks are then protected against having insufficient training data by using texture analysis on face expression images to identify multiple attributes. Four texture features are extracted and combined with the image's original characteristics. Finally, the integrated features retrieved by two networks are used to classify seven facial expressions. Experimental results reveal that the proposed technique achieves an average accuracy of 0.9914, 0.9894, 0.9796, 0.8756, and 0.8072 on Japanese Female Facial Expressions, Extended CohnKanade, Karolinska Directed Emotional Faces, Real-world Affective Face Database, and Facial Expression Recognition 2013 databases, respectively. Moreover, experimental outcomes depict significant reliability when compared to competing approaches.

Effect of Prosthodontic Rehabilitation on Perioral Units in Edentulous Patients

Background: Complete edentulous patients experience loss in fundamental mastication functions, the progressive absorption of the alveolar bone, and variation in facial appearance. Complete dentures (CD) fundamentally target edentulous patients and not only increase functionality but also restore the appearance of the face. This study aims to evaluate the effect of rehabilitation with removable CD on perioral wrinkles in complete edentulous patients, in line with growing aesthetic demand.
 Methods: Patients who had CD rehabilitation performed due to total edentulism were photographed before and after rehabilitation from the same angles to evaluate the perioral soft tissue changes. The photographs were scored using verified ranking scales and through evaluation by two observers. To evaluate the repeatability of the scales, measurements were repeated with the same method one month later.
 Results: When the effect of prosthetic rehabilitation administered successfully on perioral aesthetics and lip volume was analyzed, a significant improvement was observed in all other parameters evaluated, except for in the labiomental crease. The improvement in the labiomental crease was not statistically significant. In addition, re-measurement reliability was high (≥0.80) for all four perioral units (nasolabial folds, corners of mouth, labiomental crease and upper radial lip). In addition, all correlations were statistically significant (p

Facial Expression Recognition With Two-Branch Disentangled Generative Adversarial Network

Facial Expression Recognition (FER) is a challenging task in computer vision as features extracted from expressional images are usually entangled with other facial attributes, e.g., poses or appearance variations, which are adverse to FER. To achieve a better FER performance, we propose a model named Two-branch Disentangled Generative Adversarial Network (TDGAN) for discriminative expression representation learning. Different from previous methods, TDGAN learns to disentangle expressional information from other unrelated facial attributes. To this end, we build the framework with two independent branches, which are specific for facial and expressional information processing respectively. Correspondingly, two discriminators are introduced to conduct identity and expression classification. By adversarial learning, TDGAN is able to transfer an expression to a given face. It simultaneously learns a discriminative representation that is disentangled from other facial attributes for each expression image, which is more effective for FER task. In addition, a self-supervised mechanism is proposed to improve representation learning, which enhances the power of disentangling. Quantitative and qualitative results in both in-the-lab and in-the-wild datasets demonstrate that TDGAN is competitive to the state-of-the-art methods.

Emotion Monitoring for Preschool Children Based on Face Recognition and Emotion Recognition Algorithms

In this paper, we study the face recognition and emotion recognition algorithms to monitor the emotions of preschool children. For previous emotion recognition focusing on faces, we propose to obtain more comprehensive information from faces, gestures, and contexts. Using the deep learning approach, we design a more lightweight network structure to reduce the number of parameters and save computational resources. There are not only innovations in applications, but also algorithmic enhancements. And face annotation is performed on the dataset, while a hierarchical sampling method is designed to alleviate the data imbalance phenomenon that exists in the dataset. A new feature descriptor, called “oriented gradient histogram from three orthogonal planes,” is proposed to characterize facial appearance variations. A new efficient geometric feature is also proposed to capture facial contour variations, and the role of audio methods in emotion recognition is explored. Multifeature fusion can be used to optimally combine different features. The experimental results show that the method is very effective compared to other recent methods in dealing with facial expression recognition problems about videos in both laboratory‐controlled environments and outdoor environments. The method performed experiments on expression detection in a facial expression database. The experimental results are compared with data from previous studies and demonstrate the effectiveness of the proposed new method.

Short search space and synthesized-reference re-ranking for face image retrieval

Face image retrieval underpins numerous applications in many computer vision domains, however facial appearance variations including age, gender and race make this task challenging. Prior art methods rely on geometric properties and relationship between local features. However, their performance is still short of what is needed, mainly because (1) they ignore the demographic information, and (2) lack age-invariant re-ranking while retrieving face images. In this paper, we aim to build a two-stage face retrieval approach. First, we search for candidate face images using demographic-assisted clustering resulting in a short search space. Second, we develop a generative model to compensate aging variations between query and candidate face images resulting into an independent aging synthesized face images reference set. We then use this reference set to re-rank candidate face images resulting into the final retrieval of face images. We show that the proposed face retrieval approach outperforms the state-of-the-art methods in terms of both the precision and scalability on publicly available longitudinal datasets including CACD and MORPH II.

Age estimation from faces using deep learning: A comparative analysis

Automatic Age Estimation (AAE) has attracted attention due to the wide variety of possible applications. However, it is a challenging task because of the large variation of facial appearance and several other extrinsic and intrinsic factors. Most of the proposed approaches in the literature use hand-crafted features to encode ageing patterns. Deeply learned features extracted by Convolutional Neural Networks (CNNs) algorithms usually perform better than hand-crafted features. The main contribution of this paper is an extensive comparative analysis of several frameworks for real AAE based on deep learning architectures. Different well-known CNN architectures are considered and their performances are compared. MORPH, FG-NET, FACES, PubFig and CASIA-web Face datasets are used in our experiments. The robustness of the best deep estimator is evaluated under noise, expression changes, “crossing” ethnicity and “crossing” gender. The experimental results demonstrate the high performances of the popular CNNs frameworks against the state-of-art methods of automatic age estimation. A Layer-wise transfer learning evaluation is done to study the optimal number of layers to fine-tune on AAE task. An evaluation framework of Knowledge transfer from face recognition task across AAE is performed. We have made our best-performing CNNs models publicly available that would allow one to duplicate the results and for further research on the use of CNNs for AAE from face images.

Art hodontics: the Search for ‘Ultimate Beauty’ in Philosophy, Classicism and Orthodontics

Abstract: The nature of beauty is one of the most imperishable and controversial themes in Western philosophy, and is a fundamental issue in philosophical aesthetics. Beauty has classically been counted among the ultimate values, with goodness, truth and justice. It is a primary theme among ancient Greek, Hellenistic and Medieval philosophers. The origins of clinical facial aesthetic analysis stem from the theoretical musings and introspection of pioneering artists and sculptors, stretching back to antiquity. The application of such analyses in clinical practice has been modified in accordance with changing cultural perceptions. Nonetheless, the work of the great artists and sculptors of past ages remains inherent in our ability to diagnose variations in facial appearance accurately. CPD/Clinical Relevance: Understanding the philosophical, artistic and clinical basis of facial harmony is essential to diagnose and treat dental patients in general, and orthodontic patients in particular.

Artificial Neural Network Based Ensemble Approach for Multicultural Facial Expressions Analysis

Facial expressions convey exhaustive information about human emotions and the most interactive way of social collaborations, despite differences in ethnicity, culture, and geography. Due to cultural differences, the variations in facial structure, facial appearance, and facial expression representation are the main challenges to the facial expression recognition system. These variations necessitate the need for multicultural facial expression analysis. This study presents several computational algorithms to handle these variations to get high expression recognition accuracy. We propose an artificial neural network-based ensemble classifier for multicultural facial expression analysis. The facial images from the Japanese female facial expression database, Taiwanese facial expression image database, and RadBoud faces database are combined to form a multi-culture facial expression dataset. The participants in the multicultural dataset originate from four ethnic regions including Japanese, Taiwanese, Caucasians, and Moroccans. Local binary pattern, uniform local binary pattern, and principal component analysis are applied for facial feature representation. Experimental results prove that facial expressions are innate and universal across all cultures with minor variations.

Gabor-oriented local order feature-based deep learning for face annotation

Face annotation, a modern research topic in the area of image processing, has useful real-life applications. It is a really difficult task to annotate the correct names of people to the corresponding faces because of the variations in facial appearance. Hence, there still is a need for a robust feature to improve the performance of the face annotation process. In this work, a novel approach called the Deep Gabor-Oriented Local Order Features (DGOLOF) for feature representation has been proposed, which extracts deep texture features from face images. Seven recently proposed face annotation methods are considered to evaluate the proposed deep texture feature under uncontrolled situations like occlusion, expression changes, illumination and pose variations. Experimental results on the LFW, IMFDB, Yahoo and PubFig databases show that the proposed deep texture feature provides efficient results with the Name Semantic Network (NSN)-based face annotation. Moreover, it is observed that the proposed deep texture feature improves the performance of face annotation, regardless of all the challenges involved.

Deep Differentiable Random Forests for Age Estimation.

Age estimation from facial images is typically cast as a label distribution learning or regression problem, since aging is a gradual progress. Its main challenge is the facial feature space w.r.t. ages is inhomogeneous, due to the large variation in facial appearance across different persons of the same age and the non-stationary property of aging. In this paper, we propose two Deep Differentiable Random Forests methods, Deep Label Distribution Learning Forest (DLDLF) and Deep Regression Forest (DRF), for age estimation. Both of them connect split nodes to the top layer of convolutional neural networks (CNNs) and deal with inhomogeneous data by jointly learning input-dependent data partitions at the split nodes and age distributions at the leaf nodes. This joint learning follows an alternating strategy: (1) Fixing the leaf nodes and optimizing the split nodes and the CNN parameters by Back-propagation; (2) Fixing the split nodes and optimizing the leaf nodes by Variational Bounding. Two Deterministic Annealing processes are introduced into the learning of the split and leaf nodes, respectively, to avoid poor local optima and obtain better estimates of tree parameters free of initial values. Experimental results show that DLDLF and DRF achieve state-of-the-art performance on three age estimation datasets.

Two-stages based facial demographic attributes combination for age estimation

Automatic age estimation from face images is a topic of growing interest nowadays, because of its great value in various applications. The main challenge in automatic facial age estimation task comes from the large intra-class facial appearance variations due to both gender and race attributes. To this end, in this paper we propose a complete approach for age estimation based on demographic classification. The proposed approach consists of three main parts: (1) Automatic face detection and alignment to extract only the regions of interest. (2) Feature extraction from facial region images using Multi-level face representation. (3) Two-Stages age Estimation (TSE). The main idea of TSE is to classify the input face image into one of demographic classes, then estimate age within the identified demographic class. The experimental results demonstrate that our proposed approach can offer better performance for age estimation when compared to the state-of-the-art methods on MORPH-II, PAL and a subset of LFW databases.

Age progression by gender-specific 3D aging model

Facial aging is an important problem of face recognition in missing children and automatic template update. As aging is a temporal process, it alters the facial appearance of the individuals. The sources of variations in facial appearance are caused by wrinkles (under eyes, forehead, around lips, and jawline), facial growth (cranial size and skull), and skin tone. The other factors such as health, lifestyle, and gender also impose variations in the aging process. Therefore, predicting facial aging with considering all those factors is a very difficult task. We present our 3D gender-specific aging model which automatically produces simulated images at age y by taking only one input image at age x irrespective of the pose and lighting conditions. The gender-specific aging model is constructed by various datasets (FG-NET, PCSO, Celebrities, BROWNS, Private), and its quality is evaluated with respect to various combinations of the datasets. We further fine-tune the aging model by changing the length of shape and texture eigenvectors and examine how these parameters affect the simulation results. Comparisons of the simulation results with state-of-the-art approaches as well as ground truth images demonstrate the effectiveness of the proposed methods. The subjective and objective evaluations are also carried out which emphasize the potential of our proposed gender-specific 3D aging model.

MPIIGaze: Real-World Dataset and Deep Appearance-Based Gaze Estimation.

Learning-based methods are believed to work well for unconstrained gaze estimation, i.e. gaze estimation from a monocular RGB camera without assumptions regarding user, environment, or camera. However, current gaze datasets were collected under laboratory conditions and methods were not evaluated across multiple datasets. Our work makes three contributions towards addressing these limitations. First, we present the MPIIGaze dataset, which contains 213,659 full face images and corresponding ground-truth gaze positions collected from 15 users during everyday laptop use over several months. An experience sampling approach ensured continuous gaze and head poses and realistic variation in eye appearance and illumination. To facilitate cross-dataset evaluations, 37,667 images were manually annotated with eye corners, mouth corners, and pupil centres. Second, we present an extensive evaluation of state-of-the-art gaze estimation methods on three current datasets, including MPIIGaze. We study key challenges including target gaze range, illumination conditions, and facial appearance variation. We show that image resolution and the use of both eyes affect gaze estimation performance, while head pose and pupil centre information are less informative. Finally, we propose GazeNet, the first deep appearance-based gaze estimation method. GazeNet improves on the state of the art by 22 percent (from a mean error of 13.9 degrees to 10.8 degrees) for the most challenging cross-dataset evaluation.

Costs of reproduction are reflected in women's faces: Post-menopausal women with fewer children are perceived as more attractive, healthier and younger than women with more children.

The costs associated with reproduction (i.e., gestation, lactation, childcare) have long-term negative consequences by elevating risk of disease and reducing lifespan. We tested the hypotheses that high parity, and thus high reproductive costs bear by women, is perceived by other people when they evaluate facial appearance of health, attractiveness and age of mothers. Using computer software we created average facial images based on real photographs of post-menopausal women with varying number of children; 3 parity categories were created (1-2, 4-5, and 7-9 children). Study participants (N = 571) were asked to choose the face they perceived as more attractive, younger and healthier via two-alternative forced choice questions asked in three randomized blocks. Women who had given birth to fewer children were judged both by men and women as more attractive, younger and healthier than women with more children. In each category the lowest scores were received by women from highest parity category (7-9 children). Mechanisms behind the observed variation in facial appearance are not known but higher levels of oxidative stress among women with high parity may explain their faster aging and lower attractiveness in older age. These results suggest that costs of reproduction might affect women's physical appearance.

Portrait Lighting Transfer Using a Mass Transport Approach

Lighting is a critical element of portrait photography. However, good lighting design typically requires complex equipment and significant time and expertise. Our work simplifies this task using a relighting technique that transfers the desired illumination of one portrait onto another. The novelty in our approach to this challenging problem is our formulation of relighting as a mass transport problem. We start from standard color histogram matching that only captures the overall tone of the illumination, and we show how to use the mass-transport formulation to make it dependent on facial geometry. We fit a three-dimensional (3D) morphable face model to the portrait, and for each pixel, we combine the color value with the corresponding 3D position and normal. We then solve a mass-transport problem in this augmented space to generate a color remapping that achieves localized, geometry-aware relighting. Our technique is robust to variations in facial appearance and small errors in face reconstruction. As we demonstrate, this allows our technique to handle a variety of portraits and illumination conditions, including scenarios that are challenging for previous methods.

Portrait Lighting Transfer Using a Mass Transport Approach

Lighting is a critical element of portrait photography. However, good lighting design typically requires complex equipment and significant time and expertise. Our work simplifies this task using a relighting technique that transfers the desired illumination of one portrait onto another. The novelty in our approach to this challenging problem is our formulation of relighting as a mass transport problem. We start from standard color histogram matching that only captures the overall tone of the illumination, and we show how to use the mass-transport formulation to make it dependent on facial geometry. We fit a three-dimensional (3D) morphable face model to the portrait, and for each pixel, we combine the color value with the corresponding 3D position and normal. We then solve a mass-transport problem in this augmented space to generate a color remapping that achieves localized, geometry-aware relighting. Our technique is robust to variations in facial appearance and small errors in face reconstruction. As we demonstrate, this allows our technique to handle a variety of portraits and illumination conditions, including scenarios that are challenging for previous methods.

Portrait lighting transfer using a mass transport approach

Lighting is a critical element of portrait photography. However, good lighting design typically requires complex equipment and significant time and expertise. Our work simplifies this task using a relighting technique that transfers the desired illumination of one portrait onto another. The novelty in our approach to this challenging problem is our formulation of relighting as a mass transport problem. We start from standard color histogram matching that only captures the overall tone of the illumination, and we show how to use the mass-transport formulation to make it dependent on facial geometry. We fit a three-dimensional (3D) morphable face model to the portrait, and for each pixel, we combine the color value with the corresponding 3D position and normal. We then solve a mass-transport problem in this augmented space to generate a color remapping that achieves localized, geometry-aware relighting. Our technique is robust to variations in facial appearance and small errors in face reconstruction. As we demonstrate, this allows our technique to handle a variety of portraits and illumination conditions, including scenarios that are challenging for previous methods.