Facial Components Research Articles

Perception of Facial Esthetics by Laypersons, Dental Assistants, General Dental Practitioners and Dental Specialists

This study sought to assess perception of facial esthetics by laypersons and dental professionals. Three standard photographs were digitally manipulated involving three facial characteristics; facial symmetry, facial profile and facial vertical proportions. A sample of 465 [142 laypersons (LP), 97 dental assistants (DA), 152 general dental practitioners (GDP) and 74 dental specialists (DS)] evaluated the standard and the manipulated images on a visual analogue scale ranging from 1 (the less pleasant image) to 5 (the standard image). The responses were analyzed using Kruskal-Wallis and Mann-Whitney tests. A p value of <0.05 was considered significant. The overall score was 61.29 ± 16.19% with no statistical difference by gender. GDP showed significantly higher perception score than LP, DA and DS did. Regarding the individual facial esthetic components, females showed significantly better perception for "facial profile", while males showed significantly better perception for "facial symmetry" and "facial vertical proportions". Facial symmetry and facial vertical proportions were perceived differently with the higher perception score for the GDP and the lowest perception scores were for LP. Remarkable variations do exist with regard to facial esthetic perception by LP, DA, GDP and DS, with GDP showed more favorable perception. Perception of facial esthetics by laypersons and dental practitioners with a different level of dental education and clinical experience gives an indication about the difference between the patients' desire and the practitioners' perception of such esthetic demands.

Semantic Face Hallucination: Super-Resolving Very Low-Resolution Face Images with Supplementary Attributes.

Given a tiny face image, existing face hallucination methods aim at super-resolving its high-resolution (HR) counterpart by learning a mapping from an exemplary dataset. Since a low-resolution (LR) input patch may correspond to many HR candidate patches, this ambiguity may lead to distorted HR facial details and wrong attributes such as gender reversal and rejuvenation. An LR input contains low-frequency facial components of its HR version while its residual face image, defined as the difference between the HR ground-truth and interpolated LR images, contains the missing high-frequency facial details. We demonstrate that supplementing residual images or feature maps with additional facial attribute information can significantly reduce the ambiguity in face super-resolution. To explore this idea, we develop an attribute-embedded upsampling network, which consists of an upsampling network and a discriminative network. The upsampling network is composed of an autoencoder with skip-connections, which incorporates facial attribute vectors into the residual features of LR inputs at the bottleneck of the autoencoder, and deconvolutional layers used for upsampling. The discriminative network is designed to examine whether super-resolved faces contain the desired attributes or not and then its loss is used for updating the upsampling network. In this manner, we can super-resolve tiny (16×16 pixels) unaligned face images with a large upscaling factor of 8× while reducing the uncertainty of one-to-many mappings remarkably. By conducting extensive evaluations on a large-scale dataset, we demonstrate that our method achieves superior face hallucination results and outperforms the state-of-the-art.

Emotion recognition using facial expression by fusing key points descriptor and texture features

Emotions have a great significance in human-to-human and in human-to-computer communication and interaction. In this paper, an effective and novel approach to recognize the emotions using facial expressions by the fusion of duplex features is proposed. The proposed approach broadly have three phases, phase-I: ROIs extraction, phase-2:Fusion of duplex features and phase-III: Classification. The proposed approach also gives a novel eye center detection algorithm to detect centres of the eyes. The outcome of the algorithm is further contribute to locate and partition the facial components. The hybrid combination of duplex features also gives the importance of fusion of features over individual features. The proposed approach classify the 5 basic emotions i.e. angry, happy, sad, disgust, surprise. The proposed method also raise the issue of high misclassification rate of emotions in higher age groups (>40) and successfully overcomes it. The proposed approach and its outcome evaluation is validated by using four datasets: the dataset created by us including 2500 images of 5 basic emotions (angry, happy, sad, disgust, surprise) having 500 images per emotions, CK+ dataset, MMI dataset and JAFEE dataset. Experimental results shows that the proposed work significantly improves the recognition rate (approx. 97%, 88%, 86%, 93%) and reduces the misclassification rate (approx.1.4%, 7.6%, 6.6%, 2.7%) even for the subjects of higher age group.

Facial semantic descriptors based on information granules

In this paper, we investigate a granular data description for facial components in which a characterization of facial components is presented by a collection of information granules. Firstly, the facial landmark detector is utilized to extract facial components automatically. Secondly, semantic concepts are formed by involving various mechanisms of fuzzy clustering based on these detected landmarks. A collection of numeric prototypes can be sought as a blueprint of the descriptors. Consequently, the information granules are being formed around the prototypes that are engaged by the fundamental ideas of Granular Computing, especially the principle of justifiable granularity. Multiple experiments on Multi-PIE facial database illustrate the proposed facial semantic descriptors based on information granules not only can characterize the key semantics of facial components of data, but also can improve the semantic classification performance in comparison with human perception.

Data Augmentation-Assisted Makeup-Invariant Face Recognition

Recently, face datasets containing celebrities photos with facial makeup are growing at exponential rates, making their recognition very challenging. Existing face recognition methods rely on feature extraction and reference reranking to improve the performance. However face images with facial makeup carry inherent ambiguity due to artificial colors, shading, contouring, and varying skin tones, making recognition task more difficult. The problem becomes more confound as the makeup alters the bilateral size and symmetry of the certain face components such as eyes and lips affecting the distinctiveness of faces. The ambiguity becomes even worse when different days bring different facial makeup for celebrities owing to the context of interpersonal situations and current societal makeup trends. To cope with these artificial effects, we propose to use a deep convolutional neural network (dCNN) using augmented face dataset to extract discriminative features from face images containing synthetic makeup variations. The augmented dataset containing original face images and those with synthetic make up variations allows dCNN to learn face features in a variety of facial makeup. We also evaluate the role of partial and full makeup in face images to improve the recognition performance. The experimental results on two challenging face datasets show that the proposed approach can compete with the state of the art.

Smile intensity recognition in real time videos: fuzzy system approach

Facial emotion is a significant way of understanding or interpreting one’s inner thoughts. Real time video at any instant exhibits the emotion which serves as the input to the emotion recognition system. Many literatures propose different strategies in identifying the emotions by working on different features in the facial components, including geometrical, appearance and motion features. This paper considers the geometrical features as a prime component in deciding the intensity of the smile expressed in the real time videos of the AM-FED (Affectiva-MIT Facial Expression Dataset). Geometrical features considered in the work are the normalized Euclidean distance between the contributing LandMarkPoints (LMPs) of the eyes and the lip portions of the face. Fuzzy logic is applied to the system to effectively classify the intensity of the emotion, i.e., happiness or smile as Maximum, Moderate, Less and neutral. Being a Land mark based assessment, evaluating the normalized values of the Euclidean distance between LMPs for each frame of the video and then mapping the values of all the frames in a range helps the fuzzy decision making stage to relate the mapped values to the smile intensity of each frame. The average recognition rate obtained is 86.54%. The system contributes a less complex but nearly accurate smile intensity recognition model when compared to other computation intensive decision making models, with a practical significance in the customer or client’s mood/satisfactory identification in the online marketing and/or communication of the intensity of the smile of the conversing person to a visually challenged person.

Multi‐stage age estimation using two level fusions of handcrafted and learned features on facial images

Age estimation from facial images is an important application of biometrics. In contrast to other facial variations like occlusions, illumination, misalignment and facial expressions, ageing variation is affected by human genes, environment, lifestyle and health which make age estimation a challenging task. In this study, the authors propose a new age estimation system which exploits multi-stage features from a generic feature extractor, a trained convolutional neural network (CNN), and precisely combined these features with a selection of age-related handcrafted features. This method utilises a decision-level fusion of estimated ages by two different approaches; the first one uses feature-level fusion of different handcrafted local feature descriptors for wrinkle, skin and facial component, while the second one uses score-level fusion of different feature layers of a CNN for its age estimation. Experiments on the publicly available MORPH-Album-2 and FG-NET databases prove the effectiveness of the novel method. Moreover, an additional experimental study on AgeDB database demonstrates that the proposed method is comparable with the best state-of-the-art system for age estimation using in-the-wild age databases.

Face Mask Extraction in Video Sequence

Inspired by the recent development of deep network-based methods in semantic image segmentation, we introduce an end-to-end trainable model for face mask extraction in video sequence. Comparing to landmark-based sparse face shape representation, our method can produce the segmentation masks of individual facial components, which can better reflect their detailed shape variations. By integrating convolutional LSTM (ConvLSTM) algorithm with fully convolutional networks (FCN), our new ConvLSTM-FCN model works on a per-sequence basis and takes advantage of the temporal correlation in video clips. In addition, we also propose a novel loss function, called segmentation loss, to directly optimise the intersection over union (IoU) performances. In practice, to further increase segmentation accuracy, one primary model and two additional models were trained to focus on the face, eyes, and mouth regions, respectively. Our experiment shows the proposed method has achieved a 16.99% relative improvement (from 54.50 to 63.76% mean IoU) over the baseline FCN model on the 300 Videos in the Wild (300VW) dataset.

Fast Landmark Localization With 3D Component Reconstruction and CNN for Cross-Pose Recognition

Two approaches are proposed for cross-pose face recognition, one is built on the handcrafted features extracted from the 3D reconstruction of facial components and the other is built on the learned features from a deep convolutional neural network (CNN). As both approaches rely on facial landmarks for alignment across large poses, we propose the Fast Hierarchical Model (FHM) for locating cross-pose facial landmarks in real time. Unlike most 3D approaches that consider holistic faces, the first proposed approach considers 3D facial components. It segments each 2D face in the gallery into components, reconstructs the 3D surface for each component, and recognizes a query face by component features. The core part of the CNN-based approach is a modified VGG network. We study the performance with different settings on the training set, including the synthesized data from 3D reconstruction, the real-life data from an in-the-wild database, and both types of data combined. The two recognition approaches and the FHM are evaluated in extensive experiments and compared with state-of-the-art methods to demonstrate their efficacy.

Component-based face recognition under transfer learning for forensic applications

In numerous scenarios of forensic applications, only a partial face image is available for recognition purposes. Hence, the systems working on components of the face and partial face images have gained great importance. This paper proposes a novel approach for component-based face recognition and association under transfer learning and demonstrates that the knowledge gained from complete face images is transferred to classify components of the face. Three important components of a face, viz. ears, lips, and nose, are used for association and recognition. These components are unique, stable, and unaltered by the change in poses and expressions. Association is found between the face and nose, face and lips, as well as face and ears. Although, these face components and the face itself are from different domains, they share common information which is utilized to transfer the knowledge gained from one domain to another. Similarly, different types of half-faces are considered: left, right, upper, and lower half and left, right, upper, and lower diagonal for the association between complete and partial face images. Because of the association between a face and its different components, the proposed method can be applied to holistic face recognition, component-based face recognition and partial face recognition.

Multi-view 3D face reconstruction with deep recurrent neural networks

Image-based 3D face reconstruction has great potential in different areas, such as facial recognition, facial analysis, and facial animation. Due to the variations in image quality, single-image-based 3D face reconstruction might not be sufficient to accurately reconstruct a 3D face. To overcome this limitation, multi-view 3D face reconstruction uses multiple images of the same subject and aggregates complementary information for better accuracy. Though appealing, there are multiple challenges in practice. Among these challenges, the most significant is the difficulty to establish coherent and accurate correspondence among a set of images, especially when these images are captured under unconstrained in-the-wild condition. This work proposes a method, Deep Recurrent 3D FAce Reconstruction (DRFAR), to solve the task of multi-view 3D face reconstruction using a subspace representation of the 3D facial shape and a deep recurrent neural network that consists of both a deep convolutional neural network (DCNN) and a recurrent neural network (RNN). The DCNN disentangles the facial identity and the facial expression components for each single image independently, while the RNN fuses identity-related features from the DCNN and aggregates the identity specific contextual information, or the identity signal, from the whole set of images to estimate the facial identity parameter, which is robust to variations in image quality and is consistent over the whole set of images. Experimental results indicate significant improvement over state-of-the-art in both the accuracy and the consistency of 3D face reconstruction. Moreover, face recognition results on IJB-A with the UR2D face recognition pipeline indicate that, compared to single-view 3D face reconstruction, the proposed multi-view 3D face reconstruction algorithm can improve the face identification accuracy of UR2D by two percentage points in Rank-1 identification rate.

A discriminative dynamic framework for facial expression recognition in video sequences

Facial expression involves a dynamic process, leading to the variation of different facial components over time. Thus, dynamic descriptors are essential for recognising facial expressions. In this paper, we extend the spatial pyramid histogram of gradients to spatio-temporal domain to give 3-dimensional facial features. To enhance the spatial information, we divide the whole face region into a group of smaller local regions to extract local 3D features, and a weighting strategy based on fisher separation criterion is proposed to enhance the discrimination ability of local features. A multi-class classifier based on support vector machine is applied for recognising facial expressions. Experiments on the CK+ and MMI datasets using leave-one-out cross validation scheme show that the proposed framework perform better than using the descriptor of simple concatenation. Compared with state-of-the-art methods, the proposed framework demonstrates a superior performance.

Facial Emotion Recognition System Based on PCA and Gradient Features

An efficient framework is proposed to deal with the facial emotions recognition problem. Since facial expressions result from facial muscle deformations, gradient features are exceptionally sensitive to the object deformations, so apply the gradients to encode these facial components as features. Then further it is joined by the testing process that classifies emotions and results are measured in terms of false acceptance rate, false rejection rate, and recognition accuracy. Proposed system was trained using random forest classifier to recognize the facial emotions. Japanese Female Facial Emotion (JAFFE) database consist of 5 typical emotions, namely, sad, happy, angry, neutral and surprise is considered for experimental results. Proposed framework can be used in real life applications like electroencephalogram in collaboration with brain computer interfaces. The average classification rate on the JAFEE dataset reaches 91.3%. In the proposed system hybridization of Gradient filter, PCA and PSO has been done for facial emotion recognition which has never been used earlier and this hybridization produces performs better than already existing techniques. Experimental results demonstrate the competitive classification accuracy of our proposed method.

Exploring the morphological and emotional correlates of infant cuteness

Ethologists have observed that “baby schema” or infant cuteness is an adaptive protective mechanism ensuring the young’s survival. Past efforts to quantify cuteness have been restricted to line measurement techniques. We developed a novel data-driven approach to quantify infant cuteness into a single metric. Using the Psychomorph program, we delineated facial elements of 72 infant pictures using 206 facial points and identified the facial components that were significantly related to subjective cuteness perceptions of the faces. 108 nulliparous females rated the pictures on cuteness and emotional dimensions. We found that cuter infants have larger cephalic curvature compared to a smaller chin, a big smile, and round chubby features among others. We also investigated the relationship between cuteness and emotional responses. Our results show that a greater degree of cuteness elicits both increased positive emotional responses and decreased negative emotional responses. Cuter infants also elicited greater feelings of alertness, interest, and the need to respond. In fact, the participants’ emotional responses were predictive of both data-driven scores and subjective perceptions of cuteness.

Contactless blood pressure sensing using facial visible and thermal images

Hypertension is one of the leading risk factors for several diseases. Measurement and monitoring of blood pressure anytime and anywhere are important to lower blood pressure and prevent pathogenesis of diseases. Non-contact blood pressure measurement is desired to monitor blood pressure anytime and anywhere. The aim of this study was to develop a non-contact blood pressure sensing system. A previous study reported that amplitude and time differences of facial photoplethysmogram (PPG) components extracted using brightness variation of facial skin color in facial visible images could be useful indices for estimating blood pressure. The maximum error between measured and estimated blood pressure using facial PPG components was 12 mmHg. An additional signal processing algorithm is desired to increase the accuracy for estimating blood pressure using facial PPG components. By contrast, facial skin temperature also reflects changes in the facial blood circulation. High-accuracy estimation of blood pressure could be expected using both facial PPG components and facial skin temperature. In this study, improvement of accuracy for estimating blood pressure using facial PPG components by attempting to apply additional signal processing to facial skin color variation. Furthermore, a correlation analysis between facial skin temperature and measured blood pressure was performed, and individual models for blood pressure estimation were created.

Crystal Formation in the Facies of Oral Fluid Their Cycles of Hydration and Degradation

Background. Analysis of photographs of crystalline structures formed during drying of biological fluids and in particular oral liquid, is widely used as a diagnostic method for assessing the state of the human body. The urgent task is the development of automated methods of numerical morphometry with the help of computer programs for analyzing the differences in the shape, size, texture, color and number of crystalline structures to exclude subjective contributions and dependence of analysis on the qualifications of the researcher.Objective. Study of processes of dehydration structurization of oral liquid and direct formation of crystalline structures in conditions of application of repeated cycles of hydration and dehydration. They studied their influence on the formation of the crystalline structure, o the quality of its reproduction on digital photos of the facies.Methods. Oral liquor samples were given to groups of volunteers aged 19–23, men and women. From the samples after their preliminary mixing and degassing to a homogeneous state, drops of the examined oral liquid were dosed and microdasitors were applied to the horizontal objective glass. There were cycles of partial hydration–dehydration.Results. The processes of repeated dissolution of the solid phase of the dried drop of the oral liquid (facies), the conditions of osmotic, diffusion, extraction displacement of crystalline substances and their structuring after dissolution of a drop of a new liquid and dehydration by drying are considered. It was found that under the influence of cycles of hydration-dehydration, crystalline substances of the oral liquid continue to preserve their crystalline properties in the derivative facies. The obtained results of the study indicate that the cycles of partial hydration - dehydration of the facials of the oral liquid – are achieved by filtration (purification) of the release of a part of the dissolved facies from foreign impurities, and the clarity of the image of the crystalline structure.Conclusions. It has been shown that the crystal-forming components of the facial fluid of the mouth retain their crystalline properties and the ability to recrystallize after drying and are capable of restoring a balanced state under the influence of solvent hydration. Hydration by a solvent of a particle of a solid facial plane restores the dissolution and extraction of the facial components in the drop field of the solvent, and the subsequent dehydration by drying forms the updated crystallization texture of the derivative of the facies; while the optically active impurities and interferences are filtered, and digital images of the crystallized structure of the derivative of the facial fluid of the mouth are formed by purified and gaining a high degree of clarity. The offered algorithms, formal procedures and means of obtaining an electronic image of crystallized textures of derivative facies are offered.

Smile prevails over other facial components of male facial esthetics

BackgroundThe aim of this study was to assess whether scores assigned to the eyes, nose, mouth, and chin regions work as predictors of full smiling face scores. MethodsIn this cross-sectional study, the authors used the facial photographs of 86 smiling men. Photographs yielded 5 components: 1 of the face itself and 4 subcomponents (eyes, nose, mouth, and chin region). Raters assigned the photographs beauty scores that the authors measured morphometrically. The authors analyzed the predictive ability of the subcomponents against that of the full face. ResultsThe subcomponents were statistically significant predictors of facial beauty (mouth: r2 = 0.38, P < .0001; eyes: r2 = 0.14, P < .0001; chin region: r2 = 0.09, P < .0001; nose: r2 = 0.02, P = .05). The more beautiful people had several statistically significant characteristics, such as narrower faces. ConclusionsFacial subcomponents are predictive factors of the male smiling face and contribute in the following descending order of importance: mouth, eyes, chin region, and nose. Practical ImplicationsThe results suggest that for many people improvement in smile esthetics also likely will exert a more positive effect on facial beauty than will other procedures (for example, rhinoplasty).

Menuju Pengenalan Ekspresi Mikro: Pendeteksian Komponen Wajah Menggunakan Discriminative Response Map Fitting

The observations made in the study of micro-expression are to recognize and track the very subtle movements of certain facial areas and in a short time. In this study, the observation of movement is held in some areas of the face component. The facial and facial components detection is the pre-process stage on micro-expression recognition system. The goal at this stage is to get face and face components accurately and quickly on every movement of the video sequence or image sequence. The face landmark point of the Discriminative Response Map Fitting (DRMF) method can be used to get face components area accurately and quickly. This can be done because the facial landmark points used in this model-based method do not change when objects are moved, rotated, or scaled. The results obtained by using this method are accurate with a 100% accuracy value compared to the Haar Cascade Classifier method with an average accuracy of 44%. In addition, the average time required in the formation of facial component boxes for each frame is 0.08 seconds, faster than the Haar Cascade Classifier method of 0.32 seconds. With the results obtained, then the detection of facial components can be obtained accurately and quickly. Furthermore, the boxes of face components obtained are expected to display the appropriate data to be processed correctly and accurately in the next stage, feature extraction and the classification of micro-expression motion stage.

Age-Group Estimation Using Feature and Decision Level Fusion

Age-group estimation is a technique of classifying an individual's face image as belonging to a particular range of ages. Stochastic nature of ageing among individuals makes age-group estimation based on face images a challenging task. Faces from different age-groups may have similar attributes making facial age-group estimation even harder. Facial ageing is manifested as change in shape in lower age-groups and variations in texture in adult to old age-groups. As one ages, wrinkles start appearing in regions like forehead, eye-corners, mouth region and cheek bone areas among others. The general appearance of the face also varies across one's lifetime from childhood to old age. Approaching age-group estimation as a multi-class classification problem, we adopt Linear Discriminant Analysis (LDA) for representation of global face appearance, landmark points and distance ratios between fiducial landmarks for shape representation, Local Binary Patterns (LBP) for local region texture description and Gabor filters for local region wrinkle representation. We fuse global shape and appearance features; local wrinkle and texture features for facial component before applying an ensemble of machine learning tools that comprise of Artificial Neural Network (ANN) and Support Vector Machines (SVM) for age-group estimation. Using majority voting scheme, we fuse decisions from global and component-based matchers to get final age-group label. This approach was experimented on four age-groups derived from Face and Gesture Network (FG-NET) ageing dataset. Experimental results show that feature and decision fusion improve age-group estimation accuracies.

Facial Landmark Detection: A Literature Survey

The locations of the fiducial facial landmark points around facial components and facial contour capture the rigid and non-rigid facial deformations due to head movements and facial expressions. They are hence important for various facial analysis tasks. Many facial landmark detection algorithms have been developed to automatically detect those key points over the years, and in this paper, we perform an extensive review of them. We classify the facial landmark detection algorithms into three major categories: holistic methods, Constrained Local Model (CLM) methods, and the regression-based methods. They differ in the ways to utilize the facial appearance and shape information. The holistic methods explicitly build models to represent the global facial appearance and shape information. The CLMs explicitly leverage the global shape model but build the local appearance models. The regression-based methods implicitly capture facial shape and appearance information. For algorithms within each category, we discuss their underlying theories as well as their differences. We also compare their performances on both controlled and in the wild benchmark datasets, under varying facial expressions, head poses, and occlusion. Based on the evaluations, we point out their respective strengths and weaknesses. There is also a separate section to review the latest deep learning-based algorithms. The survey also includes a listing of the benchmark databases and existing software. Finally, we identify future research directions, including combining methods in different categories to leverage their respective strengths to solve landmark detection "in-the-wild".