Facial Subregions Research Articles

Searching for Signs of Plastic Surgery on the Face: Tracking the Eyes of Where Observers Look.

Background: Objective measurement of where observers direct their attention to faces when searching for signs of facial plastic surgery (FPS) is currently lacking. Objective: To compare where laypersons direct their attention on facial photographs using eye-tracking software when they are asked to (1) search for signs of aesthetic facial surgery or (2) allowed to gaze without direction (free-gaze). Methods: Naïve observers either free-gazed or examined faces for signs of FPS (FPS-prompted) for 10 s per face while their gaze was recorded by an eye-tracking system. Faces had no known history or signs/stigmata of FPS and were selected from the FACES and CFD databases with a diverse demographic distribution. Gaze times in nine facial subregions were analyzed using mixed-effects linear regression. Results: In FPS-prompted observers (n = 50, mean age 32.7 ± 11.3 years, 23/50 (46%) female), the nose, mouth, cheeks, and forehead experienced the most substantial increases (p < 0.001) and a high percentage of overall gaze time (17.9%, 12.5%, 12.0%, 9.6%, respectively) compared to free-gazing observers [n = 57, 35.5 ± 13.9 years, 31/57 (54%) female]. Conclusions: Observers direct attention differently on a face when searching for signs of plastic surgery with increased attention on the nose, mouth, cheeks, and forehead.

The 3-Level Approach to Facelift Planning: A Simple Paradigm for Surgical Rejuvenation of the Face.

Our role as aesthetic surgeons demands individualized surgical planning that maximizes patient input and understanding. The value of such shared decision-making (SDM) in aesthetic surgery is becoming increasingly appreciated. This is particularly true for potential patients seeking surgical rejuvenation of the face, where the volume of "educational" information available on the internet, and through various social medial channels, may be overwhelming and even misleading. Presented is a "3-Level approach to facelift planning" named for the facial subregions targeted. This novel paradigm maximizes SDM with its simplicity and reproducibility, serves as an invaluable educational tool for patients, novice and seasoned surgeons alike, and facilitates communication between senior surgeons through its descriptive standardization. LEVEL OF EVIDENCE V: This journal requires that authors assign a level of evidence to each article. For a full description of these Evidence-Based Medicine ratings, please refer to the Table of Contents or the online Instructions to Authors www.springer.com/00266 .

Estimating the Change in Facial Subunits During Positive and Negative Facial Expression Using Three-Dimensional Stereophotogrammetry Facial Analysis.

Background: A more refined and clinically related facial expression analysis is required for patients who wish to be perceived more emotionally positive. Objective: To measure the change in skin vector and volume in facial subunits when expressing positive expression (happiness) compared with negative expressions (sadness, fear, disgust, and anger), using three-dimensional (3D) stereophotogrammetry analysis. Methods: This study took 3D photographs of 20 volunteers' face at rest and during positive and negative expression. The directions of skin vector and volume changes in each facial subregion were recorded and calculated. Results: In the positive expression, 78.3% (95% confidence interval [CI] 66.8-89.9) of the medial midfacial subregions presented superolateral vector and volume increase, whereas volume decrease in 82.5% (95% CI 78.5-86.5) of the lip subregions could be observed. In the negative expression, the vector changes were predominantly inferomedial in 26.0% (95% CI 15.4-36.5) of the forehead and 36.8% (95% CI 33.2-40.3) of the upper eyelid subregions, whereas volume increases in 34.0% (95% CI 30.4-37.7) of the upper eyelid subregions were observed. Conclusions: This 3D stereophotogrammetry analysis presents the morphological difference between the positive and negative expression.

Spatial averaging method based on adaptive weight for imaging photoplethysmography.

Imaging photoplethysmography (iPPG) is a non-contact measuring technology for several physiological parameters reflecting personal health status without a special sensor. However, the pulse signal obtained using the iPPG usually is contaminated by various noises, and the intensity of the interesting pulse signal is relatively weak compared to the noises, emphasizing the necessity of obtaining high-quality pulse signals to measure physiological parameters correctly. Various regions of the face harbor distinct pulse information. We propose a spatial averaging method based on adaptive weights, which can obtain high-quality pulse signals by applying different weights to facial sub-regions of interest (sub-ROIs; sROIs). First, the facial ROI is divided into seven sROIs and the coarse heart rate (HR) is calculated from them. Next, the signal-to-noise ratio (SNR) of the raw signal obtained from each sROI is calculated using the coarse HR, and then a high-quality pulse signal is obtained by assigning positive or negative weights to each sROI based on the SNRs. We compare our method with others through the quality analysis of the obtained pulse signals using the self-collected database and the public database PURE. The comparison results show that the proposed method can provide a better pulse signal compared to other methods under various resolutions and states. This proposed method can obtain the pulse signal with better quality, which is helpful to accurately measure physiological parameters, such as HR and HR variability.

Analysis of Facial Occlusion Challenge in Thermal Images for Human Affective State Recognition.

Several studies have been conducted using both visual and thermal facial images to identify human affective states. Despite the advantages of thermal facial images in recognizing spontaneous human affects, few studies have focused on facial occlusion challenges in thermal images, particularly eyeglasses and facial hair occlusion. As a result, three classification models are proposed in this paper to address the problem of thermal occlusion in facial images, with six basic spontaneous emotions being classified. The first proposed model in this paper is based on six main facial regions, including the forehead, tip of the nose, cheeks, mouth, and chin. The second model deconstructs the six main facial regions into multiple subregions to investigate the efficacy of subregions in recognizing the human affective state. The third proposed model in this paper uses selected facial subregions, free of eyeglasses and facial hair (beard, mustaches). Nine statistical features on apex and onset thermal images are implemented. Furthermore, four feature selection techniques with two classification algorithms are proposed for a further investigation. According to the comparative analysis presented in this paper, the results obtained from the three proposed modalities were promising and comparable to those of other studies.

Real-world human gender classification from oral region using convolutional neural netwrok

Gender classification is an important biometric task. It has been widely studied in the literature. Face modality is the most studied aspect of human-gender classification. Moreover, the task has also been investigated in terms of different face components such as irises, ears, and the periocular region. In this paper, we aim to investigate gender classification based on the oral region. In the proposed approach, we adopt a convolutional neural network. For experimentation, we extracted the region of interest using the RetinaFace algorithm from the FFHQ faces dataset. We achieved acceptable results, surpassing those that use the mouth as a modality or facial sub-region in geometric approaches. The obtained results also proclaim the importance of the oral region as a facial part lost in the Covid-19 context when people wear facial mask. We suppose that the adaptation of existing facial data analysis solutions from the whole face is indispensable to keep-up their robustness.

Deep3DCANN: A Deep 3DCNN-ANN framework for spontaneous micro-expression recognition

Facial micro-expressions play a significant role in revealing concealed emotions. However, the recognition of micro-expressions is challenging due to their fleeting nature. Moreover, the visual features of the face and the visual relationships between the facial sub-regions have a strong influence on the presence of micro-expressions. In this work, a novel end-to-end facial micro-expression detection framework, called Deep3DCANN, is proposed to integrate these components for effective micro-expression detection. The first component of our framework is a deep 3D convolutional neural network that learns useful spatiotemporal features from a sequence of facial images. In the second component, a deep artificial neural network is utilized to trace the useful visual associations between different sub-regions of the face. Furthermore, a carefully crafted fusion mechanism is built to combine the learned facial features and the semantic relationships between the regions to predict the micro-expressions. We also construct a new loss function to jointly optimize both modules of our proposed architecture. Our proposed method performs favourably on five benchmark spontaneous micro-expression databases compared to existing micro-expression recognition baselines on videos. In addition, through an extended experiment, we show that our proposed approach can effectively recognize the frame-wise micro-expression changes in a sequence of video frames.

How We Look At Mature Faces-An Eye-Tracking Investigation Into the Perception of Age.

It is still unclear which facial region contributes most to the perception of an aged face when evaluated by eye-tracking analyses. The authors sought to apply eye-tracking technology to identify whether mature faces require longer fixation durations than young faces and which facial region contributes most to the perception of a mature face. Eye-tracking analyses were conducted in 74 volunteers (37 males, 37 females; 43 ≤ 40 years, 31 > 40 years) evaluating their gaze pattern and the fixation durations for the entire face and 9 facial subregions. Frontal facial images of 16 younger (<40 years) and older (>40 years) gender-matched individuals were presented in a standardized setting. Independent of age or gender of the observer, a younger stimulus image was viewed shorter than an older stimulus image with 0.82 (0.63) seconds vs 1.06 (0.73) seconds with P < 0.001. There was no statistically significant difference in their duration of a stable eye fixation when observers inspected a male vs a female stimulus image [0.94 (0.70) seconds vs 0.94 (0.68) seconds; P = 0.657] independent of the observer's age or gender. The facial image that captured the most attention of the observer (rank 9) was the perioral region with 1.61 (0.73) seconds for younger observers and 1.57 (0.73) seconds for older observers. It was revealed that the perioral region attracts the most attention of observers and contributes most to an aged facial appearance. Practitioners should be mindful of the importance of the perioral region when designing an aesthetic treatment plan.

A Geometric Morphometric Study on Sexual Dimorphism in Viscerocranium.

Simple SummarySex estimation is a crucial step in the identification of unknown bone remains. The accuracy of sex estimation methods depends on the level of sexual dimorphism manifested by the human bones. Therefore, the evaluation of sex differences of particular bones of the skeleton is an important preceding stage. We have used geometric morphometric techniques to investigate the sexual dimorphism in the size and shape of the facial part of the skull and its subregions. Our results show that the facial skeleton in males and females differs more in size than in shape, so its overall size is a more useful sex indicator than its shape. The same result is observed for all facial subregions studied here. However, the best discrimination between the male and female skulls is achieved when both size and shape are considered together.The level of sexual dimorphism manifested by human bones is an important factor for development of effective sex estimation methods. The aim of the study was to investigate the sexual dimorphism in the size and shape of the viscerocranium using geometric morphometric techniques. It also aimed to explore the sex differences in distinct viscerocranial regions and to establish the most dimorphic region with regard to size and shape. Computed tomography images of 156 males and 184 females were used in the study. Three-dimensional coordinates of 31 landmarks were acquired. Five landmark configurations were constructed from the viscerocranium and its orbital, nasal, maxillary, and zygomatic region. Generalized Procrustes superimposition, principal component analysis, and discriminant analysis were applied to each configuration. The significance of the sex differences in size and shape was assessed and significant differences were found in all configurations. The highest accuracy was obtained from both shape and size of the whole viscerocranium. Based on size only, the highest accuracy was achieved by the nasal region. The accuracy based on shape was generally low for all configurations, but the highest result was attained by the orbital region. Hence, size is a better sex discriminator than shape.

Epidemiological survey of the quantity and anatomical position of facial pigmented spots in relation to age in 454 Japanese women.

Facial pigmented spots are one of the phenotypes of skin aging, but no quantitative analysis of spot features such as color intensity, size, anatomical position, and number on the cheek has yet been performed. In the current study, we conducted an epidemiological survey of 454 Japanese women in their 20s to 70s and analyzed age-related changes and site differences of facial pigmented spots. Using image analysis of high-resolution digital facial photographs, 4912 individual pigmented spots were quantified according to color, size, anatomical position, and total number on the cheek. As a result of color analysis, the color intensity of individual pigmented spots increased with aging, significantly so between ages 30s and 50s. The age-related increase in melanin index of facial spots was confirmed in all sites but did not significantly differ between facial subregions. Regarding the size of pigmented spots, the frequency of large spots increased with age, and large spots were detected in all facial sites. The total number of pigmented spots on the entire cheek increased with aging, significantly so between the 20s and 40s. The number of pigmented spots tended to increase from the region near the canthi to the lower cheeks. The number of spots was markedly increased in the buccal regions compared with the infraorbital and zygomatic regions. The data and methodology presented in the current study can link the state of facial pigmentation with the various factors involved in the histological development of pigmented spots, opening new possibilities in the fields of skin pharmacology and dermatology.

Detecting Pulse Wave From Unstable Facial Videos Recorded From Consumer-Level Cameras: A Disturbance-Adaptive Orthogonal Matching Pursuit.

Modern consumer-level cameras can detect subtle changes in human facial skin color due to varying blood flow; they are beginning to be used as noncontact devices to detect pulse waves. Little, however, do we know about their capacity to perform pulse wave detection when the recorded faces are unstable. Here, we propose a novel method that can extract pulse waves from videos with drastic facial unsteadiness such as head twists and alternating expressions. The method first uses chrominance characteristics in multiple facial sub-regions to construct a raw pulse matrix. Subsequently, it employs a disturbance-adaptive orthogonal matching pursuit (DAOMP) algorithm to recover the underlying pulse matrix corrupted by facial unsteadiness. To evaluate the efficacy of the method, we perform analyses on two datasets including 268 samples from 67 testing subjects. The results demonstrate that the proposed method outperforms state-of-the-art algorithms, especially in the terrain where drastic facial unsteadiness is present. The proposed framework shows promise to achieve videos-based noncontact pulse wave detection from both steady and unsteady faces recorded by consumer-level cameras. By employing the proposed method, disturbance robustness in noncontact pulse wave detection can be significantly improved.

Recognition of partially occluded faces using regularized ICA

Face recognition approaches that use subspace projection are heavily related to basis images, especially in the case of partial occlusion. To improve the recognition performance, the occlusion should be excluded from the test image during the recognition process. In terms of similarity with image reconstruction, the proposed approach aims at representing the whole face image based on facial subregion. In this respect, face representation can be considered as an inverse problem. The Tikhonov regularization approach is combined with independent component analysis (ICA) in order to obtain the image parameter from the occluded image, where this parameter is compared with that trained by ICA. The combined algorithm is named as RegICA and is performed on face images in the AR Face Database. Cumulative match characteristics was taken as a measure for evaluating the performance of RegICA with occlusion. Compared with ICA on facial occlusion problem, it was found that the proposed approach outperforms ICA. In addition, it has the ability to recognize faces using any facial subregion even if it is small. Furthermore, it is shown that RegICA is not time-consuming, and it outperforms some of the recent approaches in terms of accuracy.

Association of Known Melanoma Risk Factors with Primary Melanoma of the Scalp and Neck

Scalp and neck (SN) melanoma confers a worse prognosis than melanoma of other sites but little is known about its determinants. We aimed to identify associations between SN melanoma and known risk genes, phenotypic traits, and sun exposure patterns. Participants were cases from the Western Australian Melanoma Health Study (n = 1,200) and the Genes, Environment, and Melanoma Study (n = 3,280). Associations between risk factors and SN melanoma, compared with truncal and arm/leg melanoma, were investigated using binomial logistic regression. Facial melanoma was also compared with the trunk and extremities, to evaluate whether associations were subregion specific, or reflective of the whole head/neck region. Compared with other sites, increased odds of SN and facial melanoma were observed in older individuals [SN: OR = 1.28, 95% confidence interval (CI) = 0.92-1.80, P trend = 0.016; Face: OR = 4.57, 95% CI = 3.34-6.35, P trend < 0.001] and those carrying IRF4-rs12203592*T (SN: OR = 1.35, 95% CI = 1.12-1.63, P trend = 0.002; Face: OR = 1.29, 95% CI = 1.10-1.50, P trend = 0.001). Decreased odds were observed for females (SN: OR = 0.49, 95% CI = 0.37-0.64, P < 0.001; Face: OR = 0.66, 95% CI = 0.53-0.82, P < 0.001) and the presence of nevi (SN: OR = 0.66, 95% CI = 0.49-0.89, P = 0.006; Face: OR = 0.65, 95% CI = 0.52-0.83, P < 0.001). Differences observed between SN melanoma and other sites were also observed for facial melanoma. Factors previously associated with the broader head and neck region, notably older age, may be driven by the facial subregion. A novel finding was the association of IRF4-rs12203592 with both SN and facial melanoma. Understanding the epidemiology of site-specific melanoma will enable tailored strategies for risk factor reduction and site-specific screening campaigns.

Content-Aware Eye Tracking for Autostereoscopic 3D Display.

This study develops an eye tracking method for autostereoscopic three-dimensional (3D) display systems for use in various environments. The eye tracking-based autostereoscopic 3D display provides low crosstalk and high-resolution 3D image experience seamlessly without 3D eyeglasses by overcoming the viewing position restriction. However, accurate and fast eye position detection and tracking are still challenging, owing to the various light conditions, camera control, thick eyeglasses, eyeglass sunlight reflection, and limited system resources. This study presents a robust, automated algorithm and relevant systems for accurate and fast detection and tracking of eye pupil centers in 3D with a single visual camera and near-infrared (NIR) light emitting diodes (LEDs). Our proposed eye tracker consists of eye–nose detection, eye–nose shape keypoint alignment, a tracker checker, and tracking with NIR LED on/off control. Eye–nose detection generates facial subregion boxes, including the eyes and nose, which utilize an Error-Based Learning (EBL) method for the selection of the best learnt database (DB). After detection, the eye–nose shape alignment is processed by the Supervised Descent Method (SDM) with Scale-invariant Feature Transform (SIFT). The aligner is content-aware in the sense that corresponding designated aligners are applied based on image content classification, such as the various light conditions and wearing eyeglasses. The conducted experiments on real image DBs yield promising eye detection and tracking outcomes, even in the presence of challenging conditions.

Learning discriminative subregions and pattern orders for facial gender classification

Facial image-based gender classification has been widely used in many real-world applications. Most of the existing work, however, focuses on designing sophisticated or specific feature descriptors for the entire face, which neglects the discriminative information carried by facial components and pattern order combinations. To address the issue, in this paper, we first propose a generalized texture operator, i.e., the multi-spatial multi-order interlaced pattern (MMIP) matrix, to represent the gender information possessed by the facial subregions with textural pattern orders. A chain-type support vector machine (CSVM) based feature vector selection scheme, is then developed to highlight the gender characteristics. As a result, the discriminative subregions and pattern orders are constructed as the feature representation for facial gender classification. We evaluate our proposed method on four benchmark datasets (i.e., FRGC 2.0, FERET, LFW and UND) for gender classification and demonstrate its interpretability, effectiveness and efficiency compared with state-of-the-art methods.

Histogram distance metric learning for facial expression recognition

Facial expression recognition is an interesting and challenging problem in computer vision. So far, much research has been performed in this area; however, facial expression recognition in uncontrolled conditions has remained an unresolved problem. The widely-used feature descriptors in computer vision are often histogram data. In this paper, a new metric learning method is presented for histogram data classification. In this method, chi-squared distance is appropriately modified for metric learning. Then, a convex cost function is proposed to use in metric learning optimization. Moreover, the proposed algorithm is redefined as Local Metric Learning for facial expression recognition problem. In this definition, the proposed metric learning method is applied locally on facial sub-regions. Experimental results on four histogram datasets (dslr, webcam, amazon, and caltech) as well as controlled and uncontrolled facial expression recognition datasets (CK+, SFEW, and RAF-DB) show that the proposed method has superior performance compared to the state-of-art methods.

Transdermal optical imaging revealed different spatiotemporal patterns of facial cardiovascular activities

Human cardiovascular activities are important indicators of a variety of physiological and psychological activities in human neuroscience research. The present proof-of-concept study aimed to reveal the spatiotemporal patterns of cardiovascular activities from the dynamic changes in hemoglobin concentrations in the face. We first recorded the dynamics of facial transdermal blood flow using a digital video camera and the Electrocardiography (ECG) signals using an ECG system simultaneously. Then we decomposed the video imaging data extracted from different sub-regions of a face into independent components using group independent component analysis (group ICA). Finally, the ICA components that included cardiovascular activities were identified by correlating their magnitude spectrum to those obtained from the ECG. We found that cardiovascular activities were associated with five independent components reflecting different spatiotemporal dynamics of facial blood flow changes. The strongest strengths of these ICA components were observed in the bilateral forehead, the left chin, and the left cheek, respectively. Our findings suggest that the cardiovascular activities presented different dynamic properties within different facial sub-regions, respectively. More broadly, the present findings point to the potential of the transdermal optical imaging technology as a new neuroscience methodology to study human physiology and psychology, noninvasively and remotely in a contactless manner.

Driver drowsiness detection using facial dynamic fusion information and a DBN

Driver drowsiness is a frequent cause of traffic accidents. Research on driver drowsiness detection methods is important to improve road traffic safety. Previous driving fatigue detection methods frequently extracted single features such as eye or mouth changes and trained shallow classifiers, which limit the generalisation capability of these methods. This study proposes a framework for recognising driver drowsiness expression by using facial dynamic fusion information and a deep belief network (DBN) to address the aforementioned problem. First, the landmarks and textures of the facial region are extracted from videos captured using a high-definition camera. Then, a DBN is built to classify facial drowsiness expressions. Finally, the authors' method is tested on a driver drowsiness dataset, which includes different genders, ages, head poses and illuminations. Certain experiments are also carried out to investigate the effects of different facial subregions and temporal resolutions on the accuracy of driver fatigue recognition. Results demonstrate the validity of the proposed method, which has an average accuracy of 96.7%.

Video-based human heart rate measurement using joint blind source separation

Abstract Remote (non-contact) measurements of human cardiopulmonary physiological parameters based on photoplethysmography (PPG) can lead to efficient and comfortable medical assessment, which is important in human healthcare. It was shown that human facial blood volume variation during cardiac cycle can be indirectly captured by common Red–Green–Blue (RGB) cameras. In this paper, we show that it is promising to incorporate data from different facial sub-regions to improve remote measurement performance. We propose a novel method for non-contact video-based human heart rate (HR) measurement by exploring correlations among facial sub-regions via joint blind source separation (J-BSS). To our knowledge, this is the first time that J-BSS approaches, instead of prevailing BSS techniques such as independent component analysis (ICA), is successfully applied in non-contact physiological parameter measurement. We test the proposed method on a large public database, which provides the subjects’ left-thumb plethysmograph signals as ground truth. Experimental results show that the proposed J-BSS method outperforms previous ICA-based methodologies.

Facial Asymmetry in Children with Unicoronal Synostosis Who Have Undergone Craniofacial Reconstruction in Infancy.

Quantitatively assess 3D spatially detailed soft-tissue facial asymmetry in children who had undergone craniofacial reconstruction for Unicoronal Synostosis (UCS), and compare the facial asymmetry to control patients. It was hypothesized that there would be no significant differences in the facial asymmetry between the groups. Clinical, retrospective follow-up study. Methodological study. Primary care center. Twenty-two children with UCS were selected after review of records. isolated UCS; surgically treated for UCS within the first 19 months of life, without secondary reconstruction; and DNA analysis for the Muenke mutation. An age- and sex-matched control group was employed. The UCS group had undergone bilateral craniotomy of the frontal bone with unilateral supraorbital rim advancement. Using 3D surface scanning, a detailed map of 3D asymmetry presenting the amount of asymmetry in the sagittal, vertical, and transverse directions was calculated for six facial subregions. The facial asymmetry in the UCS group was significantly larger than in the control group for all regions, to the largest extent in the sagittal direction (level of significance: 5%). The regions with the most pronounced asymmetry were cheeks (mean: 5.45 mm; SD: 1.83 mm), forehead (mean: 5.00 mm; SD: 1.57 mm), and eyes (mean: 4.26 mm; SD: 1.44 mm). Ninety percent of the UCS patients in the study had significant facial asymmetry throughout the facial area. The study demonstrates a methodology of facial asymmetry quantification well suited for soft-tissue surgical outcome evaluations and long-term follow-up studies in patients with craniofacial anomalies.