3D Facial Images Research Articles

Abstract: Volumetric Changes of the Mid and Lower Face with Animation and the Standardization of 3D Facial Imaging

GOALS: 3D photography provides volumetric data and surface topography with sub-millimeter precision, but current methods lack standardization for facial expression and position. For example, smiling can mimic the desired outcomes of a facelift and fat grafting by increasing malar volume while decreasing volume in the jowls. Our goal was to quantify volumetric changes of the mid and lower face caused by facial expression so that we can accurately assess volumetric changes after surgery. Secondarily, we aimed to identify soft tissue landmarks that can be used to detect these changes in facial expression or head position to ensure the standardization of 3D images. METHODS: 3D facial images of 16 subjects performing 22 facial expressions or changes in head position were captured. Variable degrees of animation during smiling and frowning were also evaluated. Volumetric changes of the malar and jowl regions with facial expressions were quantified using a 3D superimposed image subtraction technique. Translation (movement >1 mm) of 14 standard soft tissue surface landmarks was assessed during various facial animations to determine which landmarks can be utilized to standardize 3D images. RESULTS: Sixteen of the 22 facial expression studied had a significant effect on malar and/or jowl volume. Significant volume changes were noted with subtle animation during smiling and frowning. During maximal smile, mean excursion of oral commissures was 13.7mm +/- 3.0 and the malar region demonstrated a 17.2mL +/- 5.7 increase in volume, whereas the jowl region demonstrated a 1.7mL+/- 0.9 decrease in volume. During maximal frowning, mean excursion of oral commissures was 7.4mm +/- 2.3. The malar region demonstrated a 3.7mL +/- 2.3 decrease in volume, whereas the jowl region demonstrated a 5.4mL +/- 0.9 increase in volume. In the act of smiling, there was significant volume augmentation of the malar region with small degrees of oral commissure excursion, such that a quarter smile (excursion = 25% of excursion with maximal smile) could induce a volume change equivalent to 75% of maximum smile. The jowl decreases in volume with smiling, but in a linear fashion (a quarter smile yields a volume change equivalent to 25% of a maximum smile). Frowning exhibited the opposite, but similarly significant changes in mid and lower facial volume. A 1mm commissure excursion during smiling can produce 5-10mL of malar augmentation or 2-4mL of jowl augmentation with frowning. We believe these results are related to isometric contraction of mimetic muscles and their influence on fat compartments. A combination of 5 landmarks (glabella, bilateral cheilion, pogonion and laryngeal prominence) was found to indicate all tested facial expressions which effect mid and facial volumes, with the exception of platysmal straining. CONCLUSION: Most facial expressions will lead to translation of the oral commissures, with even a subtle (1mm) excursion leading to 5-10mL of malar augmentation during smiling or 2-4mL of jowl augmentation during frowning. We identified that the glabella, pogonion, laryngeal prominence, and bilateral cheilion can be used as a referencing system to allow identification of subtle changes in head position and facial expression for standardization of 3D images.

Face Recognition Based on Rearranged Modular Two-Dimensional Locality Preserving Projection

This paper proposes a novel algorithm for feature extraction for face recognition, namely the rearranged modular two-dimensional locality preserving projection (Rm2DLPP). In the proposed algorithm, the original images are first divided into modular blocks, then the subblocks are rearranged to form two-dimensional matrices and finally the two-dimensional locality preserving projection algorithm is applied directly on the arranged matrices. The advantage of the Rm2DLPP algorithm is that it can utilize the local block features and global spatial structures of 2D face images simultaneously. The performance of the proposed method is evaluated and compared with other face recognition methods on the ORL, AR and FERET databases. The experimental results demonstrate the effectiveness and superiority of the proposed approach.

Objectively Measuring Observer Attention in Severe Thyroid-Associated Orbitopathy: A 3D Study.

Measure the attentional distraction of facial deformity related to severe thyroid-associated orbitopathy using three-dimensional (3D) images and eye-tracking technology. Observers recruited at an academic tertiary referral center viewed 3D facial images of patients with severe thyroid-associated orbitopathy (TAO) and controls without TAO. An infrared eye-tracking monitor recorded their eye movements and fixations in real time. Multivariate Hotelling's analysis, followed by planned posthypothesis testing, was used to compare fixation durations for predefined regions of interest, including the eyes, nose, mouth, central triangle, and remaining face without the central triangle between severe TAO patients and controls. One hundred sixteen observers (mean age 26.4 years, 51% female) successfully completed the eye-tracking experiment. The majority of their attention was directed toward the central triangle (eyes, nose, mouth). On multivariate analysis, there were significant differences in the distribution of attention between control and severe TAO faces (T2 = 49.37; F(5,922) = 9.8314, P < 0.0001). On planned posthypothesis testing, observers attended significantly more to the eyes (0.77 seconds, P < 0.0001, 95% confidence interval [CI], 0.51, 1.03 seconds) and less to the nose (-0.42 seconds, P < 0.0001, 95% CI, -0.23, -0.62 seconds) in severe TAO patients. There was no significant difference in time spent on the mouth, the total time spent on the central triangle, or time spent in the remaining face between the two groups. Severe TAO distracted observer attention toward the eyes compared to control patients. These data lend insight into how TAO may alter observers' perceptions of these patients. Future studies should investigate how these changes in observer gaze patterns may reflect the social perception of TAO patients. NA Laryngoscope, 129:1250-1254, 2019.

Age Estimation and Age-related Facial Reconstruction of Xinjiang Uygur Males by Three-dimensional Human Facial Images.

To search age-correlated facial features and construct an age estimation model based on the three-dimensional （3D） facial images of Xinjiang Uygur males, and to structure individual face images of old age and young age. Pretreatment was performed to collect 105 3D facial images of Xingjiang Uygur males aged between 17-57 years by Artec Studio software. The facial images were transferred to high-density 3D dot matrix data by FaceAnalysis software, and each image could be represented with 32 251 vertexes. Central correction of the facial images was done and all the data were aligned to a standard coordinate frame by generalized Procrustes analysis （GPA）. The age estimation model was established by partial least square regression （PLSR）. Furthermore, the changes of age-correlated facial features were presented on the heat map of average face, and the reconstruction of facial images at different ages was performed based on this model. With age, the average faces showed a series of changes including the nasolabial sulcus deepening, cheek sinking, cheekbone protruding and eye corner drooping. The Pearson correlation coefficient （r） between estimated age and chronological age was 0.71. The mean absolute deviation （MAD） of age estimation was 6.37 years. The results of age estimation in >30-40 years group showed a best accuracy （MAD=4.27 years）, and the deviations increased with age after 40 years. The composite facial images represented a significant result with age on facial morphological features and aging. The results of this study reveal the age-correlated facial features and aging markers in Uygur population, which help to construct a reliable age estimation model.

A Coarse-to-Fine Approach for 3D Facial Landmarking by Using Deep Feature Fusion

Facial landmarking locates the key facial feature points on facial data, which provides not only information on semantic facial structures, but also prior knowledge for other kinds of facial analysis. However, most of the existing works still focus on the 2D facial image which may suffer from lighting condition variations. In order to address this limitation, this paper presents a coarse-to-fine approach to accurately and automatically locate the facial landmarks by using deep feature fusion on 3D facial geometry data. Specifically, the 3D data is converted to 2D attribute maps firstly. Then, the global estimation network is trained to predict facial landmarks roughly by feeding the fused CNN (Convolutional Neural Network) features extracted from facial attribute maps. After that, input the local fused CNN features extracted from the local patch around each landmark estimated previously, and other local models are trained separately to refine the locations. Tested on the Bosphorus and BU-3DFE datasets, the experimental results demonstrated effectiveness and accuracy of the proposed method for locating facial landmarks. Compared with existed methods, our results have achieved state-of-the-art performance.

Genome-wide variants of Eurasian facial shape differentiation and a prospective model of DNA based face prediction

It is a long-standing question as to which genes define the characteristic facial features among different ethnic groups. In this study, we use Uyghurs, an ancient admixed population to query the genetic bases why Europeans and Han Chinese look different. Facial traits were analyzed based on high-dense 3D facial images; numerous biometric spaces were examined for divergent facial features between European and Han Chinese, ranging from inter-landmark distances to dense shape geometrics. Genome-wide association studies (GWAS) were conducted on a discovery panel of Uyghurs. Six significant loci were identified, four of which, rs1868752, rs118078182, rs60159418 at or near UBASH3B, COL23A1, PCDH7 and rs17868256 were replicated in independent cohorts of Uyghurs or Southern Han Chinese. A prospective model was also developed to predict 3D faces based on top GWAS signals and tested in hypothetic forensic scenarios.

Combined Face-Brain Morphology and Associated Neurocognitive Correlates in Fetal Alcohol Spectrum Disorders.

Since the 1970s, a range of facial, neurostructural, and neurocognitive adverse effects have been shown to be associated with prenatal alcohol exposure. Typically, these effects are studied individually and not in combination. Our objective is to improve the understanding of the teratogenic effects of prenatal alcohol exposure by simultaneously considering face-brain morphology and neurocognitive measures. Participants were categorized as control (n=47), fetal alcohol syndrome (FAS, n=22), or heavily exposed (HE) prenatally, but not eligible for a FAS diagnosis (HE, n=50). Structural brain MRI images and high-resolution 3D facial images were analyzed using dense surface models of features of the face and surface shape of the corpus callosum (CC) and caudate nucleus (CN). Asymmetry of the CN was evaluated for correlations with neurocognitive measures. (i) Facial growth delineations for FAS, HE, and controls are replicated for the CN and the CC. (ii) Concordance of clinical diagnosis and face-based control-FAS discrimination improves when the latter is combined with specific brain regions. In particular, midline facial regions discriminate better when combined with a midsagittal profile of the CC. (iii) A subset of HE individuals was identified with FAS-like CN dysmorphism. The average of this HE subset was FAS-like in its facial dysmorphism. (iv) Right-left asymmetry found in the CNs of controls is not apparent for FAS, is diminished for HE, and correlates with neurocognitive measures in the combined FAS and HE population. Shape analysis which combines facial regions with the CN, and with the CC, better identify those with FAS. CN asymmetry was reduced for FAS compared to controls and is strongly associated with general cognitive ability, verbal learning, and recall in those with prenatal alcohol exposure. This study further extends the brain-behavior relationships known to be vulnerable to alcohol teratogenesis.

Influence of involuntary facial expressions on reproducibility of 3D stereophotogrammetry in children with and without complete unilateral cleft lip and palate from 3 to 18\xa0months of age

ObjectivesTo assess the influence of involuntary facial expressions on 3D facial stereophotogrammetry reproducibility in children with and without unilateral cleft lip, alveolus and palate (UCLP) aged 3–18 months.Materials and methodsThree to eight 3D facial images per time point were acquired within 10 min of 31 children with UCLP and 50 controls at 3, 12 and 18 months of age. 3D mapping of two 3D facial images per subject per age was performed. Distance kits of the full face and nasolabial area were calculated.ResultsIn the total subject pool, mean variation between two 3D facial images ranged from 0.38–0.88 mm. There were no significant differences within groups for the various ages. Variation between controls and UCLP subjects did not differ significantly. Variation was higher in the nasolabial area than in the full face.ConclusionsThe influence of involuntary facial expressions on the estimation of facial growth should not be underestimated, especially in the nasolabial region of UCLP subjects aged 3 months. To improve 3D facial imaging reliability, image capturing should be performed by a trained photographer following a meticulous image capturing protocol, including thorough review after capture.Clinical relevanceFacial 3D stereophotogrammetry is a useful tool for monitoring facial growth longitudinally in young children with facial deformities, as no radiation is involved and image capture is easy and fast. It can be performed reliably in children with and without UCLP aged 3–18 months by an experienced photographer utilising a meticulous image capturing protocol.

Female Facial Attractiveness Assessed from Three-Dimensional Contour Lines by University Students.

Background: Three-dimensional (3D) images could provide more accurate evaluation for facial attractiveness than two-dimensional (2D) images. The 3D facial image could be simplified into gray scale 3D contour lines. Whether female facial attractiveness could be perceived in these simplified 3D facial contour lines should be determined. Methods: A series of 100 2D photographs (one frontal and two lateral views) and 3D contour lines extracted from 3D facial images of females were projected onto a screen. Each image presentation lasted 5 s, and the evaluators marked their impression of each image’s facial attractiveness on a five-point Likert scale within 3 s of its presentation. The evaluation of the 3D contour lines was performed twice, 2 weeks apart. The evaluators were university students. Results: High consistency (r = 0.92) was found for the first and second evaluation of 3D facial contour lines for female facial attractiveness. The judgments of unattractive face were more consistent than the judgments of attractive face. Male students tended to give lower scores than female students in the evaluation of female facial attractiveness. Conclusions: Female facial attractiveness could be evaluated by 3D facial contour lines. 3D facial contour lines should be one of the key factors of facial attractiveness.

CNN-Based Real-Time Dense Face Reconstruction with Inverse-Rendered Photo-Realistic Face Images.

With the powerfulness of convolution neural networks (CNN), CNN based face reconstruction has recently shown promising performance in reconstructing detailed face shape from 2D face images. The success of CNN-based methods relies on a large number of labeled data. The state-of-the-art synthesizes such data using a coarse morphable face model, which however has difficulty to generate detailed photo-realistic images of faces (with wrinkles). This paper presents a novel face data generation method. Specifically, we render a large number of photo-realistic face images with different attributes based on inverse rendering. Furthermore, we construct a fine-detailed face image dataset by transferring different scales of details from one image to another. We also construct a large number of video-type adjacent frame pairs by simulating the distribution of real video data.11.All these coarse-scale and fine-scale photo-realistic face image datasets can be downloaded from https://github.com/Juyong/3DFace. With these nicely constructed datasets, we propose a coarse-to-fine learning framework consisting of three convolutional networks. The networks are trained for real-time detailed 3D face reconstruction from monocular video as well as from a single image. Extensive experimental results demonstrate that our framework can produce high-quality reconstruction but with much less computation time compared to the state-of-the-art. Moreover, our method is robust to pose, expression and lighting due to the diversity of data.

Patch-based face recognition using a hierarchical multi-label matcher

This paper proposes a hierarchical multi-label matcher for patch-based face recognition. In signature generation, a face image is iteratively divided into multi-level patches. Two different types of patch divisions and signatures are introduced for 2D facial image and texture-lifted image, respectively. The matcher training consists of three steps. First, local classifiers are built to learn the local matching of each patch. Second, the hierarchical relationships defined between local patches are used to learn the global matching of each patch. Three ways are introduced to learn the global matching: majority voting, ℓ1-regularized weighting, and decision rule. Last, the global matchings of different levels are combined as the final matching. Experimental results on different face recognition tasks demonstrate the effectiveness of the proposed matcher at the cost of gallery generalization. Compared with the UR2D system, the proposed matcher improves the Rank-1 accuracy significantly by 3% and 0.18% on the UHDB31 dataset and IJB-A dataset, respectively.

Estimating age and synthesising growth in children and adolescents using 3D facial prototypes

3D facial images are becoming increasingly common. They provide more information about facial form than their 2D counterparts and will be useful in future forensic applications. These include age estimation and predicting changes in appearance of missing persons (synthetic growth). We present a framework for both age estimation and synthetic growth of children and adolescents from 3D photographs. Age estimation accuracy was substantially better than for existing approaches (mean absolute error=1.19 years). Our synthetically ‘grown’ images were compared to actual longitudinal images of the same cases. On average 75% of the head overall and 85% of the face were predicted correctly to within three millimetres. We find that our approach is most suitable for ageing children from late childhood into adolescence. The work can be improved in the future by modelling skin colouring and taking account of other factors that influence face shape such as BMI.

Research On the Standard Operation Procedure of Mobile 3D Scanner

1. Abstract 1.1. Background: For the past few years, 3D facial imaging technology has been widely used as preoperative evaluation and surgery design for plastic and cosmetic surgery. Compare to the stationary scanner, handheld scanner is more portable and mobile, which leads to a higher demand for the operator manipulation. The aim of this study is to search and improve the standardized procedure of handheld 3D scanner, in order to collect preferable data and improve treatment accuracy. 1.2. Methods: Artec Eva 3D scanner was used to scan the subject head-face area. By altering the distance and angle between the scanner and subject, changing the scanner movement patterns and tracks, and modifying the light and background, multiple images were collected and analysed. By comparing the differences between generated images and the amount of software errors, the best operation procedure for mobile 3D scanner was concluded. 1.3. Results: With adequate light, the optimal distance between scanner and subject (working distance) was 80cm, and the scanner was 5cm above the subject head. The scanner was stabilized on a tripod with wheels and scans following an arc track. The optimal scanning time was 8s, which was easy to control and could keep the integrity of collected images at the same time. 1.4. Conclusion: By the comparative study of light, background, distance, angle and scanner movement pattern, we found that handheld scanner with standardized operation procedure can stable the scanning results and improve the collection efficiency and accuracy. 2. Keywords: Accurate Treatment Method; Facial Scanning; Mobile 3D Scanner; Standardized Operation

Accurate landmarking from 3D facial scans by CNN and cascade regression

Facial landmarking locates the key facial feature points on facial data, which provides not only information on semantic facial structures, but also prior knowledge for other types of facial analysis. However, most of the existing works still focus on the 2D facial image which is quite sensitive to the lighting condition changes. In order to address this limitation, this paper proposed a coarse-to-fine method only based on the 3D facial scan data extracted from professional equipment to automatically and accurately estimate the landmark localization. Specifically, we firstly trained a convolutional neural network (CNN) to initialize the face landmarks instead of the mean shape. Then the proposed cascade regression networks learn the mapping function between 3D facial geometry feature and landmarks location. Tested on Bosphorus database, the experimental results demonstrated effectiveness and accuracy of the proposed method for [Formula: see text] landmarks. Compared with other methods, the results in several points demonstrate state-of-the-art performance.

Ensemble landmarking of 3D facial surface scans

Landmarking of 3D facial surface scans is an important analysis step in medical and biological applications, such as genome-wide association studies (GWAS). Manual landmarking is often employed with considerable cost and rater dependent variability. Landmarking automatically with minimal training is therefore desirable. We apply statistical ensemble methods to improve automated landmarking of 3D facial surface scans. Base landmarking algorithms using features derived from 3D surface scans are combined using either bagging or stacking. A focus is on low training complexity of maximal 40 training samples with template based landmarking algorithms that have proved successful in such applications. Additionally, we use correlations between landmark coordinates by introducing a search strategy guided by principal components (PCs) of training landmarks. We found that bagging has no useful impact, while stacking strongly improves accuracy to an average error of 1.7 mm across all 21 landmarks in this study, a 22% improvement as compared to a previous, comparable algorithm. Heritability estimates in twin pairs also show improvements when using facial distances from landmarks. Ensemble methods allow improvement of automatic, accurate landmarking of 3D facial images with minimal training which is advantageous in large cohort studies for GWAS and when landmarking needs change or data quality varies.

Linear accuracy of cone-beam computed tomography and a 3-dimensional facial scanning system: An anthropomorphic phantom study.

PurposeThis study was conducted to evaluate the accuracy of linear measurements of 3-dimensional (3D) images generated by cone-beam computed tomography (CBCT) and facial scanning systems, and to assess the effect of scanning parameters, such as CBCT exposure settings, on image quality.Materials and MethodsCBCT and facial scanning images of an anthropomorphic phantom showing 13 soft-tissue anatomical landmarks were used in the study. The distances between the anatomical landmarks on the phantom were measured to obtain a reference for evaluating the accuracy of the 3D facial soft-tissue images. The distances between the 3D image landmarks were measured using a 3D distance measurement tool. The effect of scanning parameters on CBCT image quality was evaluated by visually comparing images acquired under different exposure conditions, but at a constant threshold.ResultsComparison of the repeated direct phantom and image-based measurements revealed good reproducibility. There were no significant differences between the direct phantom and image-based measurements of the CBCT surface volume-rendered images. Five of the 15 measurements of the 3D facial scans were found to be significantly different from their corresponding direct phantom measurements (P<.05). The quality of the CBCT surface volume-rendered images acquired at a constant threshold varied across different exposure conditions.ConclusionThese results proved that existing 3D imaging techniques were satisfactorily accurate for clinical applications, and that optimizing the variables that affected image quality, such as the exposure parameters, was critical for image acquisition.

Effective Descriptors based Face Recognition Technique for Robotic Surveillance Systems

The aim is to propose SPHORB Face Recognition technique for Robotic Surveillance systems. Surveillance bots with efficient face recognition system becomes even more powerful and adds to the applications of the bots. In this paper we compare the results of face recognition using SPHORB (a new fast and robust binary feature detector and descriptor for spherical panoramic image) [1] algorithm with ORB (Oriented Fast and Rotated Brief) [4] algorithm for classifying 2D and 3D images Database. In the first part 2D face images are randomly selected from LFW [32] with good sample of images across gender and ethnicity. The number of keypoints that were identified and number of matching keypoints between the 2D images using SPHORB and ORB were compared. From the results we determine that the SPHORB algorithm has identified more keypoints for the same 2D image than ORB, it has processed more images but ORB gives better accuracy than SPHORB for 2D images. In the next part we investigated the image matching between a front face 2D image with a 3D image. The front face image and 3D images are randomly selected from the ThatsMyFace.com [2] with good sample of images across genders, ethnicity and age. The front face image is compared with a randomly selected 3D image from the sample class set using SPHORB and ORB algorithms. The number of keypoints that were identified and number of matching keypoints between images using SPHORB and ORB were compared. From the results we determine that the SPHORB algorithm has identified more keypoints for the same image than ORB, it has processed more images and has accuracy comparable to the ORB algorithm.

Soft-tissue 3D Facial Imaging in Children and Adolescents: Towards the Definition of New Reference Standards

3D optical, surface imaging is becoming an important instrument for diagnosis, treatment planning and follow up of children, who should not receive ionizing radiations unless strictly necessary. The instruments are fast, safe, and neither painful nor fastidious. Their use for clinical applications should be accompanied by a systematic data collection in healthy young people, spanning from birth to late adolescence into young adulthood. Researchers all over the world should collaborate towards this goal, with the definition of these new facial standards.

3D Face Feature Location Method Based on Stripes and Shape Index

A 3D face feature location method based on stripes and shape index is proposed in order to locate the feature points in face exactly and quickly. The grating projection technique is used to obtain 3D face image. Based on the difference between the background fringe image and the deformed fringe image, the basic information of the human face in the image is determined. The basic information includes left and right edge lines, upper and lower edge coordinates and the width of the human face in the image. And then, the approximate position of the ear is quickly determined according to the left or right edge line. The found areas of the tip of the nose and the inner canthus are reduced according to the position of the ear. The position of the tip of the nose and the inner canthus are determined according to the height and shape index information. The experiment was conducted in a dark environment, with an average total time of 4.05 seconds and an average time of positioning of 1.07 seconds. When the allowable error is 15 pixels, the positioning accuracy is 85.34% for different poses, and the positioning accuracy is 96.88% when the face rotation angle is less than 20 degrees.

The face in marfan syndrome: A 3D quantitative approach for a better definition of dysmorphic features.

Marfan syndrome (MFS) is a rare hereditable disorder of connective tissue caused by mutations in the fibrillin-1 gene FBN1. Timely diagnosis of MFS is essential to prevent life-threatening cardiovascular complications; nevertheless it can be difficult owing to the phenotypic variability of the syndrome. No clear quantitative definition of facial abnormalities associated with MFS is available. The aim of this study was to improve the definition of the facial phenotype associated with MFS and to verify the usefulness of a 3D noninvasive quantitative approach for its early recognition. 3D facial images of 61 Italian subjects with MFS, aged 16-64 years (21 males, 38 ± 15 years; 40 females, 41 ± 13 years) were obtained by stereophotogrammetry. From the coordinates of 17 soft-tissue facial landmarks, linear distances and angles were computed; z score values were calculated to compare patients with healthy reference subjects (400 males, 379 females) matched for sex and age. Student's t test was used for statistical comparisons. All subjects with MFS showed greater facial divergence (P < 0.001; mean z score +1.9) and a lower facial height index (P < 0.001; mean z score -1.9) than reference subjects, both values being influenced by a shorter mandibular ramus (P < 0.001; mean z score -1.9) and a mild but significant increase in facial height (P < 0.001; mean z score +1.2). Palpebral down-slanting was found in 85% of MFS subjects. There were no sex differences. Quantitative abnormalities identified in this study enrich information about the facial dysmorphism in MFS and confirm its usefulness for early recognition of the disease. Clin. Anat. 31:380-386, 2018. © 2017 Wiley Periodicals, Inc.