3D Facial Images Research Articles

A reconstruction method of 3D face model from front and side 2D face images using deep learning model

In this study, we focus on automatic three-dimensional (3D) face reconstruction from two-dimensional (2D) face images using a deep learning model. The conventional methods have been used to develop models that can reconstruct 3D faces from 2D images. However, for Japanese faces, the models cannot accurately reconstruct images, large errors occur in areas such as the nose and mouth, because most of the training data are foreigner’s face images. To solve this problem, we proposed a method that uses not only a frontal 2D face image but also a side-view 2D face image for the 3D face reconstruction, and the resulting 3D model is a combination of two 3D reconstructed models, which are created from the frontal and side-view 2D face images using iterative closest point algorithm. As a result, the accuracy of the proposed method is better than the conventional method.

An efficient encoder–decoder model for portrait depth estimation from single images trained on pixel-accurate synthetic data

Depth estimation from a single image frame is a fundamental challenge in computer vision, with many applications such as augmented reality, action recognition, image understanding, and autonomous driving. Large and diverse training sets are required for accurate depth estimation from a single image frame. Due to challenges in obtaining dense ground-truth depth, a new 3D pipeline of 100 synthetic virtual human models is presented to generate multiple 2D facial images and corresponding ground truth depth data, allowing complete control over image variations. To validate the synthetic facial depth data, we propose an evaluation of state-of-the-art depth estimation algorithms based on single image frames on the generated synthetic dataset. Furthermore, an improved encoder–decoder based neural network is presented. This network is computationally efficient and shows better performance than current state-of-the-art when tested and evaluated across 4 public datasets. Our training methodology relies on the use of synthetic data samples which provides a more reliable ground truth for depth estimation. Additionally, using a combination of appropriate loss functions leads to improved performance than the current state-of-the-art network performances. Our approach clearly outperforms competing methods across different test datasets, setting a new state-of-the-art for facial depth estimation from synthetic data.

Orthodontics in the Era of Digital Innovation – A Review

The field of orthodontics in its new era is venturing ahead to more up-to-date technological point of view. Digital technology has a significant effect on our lives ever since the modernization of mobile phones. The advances in technology have remodelled the diagnosis and treatment plan in the field of medicine. Digital workflows are currently increasing in the orthodontic practice and has touched every aspect of orthodontics – with transformations in the documentation, study casts, analysis of a dental malocclusion, smile designing, treatment planning and for fabrication of orthodontic appliances. Three - dimensional imaging of the dentition, skeletal components and the face allows for treatment planning in three dimension and use of computer aided design (CAD) and computer aided manufacturing (CAM) for customization of orthodontic appliances. Software integration of digital models, 3D facial imaging and cone-beam computed tomography (CBCT) makes it possible to simulate the treatment plan and to attain a good communication with the patients. Recent advancement in digital videography has allowed the clinicians to capture patient’s speech, oral and pharyngeal function, and smile at the same time. Effective and optimal tooth movement required for the patient can also be monitored with the digitalization. Moreover, this digital platform has created the advantage of accessing the patient’s information from any location with the help of cloud based computing storage systems. These advancements have improved the efficiency, accuracy, consistency, and predictability of the treatment outcomes and have also led to progress in educational component and communication. The digitalization is bringing about a revolutionary change in the field of diagnosis and treatment planning, posing a challenge to clinical efficiency and knowledge. KEY WORDS Digital Orthodontics, Digital Workflow, Digital Dental Models, Digital Radiography, CAD CAM, Orthodontics, Review

Automatic Detection of Fiducial Landmarks Toward the Development of an Application for Digitizing the Locations of EEG Electrodes: Occipital Structure Sensor-Based Work.

The reconstruction of electrophysiological sources within the brain is sensitive to the constructed head model, which depends on the positioning accuracy of anatomical landmarks known as fiducials. In this work, we propose an algorithm for the automatic detection of fiducial landmarks of EEG electrodes on the 3D human head model. Our proposal combines a dimensional reduction approach with a perspective projection from 3D to 2D object space; the eye and ear automatic detection in a 2D face image by two cascades of classifiers and geometric transformations to obtain 3D spatial coordinates of the landmarks and to generate the head coordinate system, This is accomplished by considering the characteristics of the scanner information. Capturing the 3D model of the head is done with Occipital Inc. ST01 structure sensor and the implementation of our algorithm was carried out on MATLAB R2018b using the Computer Vision Toolbox and the FieldTrip Toolbox. The experimental results were aimed at recursively exploring the efficacy of the facial feature detectors as a function of the projection angle; they show that robust results are obtained in terms of false acceptance rate. Our proposal is an initial step of an approach for the automatic digitization of electrode locations. The experimental results demonstrate that the proposed method detects anatomical facial landmarks automatically, accurately, and rapidly.

Does Head Orientation Influence 3D Facial Imaging? A Study on Accuracy and Precision of Stereophotogrammetric Acquisition.

This study investigates the reliability and precision of anthropometric measurements collected from 3D images and acquired under different conditions of head rotation. Various sources of error were examined, and the equivalence between craniofacial data generated from alternative head positions was assessed. 3D captures of a mannequin head were obtained with a stereophotogrammetric system (Face Shape 3D MaxiLine). Image acquisition was performed with no rotations and with various pitch, roll, and yaw angulations. On 3D images, 14 linear distances were measured. Various indices were used to quantify error magnitude, among them the acquisition error, the mean and the maximum intra- and inter-operator measurement error, repeatability and reproducibility error, the standard deviation, and the standard error of errors. Two one-sided tests (TOST) were performed to assess the equivalence between measurements recorded in different head angulations. The maximum intra-operator error was very low (0.336 mm), closely followed by the acquisition error (0.496 mm). The maximum inter-operator error was 0.532 mm, and the highest degree of error was found in reproducibility (0.890 mm). Anthropometric measurements from alternative acquisition conditions resulted in significantly equivalent TOST, with the exception of Zygion (l)–Tragion (l) and Cheek (l)–Tragion (l) distances measured with pitch angulation compared to no rotation position. Face Shape 3D Maxiline has sufficient accuracy for orthodontic and surgical use. Precision was not altered by head orientation, making the acquisition simpler and not constrained to a critical precision as in 2D photographs.

Evaluation of the Soft Tissue Changes after Rapid Maxillary Expansion Using a Handheld Three-Dimensional Scanner: A Prospective Study.

Facial soft tissue esthetics is a priority in orthodontic treatment, and emerging of the digital technologies can offer new methods to help the orthodontist toward an esthetic outcome. This prospective study aimed to assess the soft tissue changes of the face after six months of retention following Rapid Maxillary Expansion (RME). The sample consisted of 25 patients (13 females, 12 males, mean age: 11.6 years) who presented with unilateral or bilateral posterior crossbite requiring RME, which was performed with a Hyrax expander. 3D facial images were obtained before treatment (T0) and at the end of a six-month retention period after the treatment (T1) using a structured-light 3D handheld scanner. Linear and angular measurements were performed and 3D deviation analyses were done for six morphological regions of the face. Significant changes in various areas of the nasal and the upper lip regions were observed. Based on the results of the study and within the limitations of the study, RME with a Hyrax expander results in significant morphological changes of the face after a six-month retention period.

Evaluation of facial asymmetry by stereophotogrammetry in individuals with unilateral maxillary impacted canine.

The aim of this study is to compare the symmetry of the facial and dentoalveolar structures of patients with unilateral impacted maxillary canine teeth with acontrol group of individuals without impacted teeth using three-dimensional face scans. The study included 28patients (10females, 18males) with unilateral impacted maxillary canine teeth and 28patients (15females, 13males) without any impacted maxillary canine between 12and 25years of age. The 3dMDface™ (3dMD Inc., Atlanta, GA, USA) imaging system was used to obtain 3D face images. 3dMD Vultus® (3dMD Inc., Atlanta, GA, USA) software was used for the measurements. According to the study findings, there were differences in linear measurements and volume measurements between the right and left sides of the face in patients with an impacted canine. However, these differences were not statistically significant. In patients with an impacted canine, surface differences between the right and left halves of the face were not found to be statistically different from the control group. There was no difference in the amount of facial asymmetry between patients with unilateral maxillary impacted canine and patients with normal tooth alignment in the control group.

Towards effective learning for face super-resolution with shape and pose perturbations

Recent development of convolutional neural networks (CNNs) has activated a lot of studies and boosted the performance greatly on image super-resolution. This paper addresses the issue of face super-resolution, which attracts a lot of interests in the photographic industry. We propose to make use of the face-specific priors to enhance the performance of face super-resolution with the convolutional neural networks. Classical facial prior models represent the 2D facial shape in a compact low-dimensional space expressed by principal components. Here, we impose perturbations on the low-dimensional space and generate face samples with novel appearance. First, we conduct 2D facial image perturbations through 2D facial landmarks. Then, we carry on the study with perturbations on 3D facial landmarks. Facial pose and shape are perturbated to generate novel appearances of a single 2D facial image. These novel facial samples are then fed into the training process of the convolutional neural networks for face super-resolution. The experimental results demonstrate that the proposed method is adaptable to various networks and achieves superior performance for the face super-resolution task.

The Intersection of the Genetic Architectures of Orofacial Clefts and Normal Facial Variation.

Unaffected relatives of individuals with non-syndromic cleft lip with or without cleft palate (NSCL/P) show distinctive facial features. The presence of this facial endophenotype is potentially an expression of underlying genetic susceptibility to NSCL/P in the larger unselected population. To explore this hypothesis, we first partitioned the face into 63 partially overlapping regions representing global-to-local facial morphology and then defined endophenotypic traits by contrasting the 3D facial images from 264 unaffected parents of individuals with NSCL/P versus 3,171 controls. We observed distinct facial features between parents and controls across 59 global-to-local facial segments at nominal significance (p ≤ 0.05) and 52 segments at Bonferroni corrected significance (p < 1.2 × 10–3), respectively. Next, we quantified these distinct facial features as univariate traits in another dataset of 8,246 unaffected European individuals and performed a genome-wide association study. We identified 29 independent genetic loci that were associated (p < 5 × 10–8) with at least one of the tested endophenotypic traits, and nine genetic loci also passed the study-wide threshold (p < 8.47 × 10–10). Of the 29 loci, 22 were in proximity of loci previously associated with normal facial variation, 18 were near genes that show strong evidence in orofacial clefting (OFC), and another 10 showed some evidence in OFC. Additionally, polygenic risk scores for NSCL/P showed associations with the endophenotypic traits. This study thus supports the hypothesis of a shared genetic architecture of normal facial development and OFC.

Soft tissue facial changes among adult females during alignment stage of orthodontic treatment: a 3D geometric morphometric study

BackgroundTo investigate changes in facial morphology during the first six months of orthodontic treatment among adult females receiving orthodontic treatment.Methods43 adult females receiving orthodontic treatment were randomly recruited. 3D facial images were taken at baseline (T0), three months (T1), and six months (T2) after treatment initiation. Spatially dense facial landmarks were digitized to allow for sufficient details in characterization of facial features. 3D geometric morphometrics and multivariate statistics were used to investigate changes in mean and variance of facial shape and facial form associated with treatment.ResultsWe observed statistically significant changes in facial shape across the three treatment stages (p = 0.0022). Pairwise comparisons suggested significant changes from T0 to T1 (p = 0.0045) and from T0 to T2 (p = 0.0072). Heatmap visualization indicated that the buccal and temporal region were invaginated while the labial region became protruded with treatment. The magnitude of shape change was 0.009, 0.004, and 0.010 from T0 to T1, T1 to T2, and T0 to T2, respectively, in unit of Procrustes distance. The average magnitude of change per-landmark was 1.32 mm, 0.21 mm, and 1.34 mm, respectively. Changes in mean facial form were not statistically significant (p = 0.1143). No changes in variance of facial shape were observed across treatment stages (p > 0.05).ConclusionRate of facial changes was twice as fast during the first three months as that during fourth to sixth month. Buccal and temporal region became invaginated while labial region became protruded with treatment.

Face spoofing detection under super-realistic 3D wax face attacks

• A new database with wax figure faces as super-realistic 3D face spoofing attacks. • High diversity and high quality of faces in database. • Fusion of attention-aware features of different face scales for face anti-spoofing. • Performance evaluation under both intra- and cross-database testing scenarios. Face spoofing attacks based on 3D face images have posed a severe security risk to face recognition systems. Despite the great effort made by the technical community in recent years, existing 3D face spoofing databases, mostly based on 3D masks, still suffer from small sample size, low diversity, or poor authenticity due to the production difficulty and high cost. To fill in this gap, we introduce a new database in this paper with 4-000 single wax figure faces, named SWFFD (Single Wax Figure Face Database), as a type of super-realistic 3D face presentation attack. Collected from online resources, this database has high diversity in terms of subjects, lighting conditions, facial poses, and recording devices. We have also designed a new detection method, which combines attention-aware features from different face scales to generate discriminative representations for realistic face spoofing attack detection. Extensive experiments have been conducted on the SWFFD as well as the CelebA-HQ database (containing real faces from the online collection). Experimental results have demonstrated the effectiveness of the proposed method in both intra-database and cross-database testing scenarios.

Smile dimensions affect self-perceived smile attractiveness

Facial expressions play a leading role in human interactions because they provide signaling information of emotion and create social perceptions of an individuals’ physical and personality traits. Smiling increases socially perceived attractiveness and is considered a signal of trustworthiness and intelligence. Despite the ample information regarding the social importance of an attractive smile, little is known about the association between smile characteristics and self-assessed smile attractiveness. Here we investigate the effect of smile dimensions on ratings of self-perceived smile attractiveness, in a group of 613 young adults using 3D facial imaging. We show a significant effect of proportional smile width (ratio of smile width to facial width) on self-perceived smile attractiveness. In fact, for every 10% increase in proportional smile width, self-perceived attractiveness ratings increased by 10.26%. In the present sample, this association was primarily evident in females. Our results indicate that objective characteristics of the smile influence self-perception of smile attractiveness. The increased strength of the effect in females provides support to the notion that females are overall more aware of their smile and the impact it has on their public image.

Validating 3D face morphing towards improving pre-operative planning in facial reconstruction surgery

ABSTRACT Pre-traumatic/pathologic 3D facial imaging is rarely available to guide craniofacial reconstruction. This study aims to independently validate the regional accuracy of a 3D morphable model to estimate face shapes from 2D photographs for craniofacial surgical planning. 3D shape estimates were generated using the Basel Face Model (BFM 2017) from 2D photographs of 100 multi-racial subjects in the Binghamton University 3D Facial Expression database. Accuracy was evaluated by the per-vertex Euclidean distance between the shape estimate and the true 3D scan within defined facial regions. The 3D estimates’ average RMS distance error across all facial regions was 2.68 ± 0.97 mm, based on photos analysed with 10,000 iterations. The eyes, cheek, chin, forehead and mouth regions fit within ~2.5 mm from the true face shape, representing only marginal perceptible error. The nose and temple regions had lower accuracy (~3.1 mm) for all subjects. Significant differences in the nose region were dependent on race (Caucasian: 2.4 mm, East-Asian: 4.8 mm) and sex (Male: 2.5 mm, Female: 3.6 mm). On average, 3D face shape estimates using the BFM yielded clinically acceptable accuracy sufficient for use in planning to guide several craniofacial reconstruction regions. However, in individual cases, considerable errors exceeded clinical limits.

A technique to match highly similar 3D objects with an application to biomedical security

Biometric technologies such as the face, fingerprint, and iris recognition have important utility in biomedical and healthcare applications. The use of biometrics in these applications ensures that critical medical information and access to secure premises and medical instruments is given only to authorized persons. In the past, the 2D face has been reliably used as biometrics in biomedical and healthcare applications. Though it provides remarkable performance in normal scenarios, the performance deteriorates in the presence of poor illumination, pose variations, and occlusions. These challenges are overcome by 3D face biometrics, where 3D face data (which provides complete geometric information of the face) is used in place of 2D face images. In this work, we develop a generic technique for matching of highly similar 3D objects and demonstrate its use in 3D face biometrics. The proposed technique combines the object classification utility from PointNet with One-Shot Learning from Siamese Network that converts the multi-class classification problem to a binary classification problem. We also propose a novel data augmentation technique that uses sub-sampling from the existing 3D data to increase the size and variability of the data, which is otherwise limited. Experimental results show that the proposed technique is considerably fast and accurate in the matching of highly similar 3D objects such as 3D human faces. It is also found to be highly efficient in terms of time and space and hence can be employed in designing real-time security solutions for biomedical, healthcare, and several applications in other fields.

Automatic Assessment of 3-Dimensional Facial Soft Tissue Symmetry Before and After Orthognathic Surgery Using a Machine Learning Model: A Preliminary Experience.

An objective and quantitative assessment of facial symmetry is essential for the surgical planning and evaluation of treatment outcomes in orthognathic surgery (OGS). This study applied the transfer learning model with a convolutional neural network based on 3-dimensional (3D) contour line features to evaluate the facial symmetry before and after OGS. A total of 158 patients were recruited in a retrospective cohort study for the assessment and comparison of facial symmetry before and after OGS from January 2018 to March 2020. Three-dimensional facial photographs were captured by the 3dMD face system in a natural head position, with eyes looking forward, relaxed facial muscles, and habitual dental occlusion before and at least 6 months after surgery. Three-dimensional contour images were extracted from 3D facial images for the subsequent Web-based automatic assessment of facial symmetry by using the transfer learning with a convolutional neural network model. The mean score of postoperative facial symmetry showed significant improvements from 2.74 to 3.52, and the improvement degree of facial symmetry (in percentage) after surgery was 21% using the constructed machine learning model. A Web-based system provided a user-friendly interface and quick assessment results for clinicians and was an effective doctor-patient communication tool. This work was the first attempt to automatically assess the facial symmetry before and after surgery in an objective and quantitative value by using a machine learning model based on the 3D contour feature map.

Retracted] Design of 3D Modeling Face Image Library in Multimedia Film and Television

The development of 3D modeling technology has promoted the development of the multimedia film and television industry. This article is aimed at studying the design of 3D modeling facial image library in multimedia film and television, at providing a more comprehensive facial image library for the multimedia film and television industry, at breaking the shackles of the traditional film and television industry with 3D technology, and at continuously surpassing traditional film and television media forms. This article deeply explores the background development of multimedia film and television and the characteristics of the development of new media. Starting from 3D technology, it extracts facial features of characters, transforms image data through deep autoencoders, and uses local binarization mode to perform the original facial image is subjected to texture feature extraction. In this paper, a number of experimental subjects were selected, and the subjects were photographed from the left, front, and right from multiple angles. Through the pinhole camera projection imaging process, the internal and external parameters of the camera were adjusted. In the process of 3D image construction, the image is first selected for feature detection, then the corresponding vector information and geometric conditions are matched to construct a 3D matrix, and the facial structure image is obtained by triangulation. This article compares the 3D production software on the market and selects the Maya platform suitable for building this system. The global constraint information is obtained by training some sample images. When searching the test image, find the appropriate feature point position according to the structural matching degree of the local image. When each search is completed, the global information will be used for constraint, so as to output reasonable feature information. The average residual range of the human face image constructed in this paper is 0.25‐0.45, and the maximum residual error does not exceed 4.0. The experimental method in this paper has good stability and robustness. Using the COM transmission model can make experimenters not need to think too much about the underlying details. This face animation‐driven simulation scheme can achieve more vivid facial expressions.

3D Face Image Inpainting with Generative Adversarial Nets

In the process of face recognition, face acquisition data is seriously distorted. Many face images collected are blurred or even missing. Faced with so many problems, the traditional image inpainting was based on structure, while the current popular image inpainting method is based on deep convolutional neural network and generative adversarial nets. In this paper, we propose a 3D face image inpainting method based on generative adversarial nets. We identify two parallels of the vector to locate the planer positions. Compared with the previous, the edge information of the missing image is detected, and the edge fuzzy inpainting can achieve better visual match effect. We make the face recognition performance dramatically boost.

Automated syndrome diagnosis by three-dimensional facial imaging

PurposeDeep phenotyping is an emerging trend in precision medicine for genetic disease. The shape of the face is affected in 30–40% of known genetic syndromes. Here, we determine whether syndromes can be diagnosed from 3D images of human faces. MethodsWe analyzed variation in three-dimensional (3D) facial images of 7057 subjects: 3327 with 396 different syndromes, 727 of their relatives, and 3003 unrelated, unaffected subjects. We developed and tested machine learning and parametric approaches to automated syndrome diagnosis using 3D facial images. ResultsUnrelated, unaffected subjects were correctly classified with 96% accuracy. Considering both syndromic and unrelated, unaffected subjects together, balanced accuracy was 73% and mean sensitivity 49%. Excluding unrelated, unaffected subjects substantially improved both balanced accuracy (78.1%) and sensitivity (56.9%) of syndrome diagnosis. The best predictors of classification accuracy were phenotypic severity and facial distinctiveness of syndromes. Surprisingly, unaffected relatives of syndromic subjects were frequently classified as syndromic, often to the syndrome of their affected relative. ConclusionDeep phenotyping by quantitative 3D facial imaging has considerable potential to facilitate syndrome diagnosis. Furthermore, 3D facial imaging of “unaffected” relatives may identify unrecognized cases or may reveal novel examples of semidominant inheritance.

Identifying Facemask-Wearing Condition Using Image Super-Resolution with Classification Network to Prevent COVID-19.

The rapid worldwide spread of Coronavirus Disease 2019 (COVID-19) has resulted in a global pandemic. Correct facemask wearing is valuable for infectious disease control, but the effectiveness of facemasks has been diminished, mostly due to improper wearing. However, there have not been any published reports on the automatic identification of facemask-wearing conditions. In this study, we develop a new facemask-wearing condition identification method by combining image super-resolution and classification networks (SRCNet), which quantifies a three-category classification problem based on unconstrained 2D facial images. The proposed algorithm contains four main steps: Image pre-processing, facial detection and cropping, image super-resolution, and facemask-wearing condition identification. Our method was trained and evaluated on the public dataset Medical Masks Dataset containing 3835 images with 671 images of no facemask-wearing, 134 images of incorrect facemask-wearing, and 3030 images of correct facemask-wearing. Finally, the proposed SRCNet achieved 98.70% accuracy and outperformed traditional end-to-end image classification methods using deep learning without image super-resolution by over 1.5% in kappa. Our findings indicate that the proposed SRCNet can achieve high-accuracy identification of facemask-wearing conditions, thus having potential applications in epidemic prevention involving COVID-19.

Three-dimensional facial-image analysis to predict heterogeneity of the human ageing rate and the impact of lifestyle.

Not all individuals age at the same rate. Methods such as the 'methylation clock' are invasive, rely on expensive assays of tissue samples and infer the ageing rate by training on chronological age, which is used as a reference for prediction errors. Here, we develop models based on convoluted neural networks through training on non-invasive three-dimensional (3D) facial images of approximately 5,000 Han Chinese individuals that achieve an average difference between chronological or perceived age and predicted age of ±2.8 and 2.9 yr, respectively. We further profile blood transcriptomes from 280 individuals and infer the molecular regulators mediating the impact of lifestyle on the facial-ageing rate through a causal-inference model. These relationships have been deposited and visualized in the Human Blood Gene Expression-3D Facial Image (HuB-Fi) database. Overall, we find that humans age at different rates both in the blood and in the face, but do so coherently and with heterogeneity peaking at middle age. Our study provides an example of how artificial intelligence can be leveraged to determine the perceived age of humans as a marker of biological age, while no longer relying on prediction errors of chronological age, and to estimate the heterogeneity of ageing rates within a population.