Face Reconstruction Method Research Articles

Shape analysis of the nasal complex among South African groups from CBCT scans

Three-dimensional (3D) anatomical extraction techniques could help the forensic anthropologist in a precise and inclusive assessment of biological phenotypes for the development of facial reconstruction methods. In this research, the nose morphology and the underlying hard tissue of two South African populations were studied. To this end, a 3D computer-assisted approach based on an automated landmarking workflow was used to generate relevant 3D anatomical components, and shape discrepancies were investigated using a data set of 200 cone-beam computer tomography (CBCT) scans. The anatomical landmarks were placed on the external nose and the mid-facial skeleton (the nasal bones, the anterior nasal aperture, the zygoma, and the maxilla). Shape differences related to population affinity, sex, age, and size were statistically evaluated and visualised using geometric morphometric methods. Population affinity, sexual dimorphism, age, and size affect the nasal complex morphology. Shape variation in the mid-facial region was significantly influenced by population affinity, emphasising that shape variability was specific to the two population groups, along with the expression of sexual dimorphism and the effect of ageing. In addition, nasal complex shape and correlations vary greatly between white and black South Africans, highlighting a need for reliable population-specific 3D statistical nose prediction algorithms.

Self-supervised single-image 3D face reconstruction method based on attention mechanism and attribute refinement

Self-supervised single-image 3D face reconstruction method based on attention mechanism and attribute refinement

Versatile Use of V-Y Advancement Flap for Facial Defect Reconstruction: A Case Series in Asian Patients.

V-Y advancement flap (VYAF) is a commonly used flap for facial reconstruction, but it is not popular in Asian society with limited aesthetic outcome evaluation. To demonstrate our experience of facial VYAF with the quantitative aesthetic outcome assessment. From January 2013 to December 2022, patients who underwent facial VYAF reconstruction were reviewed. Postoperative photographs were collected and independently graded by three plastic surgeons, three nurses, and six non-medical personnel using Manchester scar scale (MSS). The representative preoperative images were selected for surgeons' reconstruction preferences survey. Forty-eight patients (27 females and 21 males), with a mean age of 66.8 (23-97) years, were included in this study. All flaps survived with no flap necrosis. Only six patients (12.5%) developed minor postoperative complications, and they were treated conservatively and resolved uneventfully. The total MSS score was 7.8 ± 1.9 (scale of 4 [best scar] to 24 [worst scar]) and the overall scar VAS rating was 1.9 ± 1.1 (0 [best scar] to 10 [worst scar]), indicating satisfactory postoperative scar condition. From the survey of 22 plastic surgeons and 11 scenarios, VYAF was rarely chosen among other local flaps which only accounted for 8.7%. VYAF is an easy and safe method for facial reconstruction with low morbidity, but its usefulness is underappreciated. With a proper design and cautious dissection, we believe that good aesthetic and functional outcomes can be achieved with VYAF. This journal requires that authors assign a level of evidence to each article. For a full description of these Evidence-Based Medicine ratings, please refer to the Table of Contents or the online Instructions to Authors www.springer.com/00266 .

IMetaTown: A Metaverse System with Multiple Interactive Functions Based on Virtual Reality.

This work aims to pioneer the development of a real-time interactive and immersive Metaverse Human-Computer Interaction (HCI) system leveraging Virtual Reality (VR). The system incorporates a three-dimensional (3D) face reconstruction method, grounded in weakly supervised learning, to enhance player-player interactions within the Metaverse. The proposed method, two-dimensional (2D) face images, are effectively employed in a 2D Self-Supervised Learning (2DASL) approach, significantly optimizing 3D model learning outcomes and improving the quality of 3D face alignment in HCI systems. The work outlines the functional modules of the system, encompassing user interactions such as hugs and handshakes and communication through voice and text via blockchain. Solutions for managing multiple simultaneous online users are presented. Performance evaluation of the HCI system in a 3D reconstruction scene indicates that the 2DASL face reconstruction method achieves noteworthy results, enhancing the system's interaction capabilities by aiding 3D face modeling through 2D face images. The experimental system achieves a maximum processing speed of 18 frames of image data on a personal computer, meeting real-time processing requirements. User feedback regarding social acceptance, action interaction usability, emotions, and satisfaction with the VR interactive system reveals consistently high scores. The designed VR HCI system exhibits outstanding performance across diverse applications.

FM-3DFR: Facial Manipulation-Based 3-D Face Reconstruction.

3-D Morphable model (3DMM) has widely benefited 3-D face-involved challenges given its parametric facial geometry and appearance representation. However, previous 3-D face reconstruction methods suffer from limited power in facial expression representation due to the unbalanced training data distribution and insufficient ground-truth 3-D shapes. In this article, we propose a novel framework to learn personalized shapes so that the reconstructed model well fits the corresponding face images. Specifically, we augment the dataset following several principles to balance the facial shape and expression distribution. A mesh editing method is presented as the expression synthesizer to generate more face images with various expressions. Besides, we improve the pose estimation accuracy by transferring the projection parameter into the Euler angles. Finally, a weighted sampling method is proposed to improve the robustness of the training process, where we define the offset between the base face model and the ground-truth face model as the sampling probability of each vertex. The experiments on several challenging benchmarks have demonstrated that our method achieves state-of-the-art performance.

3D face reconstruction from a single image based on hybrid-level contextual information with weak supervision

Currently, deep learning-based 3D face reconstruction methods have shown promising results. However, they ignore the contextual information of the face, which is a topologically unified entirety. This paper proposes a 3D face reconstruction approach based on hybrid-level contextual information. Firstly, we suggest a regression network with contextual modeling capability at the feature level, PPR-CNet, which adopts a preferential parameter regression to regress the 3DMM parameters dynamically based on their various impacts on the reconstructed 3D face. Furthermore, we design a contextual landmark loss to constrain the face geometry at the landmark level. We introduce a differentiable renderer combined with multiple loss functions for weakly-supervised training. Quantitative experiments on two benchmarks show our method outperforms several SOTA methods. Extensive qualitative experiments indicate that our method performs efficiently in realism, facial proportion, and occlusion.

A Case of Systemic Connective Tissue Disorder in a Pazyryk Male from the Altai Mountains

We present the results of an anthropological and paleopathological study of a male skeleton from a Pazyryk kurgan 1 burial 1 at Ak-Alakha-1, the Altai Mountains. The archaeological context testifi es to a rather high social status of this person. His appearance was modeled using a modern facial reconstruction method. The combination of craniometric and anthropometric traits indicates a brachycranic Caucasoid type, which predominated in the male population of the Volga-Ural region and Western Kazakhstan in 600–200 BC. The individual suffered from a systemic disorder of the connective tissue affecting virtually all parts of the skeleton. The case is unique, and the condition is diagnosed as DISH—diffuse idiopathic skeletal hyperostosis, possibly caused by grave metabolic and endocrine disorders. Judging by his physical type, the male was not native to the Altai Mountains and belonged to a population that was not genetically adapted to that region. This may account for the severity of the rheumatoid disorder, which was aggravated by 4th degree osteoporosis of lumbar vertebrae and pelvic bones and by traumatic lesions of the shoulder girdle.

A Lightweight Monocular 3D Face Reconstruction Method Based on Improved 3D Morphing Models.

In the past few years, 3D Morphing Model (3DMM)-based methods have achieved remarkable results in single-image 3D face reconstruction. However, high-fidelity 3D face texture generation has been successfully achieved with this method, which mostly uses the power of deep convolutional neural networks during the parameter fitting process, which leads to an increase in the number of network layers and computational burden of the network model and reduces the computational speed. Currently, existing methods increase computational speed by using lightweight networks for parameter fitting, but at the expense of reconstruction accuracy. In order to solve the above problems, we improved the 3D deformation model and proposed an efficient and lightweight network model: Mobile-FaceRNet. First, we combine depthwise separable convolution and multi-scale representation methods to fit the parameters of a 3D deformable model (3DMM); then, we introduce a residual attention module during network training to enhance the network's attention to important features, guaranteeing high-fidelity facial texture reconstruction quality; and, finally, a new perceptual loss function is designed to better address smoothness and image similarity for the smoothing constraints. Experimental results show that the method proposed in this paper can not only achieve high-precision reconstruction under the premise of lightweight, but it is also more robust to influences such as attitude and occlusion.

JR2Net: Joint Monocular 3D Face Reconstruction and Reenactment

Face reenactment and reconstruction benefit various applications in self-media, VR, etc. Recent face reenactment methods use 2D facial landmarks to implicitly retarget facial expressions and poses from driving videos to source images, while they suffer from pose and expression preservation issues for cross-identity scenarios, i.e., when the source and the driving subjects are different. Current self-supervised face reconstruction methods also demonstrate impressive results. However, these methods do not handle large expressions well, since their training data lacks samples of large expressions, and 2D facial attributes are inaccurate on such samples. To mitigate the above problems, we propose to explore the inner connection between the two tasks, i.e., using face reconstruction to provide sufficient 3D information for reenactment, and synthesizing videos paired with captured face model parameters through face reenactment to enhance the expression module of face reconstruction. In particular, we propose a novel cascade framework named JR2Net for Joint Face Reconstruction and Reenactment, which begins with the training of a coarse reconstruction network, followed by a 3D-aware face reenactment network based on the coarse reconstruction results. In the end, we train an expression tracking network based on our synthesized videos composed by image-face model parameter pairs. Such an expression tracking network can further enhance the coarse face reconstruction. Extensive experiments show that our JR2Net outperforms the state-of-the-art methods on several face reconstruction and reenactment benchmarks.

Mikhail Mikhailovich Gerasimov (1907–1970) and his method of face reconstruction from the skull

The article is dedicated to the 115th anniversary of the birth of Mikhail Mikhailovich Gerasimov, a world-famous scientist, pioneer of historical and forensic portrait reconstruction, founder of the Plastic Reconstruction Laboratory at the Institute of Ethnography of the Academy of Sciences of the Soviet Union (currently the Institute of Ethnology and Anthropology of the Russian Academy of Sciences). M.M. Gerasimov created the most famous gallery of sculptural portraits of a number of historical persons, and also recreated the appearances of fossil peo-ple from the available bone remains. A man of encyclopedic knowledge, anthropologist, archaeologist and sculp-tor rolled into one, Gerasimov laid the foundations of craniofacial reconstruction and created his own scientific school. For his invaluable contribution to science and practical anthropology, as well as to forensic medicine and forensic science, the scientist was awarded the State Prize in 1950, as well as the title of Honored Scientist of the RSFSR in 1969. The method of facial reconstruction developed by Gerasimov formed the basis of modern meth-ods equipped with advanced imaging and computer technologies. The laboratory created by M.M. Gerasimov exists and is actively functioning up to the present days, and his students successfully continue the work he started. For decades, the authority of M.M. Gerasimov in anthropological reconstruction has remained unshakable, the method of reconstruction of the appearance of fossil people, developed by the scientist, has received world-wide recognition, and it is fundamental for modern researchers. Followers of M.M. Gerasimov continue to develop and improve the method, which is of paramount importance in paleoanthropology, ethnography, historical sci-ences, and which has found practical application in forensic science, forensic medicine and plastic surgery.

Facial Photo-Guided Head Anatomy Modeling Based on Deep Learning and 2D/3D Shape Prior Model Registration

Three-dimensional (3D) models of individualized head anatomy are often used in the preoperative planning of plastic surgery. Most current methods use Computed Tomography (CT) to acquire the head anatomy, but CT scan exposes the patients to X-ray radiation. Alternatively, artificial intelligence-based estimation of 3D head anatomy from the facial photo is a safer and more efficient choice, especially for scenarios that do not require high modeling accuracy of internal structures (e.g., doctor-patient communication and surgery procedure demonstration). We develop a method for constructing a personalized 3D model of complete head anatomy (including skin and internal structures) from the front and side view photos. We first detect key facial landmarks from the photos with a convolution neural network (CNN), then perform 2D/3D registration to morph a statistical shape model (SSM) of complete head anatomy to match the dual-view photos. Our method uses deep learning to achieve accurate facial landmark detection and employ the anatomy shape prior to yielding reasonable internal structure estimation. We evaluated the method based on 12 subjects (including 7 males and 5 females). The facial surface reconstruction error was assessed using the 3D surface scan of the subjects as the ground truth. Our method’s root-mean-square-error (RMSE) was 3.60±0.49 mm, which was 1.06±0.39 mm lower than the state-of-the-art (SOTA) CNN-based face reconstruction method. Our method also predicts all the internal head structures (bones, muscles, vessels, nerve fibers, fat, glands, and brain structures) which are not provided by the SOTA method. To the best of our knowledge, this is the first study modeling complete head anatomy from facial photos toward clinical applications.

Using an Expanded Cervical Flap With an Overlapping Tissue Expansion Technique to Resurface Middle and Lower Facial Defects.

Extensive facial burn scars are a tragedy for patients and often pose a great challenge to surgeons because of the high esthetic and functional demands. For patients with healthy skin in the neck region, a cervical flap is highly recommended for facial resurfacing; however, the skin on the midline of the neck often needs more expansion than that on either side, especially for the treatment of large facial defects. The sufficient longitudinal soft tissue in the anterior neck ensures a normal neck shape as well as a normal range of cervical extension, rotation, and lateral flexion. To overcome this, we developed an expanded cervical flap with an overlapping tissue expansion technique to gain more length centrally. First, 2 tissue expanders were embedded in the anterior neck region overlapping each other at the midline of the neck. After adequate inflation of the expander, the expanded flap was dissected and rotated to repair defects in the middle and lower face. The anchor position of the flap was placed on the horizontal line of the thyroid cartilage to restore the cervicomental angle. Sixteen patients were treated with this method in this single-center study. All defects affected the middle and lower face, with an area ranging from 135 to 185cm 2 , and were caused by a massive facial burn. Among them, 12 patients suffered ectropion of the lower lip, 3 suffered limited mouth opening due to scar contraction, and one patient had a cervicomental adhesion. The area of the expanded flap was approximately 163 to 266cm 2 . The average period of expansion was 89.5 days. Patients were followed up after the operation, with the follow-up period ranging from 6 to 12 months. In all cases, good defect coverage was achieved, with primary closure of the donor sites and a good postoperative cervical configuration. We conclude that the expanded cervical flap with the overlapping tissue expansion technique proved to be a reliable method for facial skin reconstruction with functional and aesthetic improvement.

The Posterior Belly of Digastric Muscle as the Landmark in Facial Nerve Anastomosis Surgery: Anatomical Study and Case Illustration

To establish a new method for fast exposure of the facial nerve and hypoglossal nerve during facial nerve anastomosis surgery. Dissection of 12 formalin-fixed cadaveric specimens was performed to explore the positional relationship between the posterior belly of digastric muscle (PBD) and the facial nerve and hypoglossal nerve. We retrospectively reviewed patients who underwent facial nerve reconstruction surgery between 2015 and 2020 at Xuanwu Hospital, Capital Medical University, and the optimized surgical strategy based on the PBD was proposed. The trunk of the hypoglossal nerve runs across the external carotid artery after giving off the descendens hypoglossi located within the 1-cm scope deep to the junction of the tendon and belly of the PBD. The mean depth difference between the hypoglossal nerve and the junction of the tendon and belly of the PBD was 5.48 ± 2.24 mm (range, 1.88-9.27 mm). The stylomastoid foramen segment of the facial nerve was revealed after the parotid gland was dissected within the angle between the anterior margin of the mastoid tip and the inferior margin of the cartilage of the external acoustic meatus. The facial nerve and hypoglossal nerve can be rapidly identified using the PBD as an anatomical landmark. The end-to-end facial-descendens hypoglossi anastomosis is a reliable facial nerve reconstruction method for patients whose facial nerve was damaged during operation.

Transformer-Based 3D Face Reconstruction With End-to-End Shape-Preserved Domain Transfer

Learning-based face reconstruction methods have recently shown promising performance in recovering face geometry from a single image. However, the lack of training data with 3D annotations severely limits the performance. To tackle this problem, we proposed a novel end-to-end 3D face reconstruction network consisting of a conditional GAN (cGAN) for cross-domain face synthesis and a novel mesh transformer for face reconstruction. Our method first uses cGAN to translate the realistic face images to the specific rendered style, with a 2D facial edge consistency loss function. The domain-transferred images are then fed into face reconstruction network which uses a novel mesh transformer to output 3D mesh vertices. To exploit the domain-transferred in-the-wild images, we further propose a reprojection consistency loss to restrict face reconstruction network in a self-supervised way. Our approach can be trained with annotated dataset, synthetic dataset and in-the-wild images to learn a unified face model. Extensive experiments have demonstrated the effectiveness of our method.

Realistic 3D Face Reconstruction Method for Single Image

To solve the problem that the 3DMM parameter fitting methods cannot generate realistic 3D face, a single-image realistic 3D face reconstruction method based on deep learning is proposed. Firstly, the RP-Net regression network is constructed, and a dataset containing 50 000 face images is constructed at the same time. The parameters are learned from the input images, and the face model is fitted to generate the 3D face geometry. Secondly, weakly supervised learning is performed by constructing a multi-level loss function, which includes low-level pixel loss, landmark loss, and high-level identity loss. Thirdly, a realistic face texture is generated by means of texture mapping. Finally, two real face data and one generated data are used to compare experiments with recent 3D face reconstruction methods. The factors affecting the reconstruction such as lighting, expression, and steering are used to test proposed method, and quantitatively evaluate the reconstruction by SSIM and PSNR. These results show that proposed method can generate accurate face shapes and realistic face textures. Compared with the recent 3D face reconstruction method, the training time and number of iterations of the proposed method are reduced by 6% and 13%, respectively, the SSIM value is increased by 0.005‒0.010, and the PSNR value is increased by 0.03‒0.08 dB on average.

Accurate Passive 3D Polarization Face Reconstruction under Complex Conditions Assisted with Deep Learning

Accurate passive 3D face reconstruction is of great importance with various potential applications. Three-dimensional polarization face reconstruction is a promising approach, but one bothered by serious deformations caused by an ambiguous surface normal. In this study, we propose a learning-based method for passive 3D polarization face reconstruction. It first calculates the surface normal of each microfacet at a pixel level based on the polarization of diffusely reflected light on the face, where no auxiliary equipment, including artificial illumination, is required. Then, the CNN-based 3DMM (convolutional neural network; 3D morphable model) generates a rough depth map of the face with the directly captured polarization image. The map works as an extra constraint to correct the ambiguous surface normal obtained from polarization. An accurate surface normal finally allows for an accurate 3D face reconstruction. Experiments in both indoor and outdoor conditions demonstrate that accurate 3D faces can be well-reconstructed. Moreover, with no auxiliary equipment required, the method ensures a total passive 3D face reconstruction.

Video-Driven Neural Physically-Based Facial Asset for Production

Production-level workflows for producing convincing 3D dynamic human faces have long relied on an assortment of labor-intensive tools for geometry and texture generation, motion capture and rigging, and expression synthesis. Recent neural approaches automate individual components but the corresponding latent representations cannot provide artists with explicit controls as in conventional tools. In this paper, we present a new learning-based, video-driven approach for generating dynamic facial geometries with high-quality physically-based assets. For data collection, we construct a hybrid multiview-photometric capture stage, coupling with ultra-fast video cameras to obtain raw 3D facial assets. We then set out to model the facial expression, geometry and physically-based textures using separate VAEs where we impose a global MLP based expression mapping across the latent spaces of respective networks, to preserve characteristics across respective attributes. We also model the delta information as wrinkle maps for the physically-based textures, achieving high-quality 4K dynamic textures. We demonstrate our approach in high-fidelity performer-specific facial capture and cross-identity facial motion retargeting. In addition, our multi-VAE-based neural asset, along with the fast adaptation schemes, can also be deployed to handle in-the-wild videos. Besides, we motivate the utility of our explicit facial disentangling strategy by providing various promising physically-based editing results with high realism. Comprehensive experiments show that our technique provides higher accuracy and visual fidelity than previous video-driven facial reconstruction and animation methods.

Robust and High-Fidelity 3D Face Reconstruction Using Multiple RGB-D Cameras

In this paper, we propose a robust and high-fidelity 3D face reconstruction method that uses multiple depth cameras. This method automatically reconstructs high-quality 3D face models from aligned RGB-D image pairs using multi-view consumer-grade depth cameras. To this end, we mainly analyze the problems in existing traditional and classical multi-view 3D face reconstruction systems and propose targeted improvement strategies for the issues related. In particular, we propose a fast two-stage point cloud filtering method that combines coarse filtering and fine filtering to rapidly extract the reconstructed subject point cloud with high purity. Meanwhile, in order to improve the integrity and accuracy of the point cloud for reconstruction, we propose a depth data restoration and optimization method based on the joint space–time domain. In addition, we also propose a method of multi-view texture alignment for the final texture fusion session that is more conducive for fusing face textures with better uniformity and visual performance. The above-proposed methods are reproducible and can be extended to the 3D reconstruction of any subject. The final experimental results show that the method is able to robustly generate 3D face models having high geometric and visual quality.

ГАЛЕРЕЯ ПОРТРЕТОВ, ВЫПОЛНЕННЫХ ПО ЧЕРЕПАМ ЭСКИМОСОВ, В ЛАБОРАТОРИИ АНТРОПОЛОГИЧЕСКОЙ РЕКОНСТРУКЦИИ

The Laboratory of Anthropological Reconstruction (LAR) of the Centerfor Physical Anthropology (CPhA) of the Institute of Ethnology and Anthropology of the Russian Academy of Sciences has a truly unique collection of graphic and sculptural portraits made from the skulls of ancient and modern Eskimos from the Old and New Worlds. The article presents the works of G.V. Lebedinskaya, T.S. Surnina, T.S. Baluyeva, as well as some reconstructions made more recently by todays LAR research fellows. These latest works were carried out, taking into account the new data based on the improvement of the method of graphic facial reconstruction from a skull. The processing technique for 3D reconstruction of skull, based on a series of photographs taken from different angles is described. Thanks to this method, it is possible to obtain accurate images of the skull from front and side profile without any perspective distortion. The presented recon-structions were made using both classical and new methods, the last ones, based on the use of computer and the Photoshop Software. An innovative way of rep-resenting lifelike facial features makes it possible to show the real color of the skin, hair and eyes. In the article is as well discussed the place of the Eskimos in the anthropological classification system and the peculiarities of their features in Greenland and Chukotka.

The accuracy of a three-dimensional face model reconstructing method based on conventional clinical two-dimensional photos

BackgroundThis study aims to investigate the accuracy of a three-dimensional (3D) face reconstruction method based on conventional clinical two-dimensional (2D) photos.MethodsTwenty-three patients were included, and Character Creator v3.2 software with the Headshot v1.0 plugin was used for 3D face model reconstruction. Various facial landmarks were finely adjusted manually to refine the models. After preprocessing and repositioning, 3D deviation analysis was performed. The accuracy of the landmarks in different dimensions was determined, and twelve facial soft tissue measurements were compared to validate the clinical potential of the method.ResultThe reconstructed 3D face models showed good facial morphology with fine texture. The average root mean square errors between face scan models and reconstructed models at perioral area (1.26 ± 0.24 mm, 95%CI: 1.15–1.37 mm) were significantly smaller than the entire facial area (1.77 ± 0.23 mm, 95%CI:1.67–1.88 mm), P < 0.01. The deviation of menton of soft tissue was significantly larger than pronasale (P < 0.01). The deviations of all landmarks in the Y-direction were significantly larger than those in the other 2 dimensions (Y > Z > X, P < 0.01). A significant difference (P < 0.05) of approximately 1.5 mm was found for facial height. Significant differences (P < 0.05) were also identified in the remaining 6 soft tissue measurements, with average deviations no greater than 0.5 mm (linear measurement) or 1.2° (angular measurements).ConclusionA 3D face modeling method based on 2D face photos was revealed and validated. The reconstruction accuracy of this method is clinically acceptable for orthodontic measurement purposes, but narrow clinical indications and labor-intensive operations remain problems.