3D Face Research Articles

Stability analysis of a 3D shallow tunnel face considering the inclined excavation and inclined ground surface

In tunnel engineering, the phenomena of inclined excavation and inclined ground surface for mountain tunnels, urban tunnels and cross-river (sea) tunnels are common due to the constraints of landform, design and construction. Inclined excavation and inclined ground surface have significant influences on the stability of shallow tunnel faces. However, their comprehensive influences on the active and passive failure modes of a shallow tunnel face need to be further studied. An improved three-dimensional failure model of a shallow tunnel face with multi-blocks is constructed in this study while considering their comprehensive influences, and the reliability of active and passive failure models is verified through comparative analysis. In addition, a series of detailed parametric analyses and key failure modes are carried out. The tunnel inclined angle (δ1) and inclined angle (δ2) of the ground surface have significant influences on the shallow tunnel face stability for active and passive failure modes. The percentages of change in the ground surface failure region are 47.81% for active failure and 5.82% for passive failure when δ1 varies from −10° to 10°, and 24.44% and 18.57%, respectively, when δ2 varies from −10° to 10°. In practical engineering, δ1 and δ2 must be controlled to within a certain range because a large δ1 can easily induce active failure, and a large δ2 can easily induce passive failure of a 3D shallow tunnel face.

Robust 3D face recognition in unconstrained environment using distance based ternary search siamese network

Robust 3D face recognition in unconstrained environment using distance based ternary search siamese network

Understanding the contagiousness of Covid-19 strains: A geometric approach

Protein–protein interaction occurs on surface patches with some degree of complementary geometric and chemical features. Building on this understanding, this study endeavors to characterize the spike protein of the SARS-CoV-2 virus at the morphological and geometrical levels in its Alpha, Delta, and Omicron variants. In particular, the affinity between different SARS-CoV-2 spike proteins and the ACE2 receptor present on the membrane of the human respiratory system cells is investigated. To achieve an adequate degree of geometrical accuracy, the 3D depth maps of the proteins in exam are filtered by developing an ad-hoc convolutional filter with a kernel implemented as a sphere of varying radius, simulating a ball rolling on the surface (similar to the 'rolling ball' filter). This ball ideally models a hypothetical molecule that could interface with the protein and is inspired by the geometric approach to macromolecule–ligand interactions proposed by Kuntz et al. in 1982. The aim is to mitigate the imperfections and to obtain a smoother surface that could be studied from a geometrical perspective for binding purposes. A set of geometric descriptors, borrowed from the 3D face analysis context is then mapped point-by-point onto protein depth maps. Following a feature extraction phase inspired by Histogram of Oriented Gradients and Local Binary Patterns, the final histogram features are used as input for a Support Vector Machine classifier to automatically classify the proteins according to their surface affinity, where a similarity in shape is observed between ACE2 and the spike protein of the SARS-CoV-2 Omicron variant. Finally, Root Mean Square Error analysis is used to quantify the geometrical affinity between the ACE2 receptor and the respective Receptor Binding Domains of the three SARS-CoV-2 variants, culminating in a geometrical explanation for the higher contagiousness of Omicron relative to the other variants under study.

Measuring the Consistency and Diversity of 3D Face Generation

3D-aware GANs have shown their impressive power on 3D controlling for synthesized portraits. While the plausible facial reality is achieved, the inherent 3D properties of the generated results have actually not been well analyzed. One of the reasons is that the wildly-used metrics, such as Inception Score (IS) or Fréchet Inception Distance (FID), focus more on the perceptual features rather than explicit 3D clues. In this paper, we propose two novel 3D metrics, which measure the face consistency and diversity on a 3D level, to compensate for IS or FID on GAN evaluation. With such metrics, the generated faces are systematically analyzed on a wide range of GANs, where we obtain reasonable but different conclusions from the wildly-used approaches. Inspired by the performance on the proposed metrics, we then discuss what is crucial for the GANs to get robust 3D properties, and what may bring uncertainty or inconsistency to the generating procedure. Finally, we propose two simple but efficient solutions to contribute to superior 3D synthesizing accuracy across different architectures, which further demonstrate the significance of the proposed metrics.

VTO Model Tester with MindAR Library

Augmented Reality is not a new technology, with more than 30 years of age, AR, which used to require large resources, can now even be run using a website application. The advantage of using a website is that the website application can be run on almost any device that supports a web browser. Web AR can also be used as a trial tool that can be used by customers to try on clothes or other products which is also known as VTO or Virtual Try On. The problem that arises from VTO Application is from the 3D designer side, which must make 3D Object and always readjust if the 3D is not suitable for VTO that requires accuracy of shape and size, that must be adjusted between 3D objects and human faces. The focus of this research is creating a website application that can connect to the internet so that 3D designers can easily access the VTO tester from any device to test 3D objects that have been created for Virtual Try On. The result of this research is that we can use local storage as a temporary storage medium and proof that local storage cannot also be used to store documents but also to store 3D objects. By immediately deleting temporary storage in local storage, the storage quota will not increase so that our local storage drives are not affected, the memory usage of only 10MB is also an advantage of this VTO Tester application, because it does not require a large memory to run the VTO Tester. In terms of memory usage, there is no memory leak that results when using VTO Tester application, this proves that the application developed can be used repeatedly without worrying about burdened computer performance.

Unsupervised performance analysis of 3D face alignment with a statistically robust confidence test

This paper addresses the problem of analyzing the performance of 3D face alignment (3DFA), or facial landmark localization. This task is usually supervised, based on annotated datasets. Nevertheless, in the particular case of 3DFA, the annotation process is rarely error-free, which strongly biases the results. Alternatively, unsupervised performance analysis (UPA) is investigated. The core ingredient of the proposed methodology is the robust estimation of the rigid transformation between predicted landmarks and model landmarks. It is shown that the rigid mapping thus computed is affected neither by non-rigid facial deformations, due to variabilities in expression and in identity, nor by landmark localization errors, due to various perturbations. The guiding idea is to apply the estimated rotation, translation and scale to a set of predicted landmarks in order to map them onto a mathematical home for the shape embedded in these landmarks (including possible errors). UPA proceeds as follows: (i) 3D landmarks are extracted from a 2D face using the 3DFA method under investigation; (ii) these landmarks are rigidly mapped onto a canonical (frontal) pose, and (iii) a statistically-robust confidence score is computed for each landmark. This allows to assess whether the mapped landmarks lie inside (inliers) or outside (outliers) a confidence volume. An experimental evaluation protocol, that uses publicly available datasets and several 3DFA software packages associated with published articles, is described in detail. The results show that the proposed analysis is consistent with supervised metrics and that it can be used to measure the accuracy of both predicted landmarks and of automatically annotated 3DFA datasets, to detect errors and to eliminate them. Source code and supplemental materials for this paper are publicly available at https://team.inria.fr/robotlearn/upa3dfa/.

Analysis of face recognition technology based on deep learning

The emergence of face recognition technology has brought great convenience to human society. After years of research and improvement, face recognition technology has matured. This paper mainly studies face recognition technology based on deep learning, and introduces five main applications, including occlusion face recognition, 3D face recognition, and expression recognition. The typical algorithms listed show that, compared with the traditional feature face algorithms and local binary patterns, algorithms such as DeepFace, FaceNet, ResNet, etc. have made significant progress in recognition efficiency and accuracy in recent years. After combining the development status of face recognition technology with the problems and security risks of the current technology, this paper proposes three solutions: increasing the amount of data, optimizing the model, and government control. In the future, legal data sharing and code open source will be major progress in this field, and dynamic face recognition technology will also be widely used.

3D Face Reconstruction: The Road to Forensics

3D face reconstruction algorithms from images and videos are applied to many fields, from plastic surgery to the entertainment sector, thanks to their advantageous features. However, when looking at forensic applications, 3D face reconstruction must observe strict requirements that still make its possible role in bringing evidence to a lawsuit unclear. An extensive investigation of the constraints, potential, and limits of its application in forensics is still missing. Shedding some light on this matter is the goal of the present survey, which starts by clarifying the relation between forensic applications and biometrics, with a focus on face recognition. Therefore, it provides an analysis of the achievements of 3D face reconstruction algorithms from surveillance videos and mugshot images and discusses the current obstacles that separate 3D face reconstruction from an active role in forensic applications. Finally, it examines the underlying datasets, with their advantages and limitations, while proposing alternatives that could substitute or complement them.

3D head-talk: speech synthesis 3D head movement face animation

3D head-talk: speech synthesis 3D head movement face animation

Automated anomalous child repetitive head movement identification through transformer networks.

The increasing prevalence of behavioral disorders in children is of growing concern within the medical community. Recognising the significance of early identification and intervention for atypical behaviors, there is a consensus on their pivotal role in improving outcomes. Due to inadequate facilities and a shortage of medical professionals with specialized expertise, traditional diagnostic methods have been unable to effectively address the rising incidence of behavioral disorders. Hence, there is a need to develop automated approaches for the diagnosis of behavioral disorders in children, to overcome the challenges with traditional methods. The purpose of this study is to develop an automated model capable of analyzing videos to differentiate between typical and atypical repetitive head movements in. To address problems resulting from the limited availability of child datasets, various learning methods are employed to mitigate these issues. In this work, we present a fusion of transformer networks, and Non-deterministic Finite Automata (NFA) techniques, which classify repetitive head movements of a child as typical or atypical based on an analysis of gender, age, and type of repetitive head movement, along with count, duration, and frequency of each repetitive head movement. Experimentation was carried out with different transfer learning methods to enhance the performance of the model. The experimental results on five datasets: NIR face dataset, Bosphorus 3D face dataset, ASD dataset, SSBD dataset, and the Head Movements in the Wild dataset, indicate that our proposed model has outperformed many state-of-the-art frameworks when distinguishing typical and atypical repetitive head movements in children.

Anime-like Character Face Generation: A Survey

Recently, anime-style modeling has become increasingly popular. Automatic anime character generation can reduce costs and save manpower. There have been numerous related studies that need to be summarized and analyzed. This paper presents an overview of anime character face generation. Firstly, it summarizes the types and methods of 2D face generation for anime characters. Secondly, we introduce methods for anime character face modeling, including deep learning-based approaches and other approaches. Finally, we summarize the challenges of anime character face generation and evaluate their performance.

The Florence multi-resolution 3D facial expression dataset

In the literature, several 3D face datasets have been collected, aiming at advancing the field of 3D face analysis from different perspectives. Data collection generally follows specific research needs, and the existing 3D face datasets all have different characteristics that are tailored for investigating different tasks, encompassing face recognition, facial expressions and emotions analysis, 3D face reconstruction. However, the majority of these datasets are either collected with high-resolution scanners, or consumer level devices, such as the Kinect, the latter being motivated by the burdensome and costly process of collecting high-quality scans. Differently from 2D imagery, the difference in resolution in 3D data represents a non negligible problem that is under-investigated, and still prevents the successful development of methods that can work in real scenarios. In this paper, we propose a new 3D face dataset, named “Florence Multi-Resolution 3D Facial Expression” (Florence 3DMRE), which aims at bridging the gap between high- and low-resolution 3D face datasets. Its peculiarity consists in (1) including high-resolution (HR) models obtained with a HR scanner, and paired samples collected with a Kinect sensor, (2) LR and HR scans are synchronized and capture extreme and asymmetric facial deformations as used in facial rehabilitation exercises. In total, our dataset consists of 14 subjects, each performing 19 complex and asymmetric expressions. For each of them, we collected a high-resolution scan, and an RGB-D sequence. Finally, to highlight the value of the dataset and the challenges it introduces, we use the collected data to perform baseline experiments for cross-resolution 3D face recognition and reconstruction. The dataset is released for research purposes only, and complies to GDPR for data treatment. The dataset can be found at this link.

Research on Fatigued-Driving Detection Method by Integrating Lightweight YOLOv5s and Facial 3D Keypoints.

In response to the problem of high computational and parameter requirements of fatigued-driving detection models, as well as weak facial-feature keypoint extraction capability, this paper proposes a lightweight and real-time fatigued-driving detection model based on an improved YOLOv5s and Attention Mesh 3D keypoint extraction method. The main strategies are as follows: (1) Using Shufflenetv2_BD to reconstruct the Backbone network to reduce parameter complexity and computational load. (2) Introducing and improving the fusion method of the Cross-scale Aggregation Module (CAM) between the Backbone and Neck networks to reduce information loss in shallow features of closed-eyes and closed-mouth categories. (3) Building a lightweight Context Information Fusion Module by combining the Efficient Multi-Scale Module (EAM) and Depthwise Over-Parameterized Convolution (DoConv) to enhance the Neck network's ability to extract facial features. (4) Redefining the loss function using Wise-IoU (WIoU) to accelerate model convergence. Finally, the fatigued-driving detection model is constructed by combining the classification detection results with the thresholds of continuous closed-eye frames, continuous yawning frames, and PERCLOS (Percentage of Eyelid Closure over the Pupil over Time) of eyes and mouth. Under the premise that the number of parameters and the size of the baseline model are reduced by 58% and 56.3%, respectively, and the floating point computation is only 5.9 GFLOPs, the average accuracy of the baseline model is increased by 1%, and the Fatigued-recognition rate is 96.3%, which proves that the proposed algorithm can achieve accurate and stable real-time detection while lightweight. It provides strong support for the lightweight deployment of vehicle terminals.

3D Face Recognition: Two Decades of Progress and Prospects

Three-dimensional (3D) face recognition has been extensively investigated in the last two decades due to its wide range of applications in many areas, such as security and forensics. Numerous methods have been proposed to deal with the challenges faced by 3D face recognition, such as facial expressions, pose variations, and occlusions. These methods have achieved superior performances on several small-scale datasets, including FRGC v2.0, Bosphorus, BU-3DFE, and Gavab. However, deep learning–based 3D face recognition methods are still in their infancy due to the lack of large-scale 3D face datasets. To stimulate future research in this area, we present a comprehensive review of the progress achieved by both traditional and deep learning–based 3D face recognition methods in the last two decades. Comparative results on several publicly available datasets under different challenges of facial expressions, pose variations, and occlusions are also presented.

A Perceptual Shape Loss for Monocular 3D Face Reconstruction

AbstractMonocular 3D face reconstruction is a wide‐spread topic, and existing approaches tackle the problem either through fast neural network inference or offline iterative reconstruction of face geometry. In either case carefully‐designed energy functions are minimized, commonly including loss terms like a photometric loss, a landmark reprojection loss, and others. In this work we propose a new loss function for monocular face capture, inspired by how humans would perceive the quality of a 3D face reconstruction given a particular image. It is widely known that shading provides a strong indicator for 3D shape in the human visual system. As such, our new ‘perceptual’ shape loss aims to judge the quality of a 3D face estimate using only shading cues. Our loss is implemented as a discriminator‐style neural network that takes an input face image and a shaded render of the geometry estimate, and then predicts a score that perceptually evaluates how well the shaded render matches the given image. This ‘critic’ network operates on the RGB image and geometry render alone, without requiring an estimate of the albedo or illumination in the scene. Furthermore, our loss operates entirely in image space and is thus agnostic to mesh topology. We show how our new perceptual shape loss can be combined with traditional energy terms for monocular 3D face optimization and deep neural network regression, improving upon current state‐of‐the‐art results.

Meta-evaluation for 3D Face Reconstruction Via Synthetic Data.

The standard benchmark metric for 3D face reconstruction is the geometric error between reconstructed meshes and the ground truth. Nearly all recent reconstruction methods are validated on real ground truth scans, in which case one needs to establish point correspondence prior to error computation, which is typically done with the Chamfer (i.e., nearest neighbor) criterion. However, a simple yet fundamental question have not been asked: Is the Chamfer error an appropriate and fair benchmark metric for 3D face reconstruction? More generally, how can we determine which error estimator is a better benchmark metric? We present a meta-evaluation framework that uses synthetic data to evaluate the quality of a geometric error estimator as a benchmark metric for face reconstruction. Further, we use this framework to experimentally compare four geometric error estimators. Results show that the standard approach not only severely underestimates the error, but also does so inconsistently across reconstruction methods, to the point of even altering the ranking of the compared methods. Moreover, although non-rigid ICP leads to a metric with smaller estimation bias, it could still not correctly rank all compared reconstruction methods, and is significantly more time consuming than Chamfer. In sum, we show several issues present in the current benchmarking and propose a procedure using synthetic data to address these issues.

Accuracy Comparison of 3D Face Scans Obtained by Portable Stereophotogrammetry and Smartphone Applications

Accuracy Comparison of 3D Face Scans Obtained by Portable Stereophotogrammetry and Smartphone Applications

Building 3D Generative Models from Minimal Data.

We propose a method for constructing generative models of 3D objects from a single 3D mesh and improving them through unsupervised low-shot learning from 2D images. Our method produces a 3D morphable model that represents shape and albedo in terms of Gaussian processes. Whereas previous approaches have typically built 3D morphable models from multiple high-quality 3D scans through principal component analysis, we build 3D morphable models from a single scan or template. As we demonstrate in the face domain, these models can be used to infer 3D reconstructions from 2D data (inverse graphics) or 3D data (registration). Specifically, we show that our approach can be used to perform face recognition using only a single 3D template (one scan total, not one per person). We extend our model to a preliminary unsupervised learning framework that enables the learning of the distribution of 3D faces using one 3D template and a small number of 2D images. Our approach is motivated as a potential model for the origins of face perception in human infants, who appear to start with an innate face template and subsequently develop a flexible system for perceiving the 3D structure of any novel face from experience with only 2D images of a relatively small number of familiar faces.

The Bond Length of Intermetallic Ternary Phases of Al-Fe-Si Alloy Using Molecular Dynamics Simulation with the Application of [001] Compression

The research on tolerance stress in aluminum alloys is focused on examining the mechanical behavior of τ4-Al3FeSi2 and τ12-Al3Fe2Si phases during [001] compression and their structural evolution. The use of MD computational bond length measurements allows for a comparison to be made with previous studies on tensile deformation. The simulations were performed at a constant strain rate of 21×1010 s-1, using NPT conditions (isothermal-isobaric), with approximately 20,000 atoms, 1 atmosphere of pressure, and 300 K temperature, using a Nosé-Hoover thermostat. Under periodic boundary conditions, the Modified Embedded Atoms Method (MEAM) potential was applied to all 3D faces, and the average bond length behavior between Al, Fe, and Si was calculated. A comprehensive investigation is carried out to explore the properties of these phases, including a detailed structural analysis at the atomic scale. This paper presents a comprehensive analysis of how changes in compound concentration affect mechanical behavior during compression. The average bond length varies depending on the applied stress axis, and it demonstrates good agreement with literature data. The mechanical deformations alter the behavior of atomic phases, as discussed in detail in the conclusion.

The presentation of a semi-supervised deep learning platform for 3D face reconstruction from 2D images

The presentation of a semi-supervised deep learning platform for 3D face reconstruction from 2D images