3D Face Database Research Articles

MeshWGAN: Mesh-to-Mesh Wasserstein GAN With Multi-Task Gradient Penalty for 3D Facial Geometric Age Transformation.

As the metaverse develops rapidly, 3D facial age transformation is attracting increasing attention, which may bring many potential benefits to a wide variety of users, e.g., 3D aging figures creation, 3D facial data augmentation and editing. Compared with 2D methods, 3D face aging is an underexplored problem. To fill this gap, we propose a new mesh-to-mesh Wasserstein generative adversarial network (MeshWGAN) with a multi-task gradient penalty to model a continuous bi-directional 3D facial geometric aging process. To the best of our knowledge, this is the first architecture to achieve 3D facial geometric age transformation via real 3D scans. As previous image-to-image translation methods cannot be directly applied to the 3D facial mesh, which is totally different from 2D images, we built a mesh encoder, decoder, and multi-task discriminator to facilitate mesh-to-mesh transformations. To mitigate the lack of 3D datasets containing children's faces, we collected scans from 765 subjects aged 5-17 in combination with existing 3D face databases, which provided a large training dataset. Experiments have shown that our architecture can predict 3D facial aging geometries with better identity preservation and age closeness compared to 3D trivial baselines. We also demonstrated the advantages of our approach via various 3D face-related graphics applications.

3D face recognition algorithm based on deep Laplacian pyramid under the normalization of epidemic control

Under the normalization of epidemic control in COVID-19, it is essential to realize fast and high-precision face recognition without feeling for epidemic prevention and control. This paper proposes an innovative Laplacian pyramid algorithm for deep 3D face recognition, which can be used in public. Through multi-mode fusion, dense 3D alignment and multi-scale residual fusion are ensured. Firstly, the 2D to 3D structure representation method is used to fully correlate the information of crucial points, and dense alignment modeling is carried out. Then, based on the 3D critical point model, a five-layer Laplacian depth network is constructed. High-precision recognition can be achieved by multi-scale and multi-modal mapping and reconstruction of 3D face depth images. Finally, in the training process, the multi-scale residual weight is embedded into the loss function to improve the network’s performance. In addition, to achieve high real-time performance, our network is designed in an end-to-end cascade. While ensuring the accuracy of identification, it guarantees personnel screening under the normalization of epidemic control. This ensures fast and high-precision face recognition and establishes a 3D face database. This method is adaptable and robust in harsh, low light, and noise environments. Moreover, it can complete face reconstruction and recognize various skin colors and postures.

Large-View 3D Color Face Reconstruction from Dingle Image via UV Location Map and CGAN

Image-based 3D reconstruction has been a hot topic in computer vision research. Since 3D face model can convey more information, it possesses expansive application prospects in comparison with the 2D face image, such as more precise identity information recognition signs, more accurate emotional expression media, etc. In order to reconstruct a realistic 3D color face model from a single large-view 2D face image, a simple but effective method is proposed. First, the encoder-decoder network is applied to generate the 2D UV location map from the original RGB image. Then a simple convolution neural network is applied to regress the 3D face from the UV location map. Finally, a network based on conditional generative adversarial networks (CGAN) is proposed to make up for the missing of UV texture caused by self-occlusion in the case of large pose. Compared with existing 3D face reconstruction methods in Stirling/ESRC 3D Face Database, proposed method can improve the accuracy. More important, a complete and realistic 3D face model can be reconstructed from the large-view face image even in the increasingly complex background.

3D face expression recognition with ensemble deep learning exploring congruent features among expressions

AbstractAutomatic and accurate 3D face expression categorization has been a challenging task in many computer vision applications. This article presents a novel approach named 3D face expression recognition with ensemble deep learning (3D‐FER‐EDL). The framework comprises three levels. Each of the first two levels gives preference to a subset of labels, considering the congruent features among expressions. Each 3D expressive face is represented as a CDS features vector, which combines corner, distance, and slope features. CDS feature vector is given as input to the first level learners. Expression probability vectors from the first level learners are jointly given as the input to the second level learners. The probability vectors from level one and two are given as the input to the third level, which is the meta classifier. The experiments are conducted on the Bosphorus database and the CASIA 3D face database to evaluate the effectiveness of the proposed approach.

A geodesic multipolar parameterization-based representation for 3D face recognition

Here, we propose a new 3D representation designed in the objective of face recognition independently of the expressions. It is defined on localities presenting limited deformations following expression variations. The proposed representation is based on the multipolar parameterization that we recently introduced in Jribi et al. (2019) which is relative invariant under three dimensional Euclidean transformations and robust to the original mesh. A choice of the number of reference points of each multipolar parameterization is made according to the shape of the two types of region of the face namely those of nose and the eyes. The main curvature field is estimated on the parameterizations of each region. The parameters of dimensional reduction algorithms applied to the overall description are adjusted so that the recognition rates remain efficient. The experiments are carried out on three challenging 3D face databases: the FRGC v2.0, the BU-3DFE and Bosphorus. Very high rates are obtained for both identification and verification scenarios. These results are very competitive with state of the art methods.

Studying factors influencing facial developmental instability

Background Developmental instability is a component of non-genetic variation that results from random variation in developmental processes. It is considered a sensitive indicator of the physiological state of individuals. It is reflected in various ways, but in this study we focussed on its reflection in fluctuating asymmetry (FA) and morphological integration. Aim To assess how, if at all, variations of facial morphology mirror developmental instability across childhood with respect to sex, growth rate and socioeconomic/environmental factors. Subjects and methods A set of 210 three-dimensional facial models (of children aged between 6.3 and 14.3 years) originating from the FIDENTIS 3D Face Database was subjected to landmark-based methods of geometric morphometrics to quantify the degree of facial asymmetry and facial morphological integration. In addition, the association with age, sex, and socioeconomic factors was assessed. Results Our results showed a nonlinear increase of FA with age up to the age of 14 years. The pattern of sex-related variants in facial FA differed in relation to age, as girls exhibited higher values of FA than boys up to the age of 9 years. We found that a signal of modularity based on functional demands and organisation of the face is of particular importance. Here, girls exhibited higher morphological covariation among modules. During more rapid adolescence-related growth, however, covariation among modules at the asymmetrical level decreased in both sexes. Conclusion We can conclude that facial morphology was shown to be strongly integrated, particularly until adolescence. This covariation can facilitate an increase of FA. In addition, the results of this study indicate there is a weak association between socioeconomic stress and facial asymmetries. In contrast, sex and growth rate are reflected in developmental instability.

Learning similarity and dissimilarity in 3D faces with triplet network

Face is the most preferred biometric trait used to recognize a person. The 2D face can be considered as a promising biometric trait; however, it may be affected by changes in the age, skin color, texture, or any other environmental factors like illumination variations, occlusion, and low image resolution. The 3D face is an emerging biometric trait, which is being used recently for human recognition. The discriminating power of the 3D face is highly motivating for many tasks such as security, surveillance, and many other technological application in day to day life. Although there are many techniques available for 3D face recognition, most of these techniques are based on volumetric or depth/range images. The conversion of 3D face data, which is originally in point cloud format to volumetric representation makes the data bulkier. Further, some of the geometric properties may be lost when 3D data is converted to a representation of lower dimensions such as depth/range images. The driving objective behind this research is to perform 3D face recognition by directly using faces represented in the form of 3D point cloud. We propose a novel approach for 3D face recognition by learning the similarity and dissimilarity in 3D faces, and for this purpose, introduce a triplet network. The network is an ensemble of our proposed Convolutional PointNet (CPN) network, used for feature extraction and triplet loss. The proposed network maps a 3D face data to Euclidean space where distance based scores represent the similarity among the 3D faces. We also introduce a new evaluation approach for computing the dissimilarity between highly similar 3D face biometric data. Experimentation has been carried out on two databases, namely IIT Indore 3D face database (our in-house database) and Bosphorus 3D face database. To handle the training issues due to the limited availability of samples for each subject in both the databases, we propose a technique for 3D data augmentation. We perform various experiments using the proposed network and show the performance in terms of verification rate and ROC curve. Our point cloud based triplet network shows encouraging performance as compared to other state-of-the-art techniques.

Age verification using random forests on facial 3D landmarks

Three-dimensional facial images are becoming more and more widespread. As such images provide more information about facial morphology than 2D imagery, they show great promise for use in future forensic applications, including age estimation and verification.This paper proposes an approach using random forests, a machine learning method, to develop and test models for classification of legal age thresholds (15 years and 18 years) using 3D facial landmarks. Our approach was developed on a set of 3D facial scans from 394 Czech individuals (194 males and 200 females) aged between 10 and 25 years. The dataset was retrieved from a sizable database of Central European faces – The FIDENTIS 3D Face Database. Three main types of input variables were processed using random forests: I) shape (size-invariant) coordinates of 3D landmarks, II) size and shape coordinates of 3D landmarks, and III) inter-landmark distances, angles and indices. The performance rates for the combinations of variables and age threshold were expressed in terms of sensitivity and specificity.The overall accuracy rates varied from 71.4%–91.5% (when the male and female samples were pooled). In general, higher accuracy was achieved for the age limit of 18 years than for 15 years. Whereas size-variant variables showed a better performance rate for the age limit of 15 years, the size-invariant variables (i.e., shape variables) were better for classifying individuals under 18 years. The verification models grounded on traditional variables (distances, angles, indices) yielded consistently higher performance rates on females than on males, whereas the inverse trend was observed for the models built on 3D coordinates. The results indicate that age verification based on 3D facial data with processing by the random forests method has high potential for further forensic or biometric applications.

Complement component face space for 3D face recognition from range images

This paper proposes a mathematical model for decomposing a range face image into four basic components (named ‘complement components’) in conjunction with a simple approach for data-level fusion to generate thirty-six additional hybrid components. These forty component faces composing a new face image space called the ‘complement component face space.’ The main challenge of this work was to extract relevant features from the vast face space. Features are extracted from the four basic components and four selected hybrid components using singular value decomposition. To introduce diversity, the extracted feature vectors are fused by applying the crossover operation of the genetic algorithm using a Hamming distance-based fitness measure. Particle swarm optimization-based feature selection is employed on the fused features to discard redundant feature values and to maximize the face recognition performance. The recognition performances of the proposed feature set with a support vector machine-based classifier on three accessible and well-known 3D face databases, namely, Frav3D, Bosphorus, and Texas3D, show significant improvements over those achieved by state-of-the-art methods. This work also studies the feasibility of utilizing the component images in the complement component face space for data augmentation in convolutional neural network (CNN)-based frameworks.

SL2E-AFRE : Personalized 3D face reconstruction using autoencoder with simultaneous subspace learning and landmark estimation

3D face reconstruction from single face image has received much attention in the past decade, as it has been used widely in many applications in the field of computer vision. Despite more accurate solutions by 3D scanners and several commercial systems, they have drawbacks such as the need for manual initialization, time and economy constraints. In this paper, a novel framework for 3D face reconstruction is presented. Firstly, landmarks are localized on the database faces with the proposed landmark-mapping strategy employing a model template. Then, an autoencoder assisted by the proposed energy function to simultaneously learn the facial patch subspace and the keypoints positions is employed to predict the landmarks. Finally, an unique 3D reconstruction is obtained with the proposed predicted landmark based deformation. Meta-parameters are incorporated into the energy function during the training phase to enhance the performance of the autoencoder network in reconstructing the face model. The experiments are carried out on two databases namely the USF Human ID 3-D Database and the Bosphorus 3D face database. The experimental results show that the Autoencoder based Face REconstruction with Simultaneous patch Learning and Landmark Estimation method (SL2E-AFRE) is efficient and the performance of the same is significantly upgraded in each iteration.

SpPCANet: a simple deep learning-based feature extraction approach for 3D face recognition

Abstract A Sparse Principal Component Analysis Network (SpPCANet) based feature extraction is proposed here for 3D face recognition. The network consists of three basic components: (1) Multistage sparse principal component analysis filters, (2) Binary hashing, and (3) Block-wise histogram computation. Here, the sparse principal component analysis is used to learn multistage filter banks at the convolution stage, which is followed by binary hashing for indexing and block-wise histogram for pooling. Finally, a linear support vector machine (SVM) is used for classifying the features extracted by SpPCANet. The proposed network SpPCANet is a lightweight deep learning network. Three well-known 3D face databases, namely, Frav3D, Bosphorus3D, and Casia3D, are used for validating the proposed system. This proposed network has been extensively studied by varying different parameters, such as the number of filters at the convolution layer and the size of filters at the convolution layer and size of non-overlapping blocks at the pooling layer. Handling all types of variation of faces available in Frav3D, Bosphorus3D, and Casia3D databases, the system has acquired 96.93%, 98.54%, and 88.80% recognition rates, respectively.

3DFaceGAN: Adversarial Nets for 3D Face Representation, Generation, and Translation

Over the past few years, Generative Adversarial Networks (GANs) have garnered increased interest among researchers in Computer Vision, with applications including, but not limited to, image generation, translation, imputation, and super-resolution. Nevertheless, no GAN-based method has been proposed in the literature that can successfully represent, generate or translate 3D facial shapes (meshes). This can be primarily attributed to two facts, namely that (a) publicly available 3D face databases are scarce as well as limited in terms of sample size and variability (e.g., few subjects, little diversity in race and gender), and (b) mesh convolutions for deep networks present several challenges that are not entirely tackled in the literature, leading to operator approximations and model instability, often failing to preserve high-frequency components of the distribution. As a result, linear methods such as Principal Component Analysis (PCA) have been mainly utilized towards 3D shape analysis, despite being unable to capture non-linearities and high frequency details of the 3D face—such as eyelid and lip variations. In this work, we present 3DFaceGAN, the first GAN tailored towards modeling the distribution of 3D facial surfaces, while retaining the high frequency details of 3D face shapes. We conduct an extensive series of both qualitative and quantitative experiments, where the merits of 3DFaceGAN are clearly demonstrated against other, state-of-the-art methods in tasks such as 3D shape representation, generation, and translation.

3D Security User Identification in Banking

Biometric systems may be divided into 2 categories betting on the characteristics used. One category uses physical characteristics that area unit associated with the form and presence of the body and body components, like fingerprint, finger knuckles, face (2-D and 3-D), DNA, hand and palm pure mathematics, iris texture, and retinal vasculature. Systems belong to the second category use activity characteristics, like gait, handwriting, keyboard writing, and speech. Analysis in face recognition has endlessly been challenged by alien (head create, lighting conditions) and intrinsic (facial expression, aging) sources of variability. During this system is employed to several organizations and lots of applications for security purpose. Many approaches area unit face recognition exists, during this project, specialize in a comparative study of 3D face recognition beneath expression variations. First 3D face databases with expressions area unit listed, and therefore the most significant ones area unit conferred and their complexness is quantified victimisation principal part analysis, linear discriminate analysis and native binary patterns. The project to be real time enforced datasets to reason the assorted varieties of expressions. Pictures in terms of the popularity performance are evaluated with 3 completely different techniques (principal component analysis, linear discriminant analysis, and native binary patterns) on face recognition grand challenge and strait 3D face databases.

An efficient 3D face recognition approach using Frenet feature of iso-geodesic curves

Extracting efficient features from the large volume of 3D facial data directly is extremely difficult in 3D face recognition (3D-FR) with the latest methods, which mostly require heavy computations and manual processing steps. This paper presents a computationally efficient 3D-FR system based on a novel Frenet frame-based feature that is derived from the 3D facial iso-geodesic curves. In terms of the evaluation of the proposed method, we conducted a number of experiments on the CASIA 3D face database, and a superior recognition performance has been achieved. The performance evaluation suggests that the pose invariance attribute of the features relieves the need of an expensive 3D face registration in the face preprocessing procedure, where we take less time to process conversely. Our experiments further demonstrate that the proposed method not only achieves competitive recognition performance when compared with some existing techniques for 3D-FR, but also is computationally efficient.

Weighted Extreme Sparse Classifier and Local Derivative Pattern for 3D Face Recognition.

A novel weighted hybrid classifier and a high-order, local normal derivative pattern descriptor is proposed for 3D face recognition. The Local derivative pattern (LDP) captures detailed information, based on the local derivative variation in different directions. The LDP is computed on three normal maps in x, y, and z directions and on different scales. The surface normal captures the orientation of a surface at each point of 3D data. More informative local shape information is extracted using the surface normal, as compared to depth. The nth-order LDP on the surface normal is proposed to encode more detailed features from the (n-1)th-order's local derivative direction variations. An extreme learning machine (ELM) based autoencoder, using a multilayer network structure, is employed to select more discriminant features and provide a faster training speed. A weighted hybrid framework is proposed to handle facial challenges using a combination of the ELM and the sparse representation classifier (SRC). The advantage of speed for the ELM and accuracy for the SRC in a weighted scheme is used to enhance the performance of the recognition system. Experimental results regarding four famous 3D face databases illustrate the generalization and effectiveness of the proposed method in terms of both computational cost and recognition accuracy.

Feature fusions for 2.5D face recognition in Random Maxout Extreme Learning Machine

Contemporary face recognition system is often based on either 2D (texture) or 3D (texture + shape) face modality. An alternative modality that utilizes range (depth) facial images, namely 2.5D face recognition emerges. In this paper, we propose a 2.5D face descriptor that based on the Regional Covariance Matrix (RCM), a powerful means of feature fusion technique and a novel classifier dubbed Random Maxout Extreme Learning Machine (RMELM). The RCM of interest is constructed based on the Principal Component Analysis (PCA) filters responses of facial texture and/or range image, wherein the PCA filters are learned from a two-layer PCA network. The RMELM is an ELM variant where the activation function is based on the locally linear maxout function, in place of typical global non-linear functions in ELM. Since the RCM is a special case of symmetric positive definite matrix that resides on the Tensor manifold; a gap exists in between RCM and RMELM, which is a vector-based classifier. To bridge the gap, we flatten the manifold by transforming the RCM to a feature vector via a matrix logarithm operator. Experimental results from two public 3D face databases, FRGC v2.0 database and Gavab database, validated our proposed method is promising in 2.5D face recognition.

Single Image 3D Face Reconstruction Based on Statistical Model

Single image 3D face reconstruction technology has a broad application in the life, also has a high technical difficulties. This paper proposes a single image 3D face reconstruction technology based on statistical model. The method established 3D face statistical model based on 3D face database in advance, and trained 2D facial feature points parameter model based on the 2D face database using the algorithm of SDM. when reconstruct 3D face model from single image, first we use 2D facial feature points parameter model to extract face feature points; Then, according to 3D face model, we use adaptive learning factor gradient descent method to iteratively optimize the energy function, to get statistical model parametric vector, namely got the 3D face model corresponding to the 2D face images. The results show that use the proposed method to rebuild the faces have high similarity.

Performance evaluation of 2D face recognition techniques under image processing attacks

Face recognition is a vastly researched topic in the field of computer vision. A lot of work have been done for facial recognition in two dimensions and three dimensions. The amount of work done with face recognition invariant of image processing attacks is very limited. This paper presents a total of three classes of image processing attacks on face recognition system, namely image enhancement attacks, geometric attacks and the image noise attacks. The well-known machine learning techniques have been used to train and test the face recognition system using two different databases namely Bosphorus Database and University of Milano Bicocca three-dimensional (3D) Face Database (UMBDB). Three classes of classification models, namely discriminant analysis, support vector machine and k-nearest neighbor along with ensemble techniques have been implemented. The significance of machine learning techniques has been mentioned. The visual verification has been done with multiple image processing attacks.

Introducing the FIDENTIS 3D Face Database

Face databases have assumed an important role in a variety of clinical and applied research domains. However, the number of datasets accessible to the scientific community is limited and the knowledge of their existence may be concealed from a wider range of specialists. In the present paper we introduce a sizeable dataset of 3D facial scans – FIDENTIS 3D Face Database (F3D-FD or FIDENTIS Database), which is accompanied by basic demographic and descriptive data. The database is structured according to recorded subjects, and comprises single-scan entries as well as a smaller number of multiscan entries. The multi-scan entries vary in the time passed between recording sessions and in the devices employed to collect the 3D data. The total number of 2476 individuals puts our database within the category of large-scale databases. The 3D scans are accessible through a web-based interface at www.fidentis.cz. A licensed version of the database is available to interested parties upon signing a license agreement. Because of its varied composition, and low target-specificity the database has capacity to be of great assistance for the worldwide research community.

Emotion information visualization through learning of 3D morphable face model

Analysis and visualization of human facial expressions and its applications are useful but challenging. This paper presents a novel approach to analyze the facial expressions from images through learning of a 3D morphable face model and a quantitative information visualization scheme for exploring this type of visual data. More specifically, a 3D face database with various facial expressions is employed to build a nonnegative matrix factorization (NMF) part-based morphable 3D face model. From an input image, a 3D face with expression can be reconstructed iteratively by using the NMF morphable 3D face model as a priori knowledge, from which basis parameters and a displacement map are extracted as features for facial emotion analysis and visualization. Based upon the features, two support vector regressions are trained to determine the fuzzy valence–arousal (VA) values to quantify the emotions. The continuously changing emotion status can be intuitively analyzed by visualizing the VA values in VA space. Our emotion analysis and visualization system, based on 3D NMF morphable face model, detect expressions robustly from various head poses, face sizes and lighting conditions and is fully automatic to compute the VA values from images or a sequence of video with various facial expressions. To evaluate our novel method, we test our system on publicly available databases and evaluate the emotion analysis and visualization results. We also apply our method to quantifying emotion changes during motivational interviews. These experiments and applications demonstrate the effectiveness and accuracy of our method.