Facial Texture Research Articles

Multi-objective evolutionary 3D face reconstruction based on improved encoder–decoder network

Three-dimensional (3D) face application has attracted significant attention the field of multimedia and image processing. However, the end-to-end 3D face reconstruction method is still immature, and there are some problems, such as overfitting caused by too few training sets and unacceptable 3D face texture alignment performance. Therefore, we design a novel approach to construct a 3D face, named multi-objective evolutionary 3D face reconstruction based on improved encoder–decoder network (MoEDN). This study introduces a regularization algorithm named feature map distortion (Disout); whose purpose is to strengthen the network generalization ability. Based on this, we construct a multi-objective evolutionary 3D face reconstruction model, in which decision variables are distortion probability, distorted block size, distorted intensity, probability step, and learning rate; and objective functions are loss and structural similarity (SSIM). We use four multi-objective evolutionary algorithms (NSGA-II, AGEII, NSLS, and MOEA/D) to optimize the proposed model. Experimental results demonstrate that NSLS has the best performance. In addition, compared with position map regression network (PRNet), 2D-assisted self-supervised learning (2DASL) and other state-of-the-art, the proposed model achieves better loss values and NME values. Therefore, the proposed multi-objective evolutionary 3D face reconstruction model has outstanding 3D facial reconstruction performance in large poses and face expression.

Identifying Gender from Images of Face

The main objective of this project is to classify the gender based on different facial features such as eyes, nose, mouth, overall features such as face contour, head shape, hair line etc. The gender classification algorithm uses machine learning technique (supervised learning). In this case the algorithm is trained on a set of male and female faces and then used to classify new data. In this project, face detection and gender classification methods are combined. The face detection acts as a pre-processing operation to the gender classifier that determines the gender.There are multiple methods in which facial recognition systems work, but in general, they work by comparing selected facial features from a given image with faces within a database. It is also described as a Biometric Artificial Intelligence based application that can uniquely identify a person by analyzing patterns based on the person's facial textures and shape.Automated gender recognition plays an important role in many application areas such as human computer interaction,biometric, surveillance, demographic statistics etc.

A Novel Multi-Level Pyramid Co-Variance Operators for Estimation of Personality Traits and Job Screening Scores

Recently, automatic personality analysis is becoming an interesting topic for computer vision. Many attempts have been proposed to solve this problem using time-based sequence information. In this paper, we present a new framework for estimating the Big-Five personality traits and job candidate screening variable from video sequences. The framework consists of two parts: (1) the use of Pyramid Multi-level (PML) to extract raw facial textures at different scales and levels; (2) the extension of the Covariance Descriptor (COV) to fuse different local texture features of the face image such as Local Binary Patterns (LBP), Local Directional Pattern (LDP), Binarized Statistical Image Features (BSIF), and Local Phase Quantization (LPQ). Therefore, the COV descriptor uses the textures of PML face parts to generate rich low-level face features that are encoded using concatenation of all PML blocks in a feature vector. Finally, the entire video sequence is represented by aggregating these frame vectors and extracting the most relevant features. The exploratory results on the ChaLearn LAP APA2016 dataset compare well with state-of-the-art methods including deep learning-based methods.

Detecting fake images by identifying potential texture difference

Fake detection has become an urgent task. Generative adversarial networks (GANs) extended to deep learning has shown its extraordinary ability in the fields of image, audio, and speech. But advanced technology benefits us, it also poses a threat to us when used in Cyber Crime. The Deepfake (common name for face manipulation methods) based on GANs can realize the replacement of different faces. Due to the development of GANs, faces generated by Deepfake can already be visually real. Deepfake can purposely replace any face to a different person, so that a fabricated event may be widely spread because of the convenience of the Internet, causing serious impacts such as personal attacks and cyber crime. Based on cutting-edge research , this paper proposes a intelligence forensic method of Deepfake detection. We first discover the subtle texture differences between real and fake image in image saliency, which shows difference in the texture of faces. To amplify this difference, we exploit guided filter with saliency map as guide map to enhance the texture artifacts caused by the post-processing and display the potential features of forgery. Resnet18 classification network efficiently learns the exposed difference and finally realizes the real and fake detection of face images. We evaluate the performance of the method and experiments verify that the proposed method can achieve the state-of-the-art detection accuracy .

Artist guided generation of video game production quality face textures

We develop a high resolution face texture generation system which uses artist provided appearance controls as the conditions for a generative network. Artists are able to control various elements in the generated textures, such as the skin, eye, lip, and hair color. This is made possible by reparameterizing our dataset to the same UV mapping, allowing us to utilize image-to-image translation networks. Although our dataset is limited in size, only 126 samples in total, our system is still able to generate realistic face textures which strongly adhere to the input appearance attribute conditions because of our training augmentation methods. Once our system has generated the face texture, it is ready to be used in a modern game production environment. Thanks to our novel SuperResolution and material property recovery methods, our generated face textures are 4K resolution and have the associated material property maps required for raytraced rendering.

Silicone mask face anti-spoofing detection based on visual saliency and facial motion

Face recognition systems are widely used for target recognition and identity authentication, such as automated teller machines, mobile phones, and entrance guard systems. However, face recognition systems are vulnerable to presentation attacks, such as photo, replay, and 3D mask attacks. In particular, silicone mask attacks pose a greater threat to face recognition systems because high-quality silicone masks do living properties. To promote the development of face anti-spoofing detection algorithms for silicone mask attacks, this paper constructs a Silicone Mask Face Motion Video Dataset (SMFMVD) containing 200 real face videos and 200 silicone mask face videos. These videos include different facial motions collected from 20 subjects. Moreover, inspired by the observation that the silicone mask face’s facial movement is not so natural as the real face, we propose a novel silicone mask face anti-spoofing detection method based on visual saliency and facial motion characteristics. Specifically, we compute the visual saliency map of a given face image by simulating two kinds of eye movement behaviors, namely “gaze” and “saccade”. Then, we propose a saliency-weighted histogram of local binary pattern operator to extract facial texture features in spatial domain and a saliency-guided histogram of oriented optical flow operator to extract facial motion features in temporal domain. Finally, the support vector machine is used to fuse two groups of facial features to distinguish real and spoof faces. Extensive experiments on public and self-built datasets show its superiority over the state-of-the-art methods.

MeInGame: Create a Game Character Face from a Single Portrait

Many deep learning based 3D face reconstruction methods have been proposed recently, however, few of them have applications in games. Current game character customization systems either require players to manually adjust considerable face attributes to obtain the desired face, or have limited freedom of facial shape and texture. In this paper, we propose an automatic character face creation method that predicts both facial shape and texture from a single portrait, and it can be integrated into most existing 3D games. Although 3D Morphable Face Model (3DMM) based methods can restore accurate 3D faces from single images, the topology of 3DMM mesh is different from the meshes used in most games. To acquire fidelity texture, existing methods require a large amount of face texture data for training, while building such datasets is time-consuming and laborious. Besides, such a dataset collected under laboratory conditions may not generalized well to in-the-wild situations. To tackle these problems, we propose 1) a low-cost facial texture acquisition method, 2) a shape transfer algorithm that can transform the shape of a 3DMM mesh to games, and 3) a new pipeline for training 3D game face reconstruction networks. The proposed method not only can produce detailed and vivid game characters similar to the input portrait, but can also eliminate the influence of lighting and occlusions. Experiments show that our method outperforms state-of-the-art methods used in games. Code and dataset are available at https://github.com/FuxiCV/MeInGame.

A General Method of Realistic Avatar Modeling and Driving for Head-Mounted Display Users

The head-mounted displays (HMDs) provide immersive experiences in the virtual reality (VR). However, the face interactions are limited due to the serious occlusion of the user’s face. Existed approaches try to recover the user’s facial expression by adding additional sensors to HMD. In this article, we develop a novel framework to reconstruct the user’s 3-D face in VR only using an RGB camera. Given a reference face, a realistic full-textured avatar is created by fitting a 3-D Morphable Model (3DMM). A self-supervised UV map generative adversarial network (GAN) is proposed to make the facial texture look more realistic. Next, we propose a novel landmark detection method to locate the landmark positions under HMD occlusion since facial landmarks are commonly used for driving the avatar. To this end, we synthesize a face data set with HMD. Our method is easy to build and popularize with low cost. The experiments on the synthetic and real HMD data demonstrate that the proposed method can detect landmark accurately and restore facial expressions faithfully despite the large occlusion of HMD.

Explaining face representation in the primate brain using different computational models

Understanding how the brain represents the identity of complex objects is a central challenge of visual neuroscience. The principles governing object processing have been extensively studied in the macaque face patch system, a sub-network of inferotemporal (IT) cortex specialized for face processing. A previous study reported that single face patch neurons encode axes of a generative model called the "active appearance" model, which transforms 50D feature vectors separately representing facial shape and facial texture into facial images. However, a systematic investigation comparing this model to other computational models, especially convolutional neural network models that have shown success in explaining neural responses in the ventral visual stream, has been lacking. Here, we recorded responses of cells in the most anterior face patch anterior medial (AM) to a large set of real face images and compared a large number of models for explaining neural responses. We found that the active appearance model better explained responses than any other model except CORnet-Z, a feedforward deep neural network trained on general object classification to classify non-face images, whose performance it tied on some face image sets and exceeded on others. Surprisingly, deep neural networks trained specifically on facial identification did not explain neural responses well. A major reason is that units in the network, unlike neurons, are less modulated by face-related factors unrelated to facial identification, such as illumination.

Performance analysis of tools with rake face textures produced using wire-EDM in turning AISI4340

ABSTRACT Tool–chip interface friction significantly affects the machining characteristics like cutting forces, machining temperature, tool wear, and surface finish. Structuring the tool rake face using micro-textures is a recent method adopted to reduce tool–chip interface friction. The effect of varying the depth of these rake face micro-textures on the machinability of AISI4340 is studied. Textures of uniform width but with depth varying from 60 to 145 μm are created on the tool rake face using wire-EDM process perpendicular to the chip flow direction. The performance of these tools is compared with a non-textured tool in terms of cutting forces, cutting zone temperature, flank wear, chip trace area and surface finish. It is found that by optimizing the texture depth, it is possible to get improved performance from the cutting tool for the same machining conditions as compared to a non-textured tool. Further, it is found that for all the tools, the chip morphology changes during each pass from regular, closely curled helix at the beginning of the pass to randomly coiled chips at the end of the pass length with simultaneous increase in temperature.

An automated integrated speech and face imageanalysis system for the identification of human emotions

ObjectiveHuman interactions are related to speech and facial characteristics. It was suggested that speech signals and/or images of facial expressions may reveal human emotions and that both interact for the verification of a person's identity. The present study proposes and evaluates an automated integrated speech signal and facial image analysis system for the identification of seven different human emotions (Normal (N), Happy (H), Sad (S), Disgust (D), Fear (F), Anger (A), and Surprise (Su)). MethodsSpeech recordings and face images from 7,441 subjects aged 20≤age≤74 were collected, normalized and filtered. From all the above recordings 55 speech signal features and 61 different image face texture features were extracted. Statistical and model multi-classification analysis were performed to select the features able to statistically significantly distinguish between the seven aforementioned human emotions (N, H, S, D, F, A and Su). The selected features alone or a combination of these features along with age and gender of the sample investigated were used to build two learning-based classifiers; and the classifiers’ accuracy was computed. ResultsFor each of the above mentioned human emotions, statistical significantly different speech and face image features were identified that may be used to distinguish between the aforementioned groups (N vs H, N vs S, N vs D, N vs F, N vs A, N vs Su). Using solely the statistically significant speech and image features identified, an overall percentage of correct classification (%CC) score of 93% was achieved. ConclusionsA significant number of speech and face image features have been derived from continuous speech and face images. Features were identified that were able to identify between seven different emotional human states. This study poses the basis for the development of an integrated system for the identification of emotional states from automatic analysis of free speech and image face analysis. Future work will investigate the development and integration of the proposed method into a mobile device.

Age prediction based on a small number of facial landmarks and texture features.

BACKGROUND:Age is an essential feature of people, so the study of facial aging should have particular significance.OBJECTIVE:The purpose of this study is to improve the performance of age prediction by combining facial landmarks and texture features.METHODS:We first measure the distribution of each texture feature. From a geometric point of view, facial feature points will change with age, so it is essential to study facial feature points. We annotate the facial feature points, label the corresponding feature point coordinates, and then use the coordinates of feature points and texture features to predict the age.RESULTS: We use the Support Vector Machine regression prediction method to predict the age based on the extracted texture features and landmarks. Compared with facial texture features, the prediction results based on facial landmarks are better. This suggests that the facial morphological features contained in facial landmarks can reflect facial age better than facial texture features. Combined with facial landmarks and texture features, the performance of age prediction can be improved.CONCLUSIONS:According to the experimental results, we can conclude that texture features combined with facial landmarks are useful for age prediction.

Cascade Regression-Based Face Frontalization for Dynamic Facial Expression Analysis

Facial expression recognition has seen rapid development in recent years due to its wide range of applications such as human–computer interaction, health care, and social robots. Although significant progress has been made in this field, it is still challenging to recognize facial expressions with occlusions and large head-poses. To address these issues, this paper presents a cascade regression-based face frontalization (CRFF) method, which aims to immediately reconstruct a clean, frontal and expression-aware face given an in-the-wild facial image. In the first stage, a frontal facial shape is predicted by developing a cascade regression model to learn the pairwise spatial relation between non-frontal face-shape and its frontal counterpart. Unlike most existing shape prediction methods that used single-step regression, the cascade model is a multi-step regressor that gradually aligns non-frontal shape to its frontal view. We employ several different regressors and make a ensemble decision to boost prediction performance. For facial texture reconstruction, active appearance model instantiation is employed to warp the input face to the predicted frontal shape and generate a clean face. To remove occlusions, we train this generative model on manually selected clean-face sets, which ensures generating a clean face as output regardless of whether the input face involves occlusions or not. Unlike the existing face reconstruction methods that are computational expensive, the proposed method works in real time, so it is suitable for dynamic analysis of facial expression. The experimental validation shows that the ensembling cascade model has improved frontal shape prediction accuracy for an average of 5% and the proposed method has achieved superior performance on both static and dynamic recognition of facial expressions over the state-of-the-art approaches. The experimental results demonstrate that the proposed method has achieved expression-preserving frontalization, de-occlusion and has improved performance of facial expression recognition.

Failure Analysis of Apennine Masonry Churches Severely Damaged during the 2016 Central Italy Seismic Sequence

This paper presents a detailed study of the damages and collapses suffered by various masonry churches in the aftermath of the seismic sequence of Central Italy in 2016. The damages will first be analyzed and then compared with the numerical data obtained through 3D simulations with eigenfrequency and then nonlinear static analyses (i.e., pushover). The main purposes of this study are: (i) to create an adequately consistent sensitivity study on several definite case studies to obtain an insight into the role played by geometry—which is always unique when referred to churches—and by irregularities; (ii) validate or address the applicability limits of the more widespread nonlinear approach, widely recommended by the Italian Technical Regulations. Pushover analyses are conducted assuming that the masonry behaves as a nonlinear material with different tensile and compressive strengths. The consistent number of case studies investigated will show how conventional static approaches can identify, albeit in a qualitative way, the most critical macro-elements that usually trigger both global and local collapses, underlining once again how the phenomena are affected by the geometry of stones and bricks, the texture of the wall face, and irregularities in the plan and elevation and in addition to hypotheses made on the continuity between orthogonal walls.

Texture descriptors for better face recognition performance

Due to, the great electronic development, which reinforced the need to define people’s identities, different methods, and databases to identification people’s identities have emerged. In this paper, we compare the results of two texture analysis methods: Local Binary Pattern (LBP) and Local Ternary Pattern (LTP). The comparison based on comparing the extracting facial texture features of 40,401 and 10 subjects taken from ORL, UFI and Self-Created databases respectively. As well, the comparison has taken in the account using three distance measurements such as; Manhattan Distance (MD), Euclidean Distance (ED), and Cosine Distance (CD). Where the maximum accuracy of the LBP method (99.23%) is obtained with a Manhattan and ORL database as standard database, while the LTP method attained (98.76%) using the same distance and database. While, the facial database of UFI shows low quality, which is satisfied 75.98% and 73.82% recognition rates using LBP and LTP respectively with Manhattan distance. Hereby, the LBP based biometric system performs excellence recognition accuracy even with difficult face images that contain different expressions, different poses, and varied illumination like the UFI dataset.

A Randomized Double-Blind Evaluation of a Novel Biotin and Silicon Ingredient Complex on the Hair and Skin of Healthy Women

Introduction: Age related changes in hair and skin impact quality of life. Interventions to mitigate these changes are of interest. Aim: To examine the safety and efficacy of LustrivaTM (a novel source of biotin and silicon) at a high or lower dose compared to placebo for impacts on hair and skin. Materials and Methods: In a randomized, double-blind study, 90 healthy female subjects with self-reported thinning hair who met Savin/Ludwig Scale criteria I-2 to II-1 by physician evaluation were randomized to one of three groups for 12 weeks (n=30/group): LustrivaTM High-Dose (LHD), LustrivaTM Low-Dose (LLD) or Placebo (PL). Hair quality and thickness measured by the TrichoScan HD testing system and skin parameters (facial wrinkles, fine lines, skin texture, skin color evenness, skin elasticity) measured by the Antera 3DTM System and the CutometerTM Dual MPA 580 system. Results: There was a significant increase in hair thickness measured by change in % vellus hair and % terminal hair and in the ratio of % vellus to terminal hair in LHD compared to PL at Week 3, maintained throughout the study (p=0.029). LHD had a significant decrease in facial wrinkles (12 Weeks) measured by a change in maximal wrinkle depth vs. PL (p=0.031). After 12 weeks compared to baseline LHD significantly improved facial wrinkle Maximum Depth, Indentation Index and Score, facial fine lines Indentation Index and Score, and facial texture Maximum Height, Roughness and Score (p<0.05), no change in PL. There were no changes for skin elasticity between groups. For some hair and skin parameters, LLD showed improvements less than LHD but that approached significance (p<0.1). All groups improved in subjective nail endpoints vs. baseline with no significant differences between groups. No adverse events reported. Conclusion: LHD significantly increased hair thickness and reduced facial wrinkle depth compared to placebo and performed better than the LLD in most parameters. Future studies are warranted.

Predominance of eyes and surface information for face race categorization

Faces can be categorized in various ways, for example as male or female or as belonging to a specific biogeographic ancestry (race). Here we tested the importance of the main facial features for race perception. We exchanged inner facial features (eyes, mouth or nose), face contour (everything but those) or texture (surface information) between Asian and Caucasian faces. Features were exchanged one at a time, creating for each Asian/Caucasian face pair ten facial variations of the original face pair. German and Korean participants performed a race classification task on all faces presented in random order. The results show that eyes and texture are major determinants of perceived biogeographic ancestry for both groups of participants and for both face types. Inserting these features in a face of another race changed its perceived biogeographic ancestry. Contour, nose and mouth, in that order, had decreasing and much weaker influence on race perception for both participant groups. Exchanging those features did not induce a change of perceived biogeographic ancestry. In our study, all manipulated features were imbedded in natural looking faces, which were shown in an off-frontal view. Our findings confirm and extend previous studies investigating the importance of various facial features for race perception.

Improving face recognition performance using TeCS2dictionary

• Combining visual cues with face recognition algorithm to improve performance. • Visual queues such as color, shape, texture and symmetry help in understanding faces. • Extensive experiments show the effectiveness of the proposed framework. • Proposed approach outperforms our previous work and other state of art approaches. Human mind processes the different primitive components of image signals such as color, shape, texture, and symmetry in a parallel and complex fashion. Deep neural networks aim to learn all these components from the image in an unsupervised manner. However, learning the primitive features is not formally assured in a deep learning formulation, and, adding these features explicitly would improve the performance. Especially in face recognition, humans intuitively and implicitly employ the usage of primitive features such as color, shape, texture, and symmetry of faces. Inspired by this observation, this paper presents a novel approach in building a learning based TeCS 2 space. This space consists of meta-level features obtained from dictionary learning and combining it with task specific deep learning classifiers (such as DenseNet) for face recognition. Confidence based fusion mechanism is presented to supplement the task specific deep learning classifier with the proposed TeCS 2 features. The effectiveness of the proposed framework is evaluated on four benchmark face recognition datasets: (i) Disguised Faces in the Wild (DFW), (ii) Labeled faces in the wild (LFW), (iii) IIITD Plastic Surgery dataset, and (iv) Point and Shoot Challenge (PaSC).

Fast-GANFIT: Generative Adversarial Network for High Fidelity 3D Face Reconstruction.

A lot of work has been done towards reconstructing the 3D facial structure from single images by capitalizing on the power of deep convolutional neural networks (DCNNs). In the recent works, the texture features either correspond to components of a linear texture space or are learned by auto-encoders directly from in-the-wild images. In all cases, the quality of the facial texture reconstruction is still not capable of modeling facial texture with high-frequency details. In this paper, we take a radically different approach and harness the power of generative adversarial networks (GANs) and DCNNs in order to reconstruct the facial texture and shape from single images. That is, we utilize GANs to train a very powerful facial texture prior from a large-scale 3D texture dataset. Then, we revisit the original 3D Morphable Models (3DMMs) fitting making use of non-linear optimization to find the optimal latent parameters that best reconstruct the test image but under a new perspective. In order to be robust towards initialisation and expedite the fitting process, we propose a novel self-supervised regression based approach. We demonstrate excellent results in photorealistic and identity preserving 3D face reconstructions and achieve for the first time, to the best of our knowledge, facial texture reconstruction with high-frequency details.

3D-Guided Face Manipulation of 2D Images for the Prediction of Post-Operative Outcome After Cranio-Maxillofacial Surgery.

Cranio-maxillofacial surgery often alters the aesthetics of the face which can be a heavy burden for patients to decide whether or not to undergo surgery. Today, physicians can predict the post-operative face using surgery planning tools to support the patient's decision-making. While these planning tools allow a simulation of the post-operative face, the facial texture must usually be captured by another 3D texture scan and subsequently mapped on the simulated face. This approach often results in face predictions that do not appear realistic or lively looking and are therefore ill-suited to guide the patient's decision-making. Instead, we propose a method using a generative adversarial network to modify a facial image according to a 3D soft-tissue estimation of the post-operative face. To circumvent the lack of available data pairs between pre- and post-operative measurements we propose a semi-supervised training strategy using cycle losses that only requires paired open-source data of images and 3D surfaces of the face's shape. After training on "in-the-wild" images we show that our model can realistically manipulate local regions of a face in a 2D image based on a modified 3D shape. We then test our model on four clinical examples where we predict the post-operative face according to a 3D soft-tissue prediction of surgery outcome, which was simulated by a surgery planning tool. As a result, we aim to demonstrate the potential of our approach to predict realistic post-operative images of faces without the need of paired clinical data, physical models, or 3D texture scans.