Face Pose Research Articles

IE-WGAN: A model with Identity Extracting for Face Frontal Synthesis

Face pose affects the effect of the frontal face synthesis, we propose a model used for frontal face synthesis based on WGAN-GP. This model includes identity extracting module, which is used to supervise the training of the face generation module. On the one hand, the model improves the quality of synthetic face images, on the other hand, it can accelerate the convergence speed of network training. We conduct verification experiments on CelebA data sets, and the results show that this model improves the graphic quality of frontal synthesis.

Face spoofing detection under super-realistic 3D wax face attacks

• A new database with wax figure faces as super-realistic 3D face spoofing attacks. • High diversity and high quality of faces in database. • Fusion of attention-aware features of different face scales for face anti-spoofing. • Performance evaluation under both intra- and cross-database testing scenarios. Face spoofing attacks based on 3D face images have posed a severe security risk to face recognition systems. Despite the great effort made by the technical community in recent years, existing 3D face spoofing databases, mostly based on 3D masks, still suffer from small sample size, low diversity, or poor authenticity due to the production difficulty and high cost. To fill in this gap, we introduce a new database in this paper with 4-000 single wax figure faces, named SWFFD (Single Wax Figure Face Database), as a type of super-realistic 3D face presentation attack. Collected from online resources, this database has high diversity in terms of subjects, lighting conditions, facial poses, and recording devices. We have also designed a new detection method, which combines attention-aware features from different face scales to generate discriminative representations for realistic face spoofing attack detection. Extensive experiments have been conducted on the SWFFD as well as the CelebA-HQ database (containing real faces from the online collection). Experimental results have demonstrated the effectiveness of the proposed method in both intra-database and cross-database testing scenarios.

A Review of Benchmark Datasets and Training Loss Functions in Neural Depth Estimation

In many applications, such as robotic perception, scene understanding, augmented reality, 3D reconstruction, and medical image analysis, depth from images is a fundamentally ill-posed problem. The success of depth estimation models relies on assembling a suitably large and diverse training dataset and on the selection of appropriate loss functions. It is critical for researchers in this field to be made aware of the wide range of publicly available depth datasets along with the properties of various loss functions that have been applied to depth estimation. Selection of the right training data combined with appropriate loss functions will accelerate new research and enable better comparison with state-of-the-art. Accordingly, this work offers a comprehensive review of available depth datasets as well as the loss functions that are applied in this problem domain. These depth datasets are categorised into five primary categories based on their application, namely (i) people detection and action recognition, (ii) faces and facial pose, (iii) perception-based navigation (i.e., street signs, roads), (iv) object and scene recognition, and (v) medical applications. The important characteristics and properties of each depth dataset are described and compared. A mixing strategy for depth datasets is presented in order to generalise model results across different environments and use cases. Furthermore, depth estimation loss functions that can help with training deep learning depth estimation models across different datasets are discussed. State-of-the-art deep learning-based depth estimation methods evaluations are presented for three of the most popular datasets. Finally, a discussion about challenges and future research along with recommendations for building comprehensive depth datasets will be presented as to help researchers in the selection of appropriate datasets and loss functions for evaluating their results and algorithms.

Motion-Robust Multimodal Heart Rate Estimation Using BCG Fused Remote-PPG With Deep Facial ROI Tracker and Pose Constrained Kalman Filter

The heart rate (HR) signal is so weak in remote photoplethysmography (rPPG) and ballistocardiogram (BCG) that HR estimation is very sensitive to face and body motion disturbance caused by spontaneous head and body movements as well as facial expressions of subjects in conversation. This article proposed a novel multimodal quasi-contactless HR sensor to ensure the robustness and accuracy of HR estimation under extreme facial poses, large-motion disturbances, and multiple faces in a video for computer-aided police interrogation. Specifically, we propose a novel landmark-based approach for a deep facial region of interest (ROI) tracker and face pose constrained Kalman filter to continuously and robustly track target facial ROIs for estimating HR from face and head motion disturbances in rPPG. This motion-disturbed rPPG signal is further fused with a minimally disturbed BCG signal by the face and head movements via a bank of notch filters with a recursive weighting scheme to obtain the dominant HR frequency for final accurate HR estimation. To facilitate reproducible HR estimation research, we synchronously acquire and publicly share a multimodal data set that contains 20 sets of ECG and BCG signals as well as uncompressed, rPPG-dedicated videos from ten subjects in a stable state and large-motion state (MS) without and with large face and body movements in a sitting position. We demonstrate through experimental comparisons that the proposed multimodal HR sensor is more robust and accurate than the state-of-the-art single-modal HR sensor solely with rPPG- or BCG-based methods. The mean absolute error (MAE) of HR estimation is 7.13 BPM lower than the BCG algorithm and 3.12 BPM lower than the model-based plane-orthogonal-to-skin (POS) algorithm in the MS.

Multi-view face pose recognition model construction based on a typical correlation analysis algorithm

Multi-view face pose recognition model construction based on a typical correlation analysis algorithm

Multi-view face pose recognition model construction based on a typical correlation analysis algorithm

In order to overcome the problems of large recognition error and low recognition accuracy in the existing face pose recognition models, the paper proposes and constructs a new multi-view face pose recognition model based on typical correlation analysis algorithm. First, the AdaBoost algorithm is used to realise multi-pose face detection and positioning. Secondly, the face image is pre-processed, including image greying, image denoising, and face image geometric normalisation, and then the typical correlation analysis algorithm is used to extract face features. Finally, multi-view facial gesture recognition is realised through convolutional neural network. Experimental results show that, compared with the traditional recognition model, the recognition accuracy of the constructed model is greatly improved, and the average accuracy (mAP) is 96.334%, which proves that the recognition performance of the constructed model is better.

Face pose correction based on morphable model and image inpainting

Face pose correction based on morphable model and image inpainting

Detailed Feature Guided Generative Adversarial Pose Reconstruction Network

Face frontalization is a critical and difficult task on face pose reconstruction. Previous researches use simple posture information as guidance, such as pose coding and facial landmarks. To explore the guidance effect of profile faces, we propose detailed features that provide much detailed information. In this paper, a D etailed Feature G uided Generative Adversarial P ose R econstruction Network (DGPR) is proposed. Firstly, frontal pose coding and profile detailed features are fed into DGPR to generate detailed features of front face. Then, the second generator combines frontal detailed features and profile face to reconstruct front face. Besides, we propose a conditional enhancement loss to strengthen the guiding role of detailed features, and a smoothing loss to reduce edge sharpness in generated faces. Experimental results show that our method generates photorealistic front faces and outperforms state-of-the-art methods on M2FPA and CAS-PEAL. Specifically, DGPR improves the face recognition accuracy under pose angles of ±60°, ±75°, ±90° by 2%, 1%, and 6% respectively over the state-of-the-art methods on M2FPA, achieves the average rank-1 recognition rate to 99.95% and improves it by 0.05% on CAS-PEAL. These results demonstrate the effects of detailed features and corresponding modules.

Improvement of a face recognition method for high jumper with a single sample based on Lucas-Kanade algorithm

In order to improve the identification accuracy of a dynamic single sample, a face recognition method based on Lucas-Kanade algorithm is proposed. The weighted Lucas-Kanade (LK) algorithm is used to obtain the single-sample affine transformation parameters of the high jumper's side face block and the corresponding front face block, and the optimal parameters of face pose correction are found through the maximum Gabor similarity, the method of face recognition for high jumper with a single sample is completed. Simulation results show that both the front face recognition rate and side-face recognition rate of the proposed method can reach more than 95% and the face recognition recall rate of the proposed method ranges from 90% to 100%. Compared with the traditional method, the recall rate has been significantly improved. In addition, when there are 440 face images, the recognition time is 1,177 ms, which is shorter than the traditional method.

FPLP3D: Security robot for face recognition in the workplace environment using face pose detection assisted controlled FACE++ tool position: A three-dimensional robot.

In recent years, several tracker systems have been developed to monitor a 3-dimensional skull position for facial action whereas, various tracker systems simultaneously analyze the single sequence of video, which can be provided with low-quality cameras and less security. Initially, implementing a 2-D face detector and an unrepentance system has been suggested; furthermore, it has been improved using an integrated 3-D face initialized scheme for the real-time tracker in the present face recognition systems. To overcome the present setbacks of the conventional systems, Face Pose Detection assisted controlled FACE++ tool position of Three-Dimensional Robot (FPLF3D) has been proposed in this article. Furthermore, the suggested proposed configuration has a high-end monitoring approach, which is used to improve the reliability of the robot's human-machine contact in the workplace environment for security assistance. Additionally, the robot's direction can be controlled by the operator's head position assessment of the camera (or any active viewing system) using a three-dimensional robot. Besides, the applications that are imitated by headers like telepresence, computer-generated reality, and video competitions will directly take advantage of the strategies introduced in this paper. Finally, real video tests at the lab-scale level show the accuracy and usefulness of the approaches proposed in this research outperform the existing methods used for tracking.

Detail 3D Face Reconstruction Based on 3DMM and Displacement Map

How to accurately reconstruct the 3D model human face is a challenge issue in the computer vision. Due to the complexity of face reconstruction and diversity of face features, most existing methods are aimed at reconstructing a smooth face model with ignoring face details. In this paper a novel deep learning‐based face reconstruction method is proposed. It contains two modules: initial face reconstruction and face details synthesis. In the initial face reconstruction module, a neural network is used to detect the facial feature points and the angle of the pose face, and 3D Morphable Model (3DMM) is used to reconstruct the rough shape of the face model. In the face detail synthesis module, Conditional Generation Adversarial Network (CGAN) is used to synthesize the displacement map. The map provides texture features to render to the face surface reconstruction, so as to reflect the face details. Our proposal is evaluated by Facescape dataset in experiments and achieved better performance than other current methods.

PFA-GAN: Pose Face Augmentation Based on Generative Adversarial Network

In this work, we propose a novel framework based on Generative Adversarial Networks for pose face augmentation (PFA-GAN). It enables a controlled pose synthesis of a new face image from a source face given a driving one while preserving the identity of the source face. We introduce a method for training the framework in a fully self-supervised mode using a large-scale dataset of unconstrained face images. Besides, some augmentation strategies are presented to expand the training set. The face verification experimental results demonstrate the effectiveness of the presented augmentation strategies as all augmented datasets outperform the baseline.

SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction.

Three-dimensional face dense alignment and reconstruction in the wild is a challenging problem as partial facial information is commonly missing in occluded and large pose face images. Large head pose variations also increase the solution space and make the modeling more difficult. Our key idea is to model occlusion and pose to decompose this challenging task into several relatively more manageable subtasks. To this end, we propose an end-to-end framework, termed as Self-aligned Dual face Regression Network (SADRNet), which predicts a pose-dependent face, a pose-independent face. They are combined by an occlusion-aware self-alignment to generate the final 3D face. Extensive experiments on two popular benchmarks, AFLW2000-3D and Florence, demonstrate that the proposed method achieves significant superior performance over existing state-of-the-art methods.

Multi-Task Head Pose Estimation in-the-Wild.

We present a deep learning-based multi-task approach for head pose estimation in images. We contribute with a network architecture and training strategy that harness the strong dependencies among face pose, alignment and visibility, to produce a top performing model for all three tasks. Our architecture is an encoder-decoder CNN with residual blocks and lateral skip connections. We show that the combination of head pose estimation and landmark-based face alignment significantly improve the performance of the former task. Further, the location of the pose task at the bottleneck layer, at the end of the encoder, and that of tasks depending on spatial information, such as visibility and alignment, in the final decoder layer, also contribute to increase the final performance. In the experiments conducted the proposed model outperforms the state-of-the-art in the face pose and visibility tasks. By including a final landmark regression step it also produces face alignment results on par with the state-of-the-art.

Unsupervised face Frontalization for pose-invariant face recognition

Face frontalization aims to normalize profile faces to frontal ones for pose-invariant face recognition. Current works have achieved promising results in face frontalization by using deep learning techniques. However, training deep models of face frontalization usually needs paired training data which is undoubtedly costly and time-consuming to acquire. To address this issue, we propose a Pose Conditional CycleGAN (PCCycleGAN) to generate authentic and identity-preserving frontal face images for pose-invariant face recognition. First, through coupling with a pair of inverse mappings, constraining with cycle consistent loss and using conditional pose label to control specific face pose generation, PCCycleGAN can be trained with unpaired samples. Second, pixel-level loss, feature space perception loss, and identity preserving loss are introduced in PCCycleGAN to help synthesize realistic and identity-preserving frontal face images. Extensive experiments on both constrained Multi-PIE dataset and unconstrained LFW and IJB-A datasets are conducted on face synthesis and pose-invariant face recognition. Results demonstrate that the proposed face frontalization model can synthesize frontal faces with high image quality as well as maintaining the identity information in both the constrained and unconstrained environments. In addition, our method enhances the performance of face recognition on the Multi-PIE, LFW and IJB-A datasets and achieves competitive face recognition performance on LFW and IJB-A datasets.

Face Pose Alignment with Event Cameras.

Event camera (EC) emerges as a bio-inspired sensor which can be an alternative or complementary vision modality with the benefits of energy efficiency, high dynamic range, and high temporal resolution coupled with activity dependent sparse sensing. In this study we investigate with ECs the problem of face pose alignment, which is an essential pre-processing stage for facial processing pipelines. EC-based alignment can unlock all these benefits in facial applications, especially where motion and dynamics carry the most relevant information due to the temporal change event sensing. We specifically aim at efficient processing by developing a coarse alignment method to handle large pose variations in facial applications. For this purpose, we have prepared by multiple human annotations a dataset of extreme head rotations with varying motion intensity. We propose a motion detection based alignment approach in order to generate activity dependent pose-events that prevents unnecessary computations in the absence of pose change. The alignment is realized by cascaded regression of extremely randomized trees. Since EC sensors perform temporal differentiation, we characterize the performance of the alignment in terms of different levels of head movement speeds and face localization uncertainty ranges as well as face resolution and predictor complexity. Our method obtained 2.7% alignment failure on average, whereas annotator disagreement was 1%. The promising coarse alignment performance on EC sensor data together with a comprehensive analysis demonstrate the potential of ECs in facial applications.

Edge‐guided single facial depth map super‐resolution using CNN

In recent years, consumer depth cameras have been widely used in digital entertainment and human-machine interaction due to the advantages of real-time performance and low cost. Facial depth maps have shown great potential in 3D-face-related studies. However, disadvantages of low resolution and precision limit its further applications. In this work, the authors propose an edge-guided convolutional neural network for single facial depth map super-resolution. It consists of two parts: an edge prediction sub-network and a depth reconstruction sub-network. The edge prediction sub-network generates an edge guidance map to guide the depth reconstruction sub-network to recover sharp edges and fine structures. Effective data augmentation methods are proposed as well. The network is patch-based and able to cope with any size of the input depth maps. In addition, it is insensitive to the face pose since the synthetic training dataset they generated covers a wide range of face poses. The proposed method is validated with three datasets including a synthetic facial depth data set, a real Kinect V2 facial depth data set and Middlebury Stereo Data set. Experimental results show that it outperforms the state-of-the-art methods on all the three data sets.

Context-Aware Attention Network for Human Emotion Recognition in Video

Recognition of human emotion from facial expression is affected by distortions of pictorial quality and facial pose, which is often ignored by traditional video emotion recognition methods. On the other hand, context information can also provide different degrees of extra clues, which can further improve the recognition accuracy. In this paper, we first build a video dataset with seven categories of human emotion, named human emotion in the video (HEIV). With the HEIV dataset, we trained a context-aware attention network (CAAN) to recognize human emotion. The network consists of two subnetworks to process both face and context information. Features from facial expression and context clues are fused to represent the emotion of video frames, which will be then passed through an attention network and generate emotion scores. Then, the emotion features of all frames will be aggregated according to their emotional score. Experimental results show that our proposed method is effective on HEIV dataset.

Unconstrained Face Identification Based on 3D Face Frontalization and Support Vector Guided Dictionary Learning

Face identification aims at putting a label on an unknown face with respect to some training set. Unconstrained face identification is a challenging problem because of the possible variations in face pose, illumination, occlusion, and facial expression. This paper presents an unconstrained face identification method based on face frontalization and learning-based data representation. Firstly, the frontal views of unconstrained face images are automatically generated by using a single, unchanged 3D face model. Then, we crop the face relevant regions of the frontal views to segment faces from the backgrounds. At last, to enhance the discriminative capability of the coding vectors, a support vector-guided dictionary learning (SVGDL) model is applied to adaptively assign different weights to different pairs of coding vectors. The performance of the proposed method FSVGDL (frontalization-based support vector guided dictionary learning) is evaluated on the Labeled Faces in the wild (LFW) database. After decision fusion, the identification accuracy yields 97.17% when using 7 images per individual for training and 3 images per individual for testing with 158 classes in total.

Identifikasi Pengenalan Wajah Perokok Menggunakan Metode Principal Component Analysis

Cigarettes are one of the biggest contributors to preventable causes of death in society. Cigarette smoke contains various chemicals that can cause various diseases such as chronic coughs, lung cancer, and other health problems. Cigarette smoke not only harms the health of the smoker itself but also the health of others. Sometimes written warnings about smoking bans are often not followed by active smokers. This study aims to identify smokers 'facial recognition in order to recognize and identify smokers' faces who do not obey the rules by using dimensional reduction techniques oriented to the Principal component Analysis (PCA) method. Principal Component Analysis will later be integrated with the Eigenface and Eucladean analysis algorithms to reduce the image size in obtaining the best value vectors to simplify the face image in the input image space and look for the threshold value which is the threshold that the test data must pass so that it can prove the data value. testing becomes recognizable data through the calculation of the distance for each weight. In this study, there were 8 smoker faces with 5 different facial poses that were tested for 40 face recognition experiments and resulted in 34 correct smoker face recognition and 6 wrong smoker face recognition with an accuracy rate of 92.5% and a long face recognition process time of 80. second. This test has proven that the Eigenface and Euclidean distance in the Principal Component Analysis (PCA) are able to handle and recognize smoker's facial image data well.