Frontal Face Research Articles

Joint Face Image Restoration and Frontalization for Recognition

In real-world scenarios, many factors may harm face recognition performance, e.g., large pose, bad illumination,low resolution, blur and noise. To address these challenges, previous efforts usually first restore the low-quality faces to high-quality ones and then perform face recognition. However, most of these methods are stage-wise, which is sub-optimal and deviates from the reality. In this paper, we address all these challenges jointly for unconstrained face recognition. We propose an Multi-Degradation Face Restoration (MDFR) model to restore frontalized high-quality faces from the given low-quality ones under arbitrary facial poses, with three distinct novelties. First, MDFR is a well-designed encoder-decoder architecture which extracts feature representation from an input face image with arbitrary low-quality factors and restores it to a high-quality counterpart. Second, MDFR introduces a pose residual learning strategy along with a 3D-based Pose Normalization Module (PNM), which can perceive the pose gap between the input initial pose and its real-frontal pose to guide the face frontalization. Finally, MDFR can generate frontalized high-quality face images by a single unified network, showing a strong capability of preserving face identity. Qualitative and quantitative experiments on both controlled and in-the-wild benchmarks demonstrate the superiority of MDFR over state-of-the-art methods on both face frontalization and face restoration.

DWT based Person Re-Identification using GAN

The recent development in person re-identification has challenging task for variations in pose, illumination, expression, and also similar appearance between two different persons. In this paper, we propose Discrete Wavelet Transform (DWT) based person re-identification using Generative Adversarial Network (GAN). The CMU multi-PIE face database with multiple viewpoints and illuminations is considered to test the model. The profile side view face images to be tested are converted into frontal face images using Two-pathway generator adversarial network (TP-GAN). The frontal face images are loaded into the server to create server database. The synthesized TP-GAN images and server database images are pre-processed to convert RGB into grayscale images and also to convert into uniform face image dimensions. The person re-identification is based on feature extraction through DWT, which generates one low frequency LL band and three high frequency bands LH, HL and HH. The LL band coefficients are considered as final features, which are noise-free and compressed number of features. The features of profile side view images and server database images are compared using Normalized Euclidean Distance (NED) and threshold values for person re-identification.

2D facial landmark localization method for multi-view face synthesis image using a two-pathway generative adversarial network approach.

One of the key challenges in facial recognition is multi-view face synthesis from a single face image. The existing generative adversarial network (GAN) deep learning methods have been proven to be effective in performing facial recognition with a set of pre-processing, post-processing and feature representation techniques to bring a frontal view into the same position in-order to achieve high accuracy face identification. However, these methods still perform relatively weak in generating high quality frontal-face image samples under extreme face pose scenarios. The novel framework architecture of the two-pathway generative adversarial network (TP-GAN), has made commendable progress in the face synthesis model, making it possible to perceive global structure and local details in an unsupervised manner. More importantly, the TP-GAN solves the problems of photorealistic frontal view synthesis by relying on texture details of the landmark detection and synthesis functions, which limits its ability to achieve the desired performance in generating high-quality frontal face image samples under extreme pose. We propose, in this paper, a landmark feature-based method (LFM) for robust pose-invariant facial recognition, which aims to improve image resolution quality of the generated frontal faces under a variety of facial poses. We therefore augment the existing TP-GAN generative global pathway with a well-constructed 2D face landmark localization to cooperate with the local pathway structure in a landmark sharing manner to incorporate empirical face pose into the learning process, and improve the encoder-decoder global pathway structure for better representation of facial image features by establishing robust feature extractors that select meaningful features that ease the operational workflow toward achieving a balanced learning strategy, thus significantly improving the photorealistic face image resolution. We verify the effectiveness of our proposed method on both Multi-PIE and FEI datasets. The quantitative and qualitative experimental results show that our proposed method not only generates high quality perceptual images under extreme poses but also significantly improves upon the TP-GAN results.

Head pose estimation: An extensive survey on recent techniques and applications

Biometric based systems are involved in many areas, from surveillance to user authentication, from autonomous systems to human-robot interactions. Head pose estimation (HPE) is the task to support biometric systems in which any of the biometric traits of the head is involved, as face, ear or iris. This particular biometric branch finds its application in driver attention detection, surveillance for recognition, face frontalization, best frame selection and so on. The goal of HPE is to determine the head pose orientation (yaw, pitch, roll). The implemented methods use different techniques depending on the kind of input. In this survey we present an overview of involved datasets, recent techniques and applications. We evaluate and compare the different approaches with respect to their advantages and practical usage. In addition, we propose a technical comparison between training and training-free techniques for the most popular HPE methods.

Numerical and experimental investigation of fiber metal laminates with elastomeric layers under low‐velocity impact

AbstractThis study aims to investigate the effects of elastomers on fiber metal laminates (FMLs) subjected to low‐velocity impact loads. A compounded natural rubber layer was added to conventional FMLs containing glass/epoxy composite plies and Al 6061‐T6 layers, measuring its effect on the behavior of the structure in low‐velocity indentation at energy levels of 25 and 45 J. It was found that the addition of an elastomeric layer to the back face of the composite layer increased structural toughness, pre‐fracture deformation, and specific energy absorption while reducing damage and the maximum load. Moreover, positioning the elastomer at a closer distance to the frontal face reduced maximum load and energy absorption capacity. Then, the standard material characterization of tensile, shear, and compressive were performed via the universal testing machine and the split Hopkinson pressure bar apparatus. A numerical model was developed based on the advanced finite element code of LS‐DYNA, and the results were validated by comparison to the experimental data.

Frontal-Centers Guided Face: Boosting Face Recognition by Learning Pose-Invariant Features

In recent years, face recognition has made a remarkable breakthrough due to the emergence of deep learning. However, compared with frontal face recognition, plenty of deep face recognition models still suffer serious performance degradation when handling profile faces. To address this issue, we propose a novel Frontal-Centers Guided Loss (FCGFace) to obtain highly discriminative features for face recognition. Most existing discriminative feature learning approaches project features from the same class into a separated latent subspace. These methods only model the distribution at the identity-level but ignore the latent relationship between frontal and profile viewpoints. Different from these methods, FCGFace takes viewpoints into consideration by modeling the distribution at both the identity-level and the viewpoint-level. At the identity-level, a softmax-based loss is employed for a relatively rough classification. At the viewpoint-level, centers of frontal face features are defined to guide the optimization conducted in a more refined way. Specifically, our FCGFace is capable of adaptively adjusting the distribution of profile face features and narrowing the gap between them and frontal face features during different training stages to form compact identity clusters. Extensive experimental results on popular benchmarks, including cross-pose datasets (CFP-FP, CPLFW, VGGFace2-FP, and Multi-PIE) and non-cross-pose datasets (YTF, LFW, AgeDB-30, CALFW, IJB-B, IJB-C, and RFW), have demonstrated the superiority of our FCGFace over the SOTA competitors.

Pre-Trained Feature Fusion and Multidomain Identification Generative Adversarial Network for Face Frontalization

The study of face frontalization is essential for improving face recognition accuracy in extreme pose scenarios. Mainstream methods like TP-GAN, CAPG-GAN, etc., have made meaningful contributions. However, they still suffer from two problems: the lack of extracted feature diversity and the blurred details in generated images. This paper proposes a pre-trained feature fusion and multi-domain identification generative adversarial network (PM-GAN) for face frontalization: the features of the model pre-trained on large-scale datasets are fused with the original features of the encoder to enhance the diversity and robustness of features. In order to fuse features more effectively, we designed a novel feature fusion module (FFM). In addition, a group of global and local discriminators is introduced to reinforce the local details and realism of generated frontal faces. Experimental results show that our proposed method outperforms state-of-the-art methods on M²FPA and CAS-PEAL datasets.

Quantitative analysis of facial proportions and facial attractiveness among Asians and Caucasians.

It has been proposed that the proportions of the human face are crucial for facial aesthetics. If this is the case, we should describe the relationship among proportions of face components quantitatively. This study aims to develop a mathematical model of facial proportions to provide a quantitative description of facial attractiveness. Furthermore, we expect that plastic surgeons can use models in clinical work to enhance communication efficiency between doctors and patients. Face alignment technique was used to analyse 5500 frontal faces with diverse properties (male/female, Asian/Caucasian, ages) to obtain the ratios among the nose length ($ {N}_{L} $), the nasal base width ($ N $), and the inner canthus width ($ {E}_{I} $). A mathematical model ($ {N}_{L}^{2} = a{E}_{I}\mathrm{*}{N}_{L}+b{E}_{I}\mathrm{*}N+cN\mathrm{*}{N}_{L} $) was developed to describe the relationship among these proportions. To validate the effectiveness of this approach, we simulated the post-operative photos using Adobe Photoshop. Our findings show that the ratio of nose length to nose width, the ratio of inner canthus width to nose length and the ratio of inner canthus to nose width play a significant role in determining facial attractiveness. These results provide a possible strategy to quantitatively describe the relationship among human face proportions.

Real Time Customer Satisfaction Analysis using Facial Expressions and Headpose Estimation

One of the most exciting, innovative, and promising topics in marketing research is the quantification of customer interest. This work focuses on interest detection and provides a deep learning-based system that monitors client behaviour. By assessing head position, the recommended method assesses customer attentiveness. Customers whose heads are directed toward the promotion or the item of curiosity are identified by the system, which analyses facial expressions and records client interest. An exclusive method is recommended to recognize frontal face postures first, then splits facial components that are critical for detecting facial expressions into iconized face pictures. Mainly consumer interest monitoring will be executed. Finally, the raw facial images are combined with the iconized face image's confidence ratings to estimate facial emotions. This technique combines local part-based characteristics through holistic face data for precise facial emotion identification. The new method provides the dimension of required marketing and product findings indicate that the suggested architecture has the potential to be implemented because it is efficient and operates in real time.

A web-based application for face detection in real-time images and videos

Face detection is widely used in the consumer industry such as advertising, user interfaces, video streaming apps and in many security applications. Every application has its own demands and constraints, and hence cannot be fulfilled by a single face detection algorithm. In this work, we developed an interactive web-based application for face detection in real-time images and videos. Pretrained face detection algorithms, namely Haar cascade classifier, HOG-based frontal face detector, Multi-task Cascaded Convolutional Neural Network (MTCNN) and Deep Neural Network (DNN), were used in the web-based application. A performance analysis of these face detection algorithms is done for various parameters such as different lighting conditions, face occlusion and frame rate. The web app interface can be used for an easy comparison of different face detection algorithms. This will help the user to decide on the algorithm that suits their purpose and requirements for various applications.

Face frontalization for uncontrolled scenes

Face frontalization for uncontrolled scenes

코의 크기 및 형태와 자가건강, 미병과의 상관성

Mibyeong (sub-health) is a concept that represents the sub-health in traditional East Asian medicine. Assuming that the nose sizes and shapes are related to respiratory function, in this study, we hypothesized that the nose size and shape features are related to the self-rated health (SRH) level and self-rated Mibyeong severity, and aimed to assess this relationship using a fully automated image analysis system. The nose size features were evaluated from the frontal and profile face images of 810 participants. The nose size features consisted of five length features, one area feature, and one volume feature. The level of SRH and the Mibyeong severity were determined using a questionnaire. The normalized nasal height was negatively associated with the self-rated health score (SRHS) (partial ρ = -0.125, p = 3.53E-04) and the Mibyeong score (MBS) (partial ρ = -.172, p = 9.38E-07), even after adjustment for sex, age, and body mass index. The normalized nasal volume (ρ = -.105, p = 0.003), the normalized nasal tip protrusion length (ρ = -.087, p = 0.014), and the normalized nares width (ρ = -.086, p = .015) showed significant correlation with the SRHS. The normalized nasal area (ρ = -.118, p = 0.001), the normalized nasal volume (ρ = -.107, p = .002) showed significant correlation with the MBS. The wider, longer, and larger the nose, the lower the SRHS and MBS, indicating that health status can be estimated based on the size and shape features of the nose.

Joint 3D facial shape reconstruction and texture completion from a single image

Recent years have witnessed significant progress in image-based 3D face reconstruction using deep convolutional neural networks. However, current reconstruction methods often perform improperly in self-occluded regions and can lead to inaccurate correspondences between a 2D input image and a 3D face template, hindering use in real applications. To address these problems, we propose a deep shape reconstruction and texture completion network, SRTC-Net, which jointly reconstructs 3D facial geometry and completes texture with correspondences from a single input face image. In SRTC-Net, we leverage the geometric cues from completed 3D texture to reconstruct detailed structures of 3D shapes. The SRTC-Net pipeline has three stages. The first introduces a correspondence network to identify pixel-wise correspondence between the input 2D image and a 3D template model, and transfers the input 2D image to a U-V texture map. Then we complete the invisible and occluded areas in the U-V texture map using an inpainting network. To get the 3D facial geometries, we predict coarse shape (U-V position maps) from the segmented face from the correspondence network using a shape network, and then refine the 3D coarse shape by regressing the U-V displacement map from the completed U-V texture map in a pixel-to-pixel way. We examine our methods on 3D reconstruction tasks as well as face frontalization and pose invariant face recognition tasks, using both in-the-lab datasets (MICC, MultiPIE) and in-the-wild datasets (CFP). The qualitative and quantitative results demonstrate the effectiveness of our methods on inferring 3D facial geometry and complete texture; they outperform or are comparable to the state-of-the-art.

Drowsy Driver Detection Using Viola-Jones Algorithm

Drowsy driving is one of the factors that lead to road accidents which can cause dead. This is because driver does not able to give fully attention while driving. There are many factors that lead to driver drowsiness such as driving for a long time, do not have enough sleep and shift work. Thus, this research is proposed to develop a system to detect and alert drowsy driver by using Viola-Jones algorithm. Blinking rate is used as the indicator to determine either the driver is in drowsy or awake state. Viola-Jones algorithm is used to detect driver’s face and eyes in real time. Haar cascade classifier for frontal face and glasses eyes are used to train the system to detect driver’s face and eyes. In order to calculate eye blink, Eye Aspect Ratio (EAR) calculation is used to calculate and estimate of the eye-opening state in this system. The results of testing showed that the system with the Viola-Jones algorithm and Haar cascade classifier able to detect eyes blinking rate at the high accuracy percentages.

An Update on the Diagnosis, Treatment, and Management of Occipital Neuralgia.

This report intends to summarize the underlying pathophysiology, relevant symptoms, appropriate diagnostic workup, necessary imaging, and medical and surgical treatments of occipital neuralgia (ON). This was done through a comprehensive literature review of peer-reviewed literature throughout the most relevant databases. The current understanding of ON is that it causes neuropathic pain in the distribution of the greater occipital nerve, the lesser occipital nerve, the third occipital nerve or a combination of the 3. It is currently a subset of headaches although there is some debate if ON should be its own condition. Occipital neuralgia causes chronic, sharp, stabbing pain in the upper neck, back of the head, and behind the ears that can radiate to the front of the head. Diagnosis is typically clinical and patients present with intermittent, painful episodes associated with the occipital region and the nerves described above. Most cases are unilateral pain, however bilateral pain can be present and the pain can radiate to the frontal region and face. Physical examination is the first step in management of this disease and patients may demonstrate tenderness over the greater occipital and lesser occipital nerves. Anesthetics like 1% to 2% lidocaine or 0.25% to 0.5% bupivacaine can be used to block these nerves and antiinflammatory drugs like corticosteroids can be used in combination to prevent compressive symptoms. Other treatments like botulinum toxin and radiofrequency ablation have shown promise and require more research. Surgical decompression through resection of the obliquus capitis inferior is the definitive treatment however there are significant risks associated with this procedure.

TFE: A Transformer Architecture for Occlusion Aware Facial Expression Recognition.

Facial expression recognition (FER) in uncontrolled environment is challenging due to various un-constrained conditions. Although existing deep learning-based FER approaches have been quite promising in recognizing frontal faces, they still struggle to accurately identify the facial expressions on the faces that are partly occluded in unconstrained scenarios. To mitigate this issue, we propose a transformer-based FER method (TFE) that is capable of adaptatively focusing on the most important and unoccluded facial regions. TFE is based on the multi-head self-attention mechanism that can flexibly attend to a sequence of image patches to encode the critical cues for FER. Compared with traditional transformer, the novelty of TFE is two-fold: (i) To effectively select the discriminative facial regions, we integrate all the attention weights in various transformer layers into an attention map to guide the network to perceive the important facial regions. (ii) Given an input occluded facial image, we use a decoder to reconstruct the corresponding non-occluded face. Thus, TFE is capable of inferring the occluded regions to better recognize the facial expressions. We evaluate the proposed TFE on the two prevalent in-the-wild facial expression datasets (AffectNet and RAF-DB) and the their modifications with artificial occlusions. Experimental results show that TFE improves the recognition accuracy on both the non-occluded faces and occluded faces. Compared with other state-of-the-art FE methods, TFE obtains consistent improvements. Visualization results show TFE is capable of automatically focusing on the discriminative and non-occluded facial regions for robust FER.

Inattentive Drowsiness Behavior Profile Detection with Heavy Eyed Interval Approach for Biomedical Applications in Health Monitoring System

The efficient tracking of vehicle drivers can be used to prevent collisions through visual human behaviour analysis. Many different methods have not been satisfactory enough such as iris-sklera research, driver’s approximation of gaze, and Hough transforming technological performance. Since these methods make it more difficult to spot drivers’ sleepiness and carelessness. This paper therefore suggested that it be careful to estimate the profile after finding the left eye, right eye, mouth and nose Absence of each of these traits marks a non-frontal approach. The Rectangular Face Classificatión control system monitors frontal faces by moving a rectangular filter on the image for testing the dullness of the face area. Once the facial regions are tracked, the Hybrid Balanced Networks separates the eye area from it depending on the greater axis and the smaller axis. Heavy Eyed Approach is often used to spot drowsiness and twitch of the brow. The intensity of the horizontal plot is measured and successive frames in the eye twitch are not counted as a closed eye for three seconds. The result of the proposed work therefore effectively improves accuracy efficiency.

다수의 얼굴 인식을 통한 자동출석체크 시스템

Non-contact biometric identification technology is once again receiving social attention these days when untact technology is required due to the Covid-19. Face recognition technology is being used in a variety of platforms such as airport immigration systems, hospital patient management systems, financial self-authentication systems, criminal suspect detection systems, and personal authentication systems for mobile devices, and is showing its effectiveness. In this paper, multiple face recognition technology is applied to the automatic attendance system. Unlike when only one person is recognized, multiple face recognition results in lower accuracy and slower processing speed. In this paper, a frontal face detection algorithm is proposed. Using the frontal face detection algorithm, only the frontal face was used for recognition, and when 36 people were recognized, 97.2% accuracy was achieved. In addition, we proposed an erasing multiple face recognition method and increased the average number of frames processed per second as the recognition completion rate increased.

Pharmacy Impact for Distinguishing Normal Face from Abnormal Face Due to COVID- 19

In today’s world face detection is the most important task. Due to the chromosomes disorder sometimes a human face suffers from different abnormalities. In the recent scenario, the entire globe is facing enormous health risks occurred due to Covid-19. To fight against this deadly disease, consumption of drugs is essential. Consumption of drugs may provide some abnormalities to human face. For example, one eye is bigger than the other, cliff face, different chin-length, variation of nose length, length or width of lips are different, etc. To assess these human face abnormalities, the application of computer vision is favoured in this study. This work analyses an input image of human’s frontal face and performs a segregation method to separate the abnormal faces. In this research work, a method has been proposed that can detect normal or abnormal faces from a frontal input image due to COVID-19. This method has used Fast Fourier Transformation (FFT) and Discrete Cosine Transformation of frequency domain and spatial domain analysis to detect those faces.

Identification of difficult to intubate patients from frontal face images using an ensemble of deep learning models.

Failure to identify difficult intubation is the leading cause of anesthesia-related death and morbidity. Despite preoperative airway assessment, 75-93% of difficult intubations are unanticipated, and airway examination methods underperform, with sensitivities of 20-62% and specificities of 82-97%. To overcome these impediments, we aim to develop a deep learning model to identify difficult to intubate patients using frontal face images. We proposed an ensemble of convolutional neural networks which leverages a database of celebrity facial images to learn robust features of multiple face regions. This ensemble extracts features from patient images (n=152) which are subsequently classified by a respective ensemble of attention-based multiple instance learning models. Through majority voting, a patient is classified as difficult or easy to intubate. Whereas two conventional bedside tests resulted in AUCs of 0.6042 and 0.4661, the proposed method resulted in an AUC of 0.7105 using a cohort of 76 difficult and 76 easy to intubate patients. Generic features yielded AUCs of 0.4654-0.6278. The proposed model can operate at high sensitivity and low specificity (0.9079 and 0.4474) or low sensitivity and high specificity (0.3684 and 0.9605). The proposed ensembled model outperforms conventional bedside tests and generic features. Side facial images may improve the performance of the proposed model. The proposed method significantly surpasses conventional bedside tests and deep learning methods. We expect our model will play an important role in developing deep learning methods where frontal face features play an important role.