Face Alignment Algorithm Research Articles

GANs-generated synthetic datasets for face alignment algorithms in complex environments

Face alignment has matured over the past several decades, but privacy violations or data abuse have also triggered global controversy. Moreover, existing face algorithms are still challenging in complex environments. For the question: ”Can synthetic datasets introduce novel variations in real-world data?”. We proposed a new research direction concerning key point detection tasks utilizing synthetic datasets, aiming to reduce the model’s reliance on real-world datasets. Considering the differences between synthetic and real-world data, our work proposed two different transfer ways based on GANs: (1) S→R model converts the synthetic face images generated by the Face middleware 3D model (FaceGen) into more realistic face images for training face alignment. (2) R→S model converts the real-world face images into a synthetic style image for testing face alignment. Extensive experiments explored the synthetic data complementarity and availability.

Diverse distant-students deep emotion recognition and visualization

Distance learning through online platforms has recently emerged as a significant addition to the educational system, providing support and enhancing traditional delivery modes in schools. Instructors need visual feedback to monitor students’ engagement. This research aims to investigate the emotional state of culturally diverse students in the country throughout the lecture and provide teachers with an interactive dashboard to monitor students’ emotional states during a particular lecture. We collected a specialized dataset that includes Arab students posing for various emotions specifically related to lecture scenarios. Face detection and alignment algorithm is then applied. Finally, we utilize Neural Architecture Search (NAS) to optimize emotion classification architecture. The overall accuracy of the model is 86.29% on our collected dataset and 98.84% on the CK+ dataset. The live testing of the system shows the emotions clearly with real-time feedback. The AI-powered dashboard gives instructors insights into student engagement during distance learning, which induces data-driven decisions to optimize teaching strategies.

Face Recognition Based Student Attendance System

The term 'biometrics' refers to a measurable characteristic that is unique to an individual such as fingerprints, facial structure, the iris or a person's voice. This project presents a face image based biometric system that records the attendance of a person by using a hand held face image biometrics. Attendance is a concept that exists in different places like institutions, organization’s, hospitals, etc. during the start and end of the day to mark a person’s presence. In early days and even now in many places attendance is recorded manually in attendance registers by calling out the names. This results in waste of time and human effort. Also there are many fraudulent issues that happen when we use a register. A student attendance system using face recognition and Grassmann classification is a biometric system that can automate the attendance taking process in a classroom setting. This system involves collecting a dataset of images of each student in the classroom and using face detection and alignment algorithms to detect and align the faces in each image. The Grassmann classification algorithm is then used to extract features from the images and classify them as belonging to a particular student. When a student enters the classroom, their face is captured by a camera and compared to the images in the dataset. If a match is found, the student is marked as present. This system has the potential to save time and increase accuracy in the attendance taking process. However, it also raises concerns about privacy and the security of the collected data. Proper safeguards must be put in place to ensure that the system is used ethically and responsibly

Non-Intrusive Real Time Eye Tracking Using Facial Alignment for Assistive Technologies.

Most affordable eye tracking systems use either intrusive setup such as head-mounted cameras or use fixed cameras with infrared corneal reflections via illuminators. In the case of assistive technologies, using intrusive eye tracking systems can be a burden to wear for extended periods of time and infrared based solutions generally do not work in all environments, especially outside or inside if the sunlight reaches the space. Therefore, we propose an eye-tracking solution using state-of-the-art convolutional neural network face alignment algorithms that is both accurate and lightweight for assistive tasks such as selecting an object for use with assistive robotics arms. This solution uses a simple webcam for gaze and face position and pose estimation. We achieve a much faster computation time than the current state-of-the-art while maintaining comparable accuracy. This paves the way for accurate appearance-based gaze estimation even on mobile devices, giving an average error of around 4.5°on the MPIIGaze dataset [1] and state-of-the-art average errors of 3.9°and 3.3°on the UTMultiview [2] and GazeCapture [3], [4] datasets respectively, while achieving a decrease in computation time of up to 91%.

Driver state detection for driver-automation shared control with fuzzy logic

This paper focuses on driver-automation shared lateral control by considering the variation of driver state which is interfered by multiple-risk abnormal behaviours. First, four abnormal behaviours, i.e. smoking, calling, yawning and drowsiness, under Driver Monitoring System are detected by a computer vision based method which combines face alignment algorithm with Haar–AdaBoost. A novel fuzzy logic system is then designed by analysing the risk of the four behaviours, aiming to identify the risk levels of driver states. Based on the identified results, a fuzzy inference logic is developed to design the driver-automation shared control by using a PID controller. Simulation experiments are conducted to illustrate the effectiveness of the detection method and the designed controller. The comparison between the simulation results shows that the proposed control architecture has comparatively better performance in lane keeping task.

Multi-face detection and alignment using multiple kernels

Multi-face detection and alignment techniques under unlimited environment are challenging issues. In recent years, the demand for face detection and alignment techniques has increased in many areas, including automatic drive and security. However, some mainstream algorithms, such as the Multi-task Cascaded Convolutional Networks (MTCNN) algorithm cannot transfer to multi-face detection and alignment, which introduces new multi-size, multi-resolution, and multi-angle challenges. This paper proposes an improved algorithm—Multi-face-MTCNN for precise original face detection and alignment algorithm when there is an overlapping face scenario. We design two new network structures: Pixelfusion-MTCNN and Twoconv-MTCNN. Moreover, we propose new data augmentation method and optimized detection process which are applied in the Multi-face-MTCNN algorithm. The limitations associated with single-scale kernel size in MTCNN are solved to obtain a satisfactory performance. Compared to the MTCNN algorithm, experimental analysis of the FDDB dataset show that 1.766% improves Multi-face-MTCNN. Meanwhile, on the WIDER FACE verification benchmark, respectively, with regards to the three sub-datasets, the proposed algorithm’s performance is improved by 3.426%, 2.776%, and 21.576%. Besides, average alignment error of the left eye, right eye, nose, left mouth, and right mouth is performed on the proposed algorithm.

Facial feature point recognition method for human motion image using GNN

<abstract> <p>To address the problems of facial feature point recognition clarity and recognition efficiency in different human motion conditions, a facial feature point recognition method using Genetic Neural Network (GNN) algorithm was proposed. As the technical platform, weoll be using the Hikey960 development board. The optimized BP neural network algorithm is used to collect and classify human motion facial images, and the genetic algorithm is introduced into neural network algorithm to train human motion facial images. Combined with the improved GNN algorithm, the facial feature points are detected by the dynamic transplantation of facial feature points, and the detected facial feature points are transferred to the face alignment algorithm to realize facial feature point recognition. The results show that the efficiency and accuracy of facial feature point recognition in different human motion images are higher than 85% and the performance of anti-noise is good, the average recall rate is about 90% and the time-consuming is short. It shows that the proposed method has a certain reference value in the field of human motion image recognition.</p> </abstract>

Robust face alignment by dual-attentional spatial-aware capsule networks

Face alignment in-the-wild still faces great challenges due to that i) partial occlusion blurs the inter-features spatial relations of faces and ii) traditional CNN makes the network more difficult to capture the spatial positional relations between landmarks. To address the issues above, we propose a face alignment algorithm named Dual-attentional Spatial-aware Capsule Network (DSCN). Firstly, the spatial-aware module builds a more accurate inter-features spatial constrained model with the hourglass capsule network (HGCaps) as the backbone, which can effectively enhance its robustness against occlusions. Then, two sorts of attention mechanisms, namely capsule attention and spatial attention, are added to the attention-guided module to make the network focus more on the advantageous features and suppress other unrelated ones for more effective feature recalibration. Our method achieves 1.08% failure rate on the COFW dataset, which is much lower than the current state-of-the-art algorithms. The mean error under 300W dataset and WFLW dataset are respectively 3.91% and 5.66%, which shows that DSCN is more robust to occlusion and outperforms state-of-the-art methods in the literature.

Face Alignment Algorithm Based on an Improved Cascaded Convolutional Neural Network

Aiming at the problem of a large number of parameters and high time complexity caused by the current deep convolutional neural network models, an improved face alignment algorithm of a cascaded convolutional neural network (CCNN) is proposed from the network structure, random perturbation factor (shake), and data scale. The algorithm steps are as follows: 3 groups of lightweight CNNs are designed; the first group takes facial images with face frame as input, trains 3 CNNs in parallel, and weighted outputs the facial images with 5 facial key points (anchor points). Then, the anchor points and 2 different windows with a shake mechanism are used to crop out 10 partial images of human faces. The networks in the second group train 10 CNNs in parallel and every 2 networks’ weighted average and colocated a key point. Based on the second group of networks, the third group designed a smaller shake mechanism and windows, to achieve more fine-tuning. When training the network, the idea of parallel within groups and serial between groups is adopted. Experiments show that, on the LFPW face dataset, the improved CCNN in this paper is superior to any other algorithm of the same type in positioning speed, algorithm parameter amount, and test error.

DC-EDN: densely connected encoder-decoder network with reinforced depthwise convolution for face alignment

High accuracy and fast face alignment algorithms play an important role in many face-related applications. Generally, the model speed is inversely related to the number of parameters. We construct our network based on densely connected encoder-decoders, which is an efficient method to balance the parameter number and localization results. In each encoder-decoder, we introduce stacking depthwise convolution and depthwise feature fusion within the same channel, which greatly improves the performance of depthwise convolution and reduces the number of model parameters. In addition, we enhance the mean square loss function by assigning different penalty weights to each coordinate according to the distance to the position corresponding to the maximum value in the label heatmap. Experiments show that the model with the improved loss function obtains better localization results. In the experiment, we compare our method to state-of-the-art methods based on 300W and WFLW. The localization error is 2.76% with the common subset of 300W and the model size (0.7M) is small and even utilizes approximately 1% of the number of parameters of the other models. The dataset and model based on WFLW are publicly available at https://github.com/iam-zhanghongliang/DC-EDN .

Attention Span Prediction Using Head-Pose Estimation With Deep Neural Networks

Automated human pose estimation is evolving as an exciting research area in human activity detection. It includes sophisticated applications such as malpractice detection in the examination, distracted driving, gesture detection, etc., and requires robust and reliable pose estimation techniques. These applications help to map the attention of the user with head pose estimation (HPE) metrics supported by emotion and gaze analysis. This paper solves the problem of attention score estimation with HPE. The proposed method ensures ease of implementation while addressing head pose estimation using 68 facial features. Further, to attain reliability and precision, head pose estimation has been implemented as a regression task. The coordinate pair angle method (CPAM) with deep neural network (DNN) regression and elastic net regression is carried out. The use of DNN ensures precision on low lighting, distorted or occluded images. CPAM methodology leverages facial landmark detection and angular difference to estimate head pose. Experimentation results showed that the proposed model could handle large datasets, real-time data processing, significant pose variations, partial occlusions, and diverse facial expressions with a mean absolute error (MAE) of 3° and less. The proposed system was evaluated on three standard databases: the 300W across large poses (300W-LP) dataset, annotated facial landmarks in the wild (AFLW2000) dataset, and the national institute of mental health child emotional faces picture set (NIMH - ChEFS ) dataset. The results achieved are on par with recent state-of-the-art methodologies such as anisotropic angle distribution learning (AADL), joint head pose estimation and face alignment algorithm (JFA), rotation axis focused attention network (RAFA-Net), and propose an MAE ranging up to 6°. The paper could achieve remarkable results for attention span prediction using head pose estimation and for many possible future applications.

Small and accurate heatmap-based face alignment via distillation strategy and cascaded architecture

Despite face alignment has made significant progress, it is still challenging to get a small and accurate face landmark detection model. In this paper, we focus on compressing heatmap-based face alignment algorithms using a novel proposed distillation strategy. We find that the activated areas of heatmaps generated from the well-trained teacher net can capture more local shape information of the facial parts than the ground-truth ones generated from the standard Gaussian distribution. To simultaneously transfer such shape information and correct the mis-predicted heatmaps generated from the teacher model, we first modify the heatmaps of teacher model by replacing the mis-predicted heatmaps with the ground-truth ones as the labels for the student net. To further improve the accuracy of the student net, we investigate the correlation between the extracted features and the predicted heatmaps, and divide the landmarks into two categories: simple and hard. A cascaded architecture is designed which firstly detects the simple points based on the extracted features, and then predicts the hard points resorting to the heatmaps of simple ones. Finally, a face alignment model with 3.64M parameters is obtained, which is about 6x smaller than the cumbersome model, and outperforms the state-of-the-art algorithms on both AFLW2000-3D and 300W-LP. The model and code are released in https://github.com/snow-rgb/Fast-Face-Alignment.

A New Dataset for Facial Motion Analysis in Individuals With Neurological Disorders.

We present the first public dataset with videos of oro-facial gestures performed by individuals with oro-facial impairment due to neurological disorders, such as amyotrophic lateral sclerosis (ALS) and stroke. Perceptual clinical scores from trained clinicians are provided as metadata. Manual annotation of facial landmarks is also provided for a subset of over 3300 frames. Through extensive experiments with multiple facial landmark detection algorithms, including state-of-the-art convolutional neural network (CNN) models, we demonstrated the presence of bias in the landmark localization accuracy of pre-trained face alignment approaches in our participant groups. The pre-trained models produced higher errors in the two clinical groups compared to age-matched healthy control subjects. We also investigated how this bias changes when the existing models are fine-tuned using data from the target population. The release of this dataset aims to propel the development of face alignment algorithms robust to the presence of oro-facial impairment, support the automatic analysis and recognition of oro-facial gestures, enhance the automatic identification of neurological diseases, as well as the estimation of disease severity from videos and images.

Multi-view face recognition using deep neural networks

F Face recognition has been widely used in modern intelligent systems, such as smart video surveillance, online payment, and intelligent access control system. Existing face recognition algorithms are prone to be attacked by various face presentation attacks (face-PAs), such as printed paper, video replay, and silicone masks. To optimally handle the aforementioned problems, we formulate a novel deep architecture to increase the accuracy of multi-view human face recognition. In particular, in the first place, a novel deep neural network is built for deeply encoding the face regions, where a novel face alignment algorithm is employed to localize the key points inside faces. Subsequently, we utilize the well-known PCA for reducing the dimensionality of the deep features and simultaneously, removing the redundant and contaminated visual features. Thereafter, we propose a joint Bayesian framework in order to evaluate the similarity of feature vectors and highly competitive face classification accuracy can be achieved. Comprehensive experiments were conducted on our compiled CAS-PEAL dataset and achieved a 98.52% face recognition performance. Moreover, our proposed face recognition system can robustly handle various face recognition attack under various contexts.

Towards stabilizing facial landmark detection and tracking via hierarchical filtering: A new method

Many facial landmark detection and tracking methods suffer from instability problems that have a negative influence on real-world applications, such as facial animation, head pose estimation and real-time facial 3D reconstruction. The instability results of landmark tracking cause face pose shaking and face movement that is not fluent enough. However, most of the existing landmark detection and tracking methods only consider the stability of face location but neglect the stability of local landmark movement. To solve the problem of landmark local shaking, we present a novel hierarchical filtering method for stabilized facial landmark detection and tracking in video frames. The proposed method addresses the challenging landmark local shaking problem and provides effective remedies to solve them. The main contribution within our solution is a novel hierarchical filtering strategy, which guarantees the robustness of global whole facial shape tracking and the adaptivity of local facial parts tracking. The proposed solution does not depend on specific face detection and alignment algorithms, and thus, can be easily deployed into existing systems. Extensive experimental evaluations and analyses on benchmark datasets and 3D head pose datasets verify the effectiveness of our proposed stabilizing method.

Robust face alignment by cascaded regression and de-occlusion

Face alignment is a typical facial behavior analysis task in computer vision. However, the performance of face alignment is degraded greatly when the face image is partially occluded. In order to achieve better mapping between facial appearance features and shape increments, we propose a robust and occlusion-free face alignment algorithm in which a face de-occlusion module and a deep regression module are integrated into a cascaded deep generative regression model. The face de-occlusion module is a disentangled representation learning Generative Adversarial Networks (GANs) which aims to locate occlusions and recover the genuine appearance from partially occluded face image. The deep regression module can enhance facial appearance representation by utilizing the recovered faces to obtain more accurate regressors. Then, by the cascaded deep generative regression model, we recover the partially occluded face image and achieve accurate locating of landmarks gradually. It is interesting to show that the cascaded deep generative regression model can effectively locate occlusions and recover more genuine faces, which can be further used to improve the performance of face alignment. Experimental results conducted on four challenging occluded face datasets demonstrate that our method outperforms state-of-the-art methods.

Face alignment using a 3D deeply-initialized ensemble of regression trees

Face alignment algorithms locate a set of landmark points in images of faces taken in unrestricted situations. State-of-the-art approaches typically fail or lose accuracy in the presence of occlusions, strong deformations, large pose variations and ambiguous configurations. In this paper we present 3DDE, a robust and efficient face alignment algorithm based on a coarse-to-fine cascade of ensembles of regression trees. It is initialized by robustly fitting a 3D face model to the probability maps produced by a convolutional neural network. With this initialization we address self-occlusions and large face rotations. Further, the regressor implicitly imposes a prior face shape on the solution, addressing occlusions and ambiguous face configurations. Its coarse-to-fine structure tackles the combinatorial explosion of parts deformation. In the experiments performed, 3DDE improves the state-of-the-art in 300W, COFW, AFLW and WFLW data sets. Finally, we perform cross-dataset experiments that reveal the existence of a significant data set bias in these benchmarks.

The Development and Challenges of Face Alignment Algorithms

A comprehensive survey of face alignment using different methods is presented in this paper. Face alignment is the fundamental task of facial applications, e.g., face recognition, 3D face modelling and face expression analysis, etc. State-of-the-art methods can be mainly categorized into the three groups: gradient descent-based, deep learning-based, and 3D model-based. In gradient descent-based methods, landmarks are localized and adjusted by solving a nonlinear regression function. Deep learning-based methods construct one or several cascaded neural networks to improve landmark localization accuracy. Beside the above two categories of methods solving problems on a 2D plane, there is also other category of methods like 3D model-based methods. Despite significant progress that has been made, face alignment faces challenges from real-world conditions: variation across poses, genders and ages, facial expressions, and facial attributes. This paper offers a brief illustration and analysis of several typical methods of face alignment, provides an overall understanding and insight into the field, which will motivate us to explore promising future directions.

Symmetric Face Normalization

Image registration is an important process in image processing which is used to improve the performance of computer vision related tasks. In this paper, a novel self-registration method, namely symmetric face normalization (SFN) algorithm, is proposed. There are three contributions in this paper. Firstly, a self-normalization algorithm for face images is proposed, which normalizes a face image to be reflection symmetric horizontally. It has the advantage that no face model needs to be built, which is always severely time-consuming. Moreover, it can be considered as a pre-processing procedure which greatly decreases the parameters needed to be adjusted. Secondly, an iterative algorithm is designed to solve the self-normalization algorithm. Finally, SFN is applied to the between-image alignment problem, which results in the symmetric face alignment (SFA) algorithm. Experiments performed on face databases show that the accuracy of SFN is higher than 0.95 when the translation on the x-axis is lower than 15 pixels, or the rotation angle is lower than 18°. Moreover, the proposed SFA outperforms the state-of-the-art between-image alignment algorithm in efficiency (about four times) without loss of accuracy.

Model compression of SDM‐based face alignment for mobile applications

Face alignment could be widely used in face recognition, expression recognition, face-based AR applications etc. Cascaded-regression-based face alignment algorithms have been popular in recent years for their low computational costs and impressive results in uncontrolled scenarios. Unfortunately, the size of the trained model is quite large for cascaded-regression-based methods which makes it unsuitable for commercial applications on mobile phones. In this study, the authors proposed a data compression method for the trained model of the supervised descent method (SDM). Firstly, the distribution of the model data was estimated using a non-parametric method. Then an adaptive quantisation algorithm was proposed to quantise the model data. Finally, their adaptive quantisation algorithm was tightly coupled with the SDM training process to fine tune the results. The quantitative experimental results proved that their proposed method could compress the data model to <20% of its original size without hurting the performances. The proposed method has been integrated into a mobile AR application, subjective evaluations proved that the proposed compression method could provide similar visual effects compared with the uncompressed counterpart.