Facial Feature Extraction Research Articles

A New Hybrid Technique for Face Identification Based on Facial Parts Moments Descriptors

Robust facial feature extraction is an effective and important process for face recognition and identification system. The facial features should be invariant to scaling, translation, illumination and rotation, several feature extraction techniques may be used to increase the recognition accuracy. This paper inspects three-moment invariants techniques and then determines how is influenced by the variation which may happen to the various shapes of the face (globally and locally) Globally means the whole face shapes and locally means face part's shape (right eye, left eye, mouth, and nose). The proposed technique is tested using CARL database images. The proposal method of the new method that collects the robust features of each method is trained by a feed-forward neural network. The result has been improved and achieved an accuracy of 99.29%.

Efficient face detection and tracking in video sequences based on deep learning

Video-based face detection and tracking technology has been widely used in video surveillance, safe driving, and medical diagnosis. In video sequences, most existing face detection and tracking methods face interference caused by occlusion, ambient illumination, and changes in human posture. To accurately track human faces in video sequences, we propose an efficient face detection and tracking framework based on deep learning, which includes a SENResNet face detection model and a Regression Network-based Face Tracking (RNFT) model. Firstly, the SENResNet model integrates the Squeeze and Excitation Network (SEN) with the Residual Neural Network (ResNet). To solve the problem that deep neural networks are difficult to train, we use ResNet to overcome the problem of gradient disappearance in deep network training. To fuse the features of each channel during the convolution operation, we further integrate the SEN module into the SENResNet model. SENResNet accurately detects facial information in each frame and extracts the position of the target face, thereby providing an initialization window for face tracking. Then, the RNFT model extracts facial features from adjacent frames and predict the position of the target face in the next frame. To address the problem of feature scaling, we add a correction network to the RNFT model. The improved RNFT model extracts the rectangular frame of the target face in the previous frame and strengthens the perception of feature scaling, thereby improving its accuracy. Extensive experimental results on public facial and video datasets show that the proposed SENResNet and RNFT models are superior to the state-of-the-art comparison methods in terms of accuracy and performance.

Leveraging recent advances in deep learning for audio-Visual emotion recognition

Emotional expressions are the behaviors that communicate our emotional state or attitude to others. They are expressed through verbal and non-verbal communication. Complex human behavior can be understood by studying physical features from multiple modalities; mainly facial, vocal and physical gestures. Recently, spontaneous multi-modal emotion recognition has been extensively studied for human behavior analysis. In this paper, we propose a new deep learning-based approach for audio-visual emotion recognition. Our approach leverages recent advances in deep learning like knowledge distillation and high-performing deep architectures. The deep feature representations of the audio and visual modalities are fused based on a model-level fusion strategy. A recurrent neural network is then used to capture the temporal dynamics. Our proposed approach substantially outperforms state-of-the-art approaches in predicting valence on the RECOLA dataset. Moreover, our proposed visual facial expression feature extraction network outperforms state-of-the-art results on the AffectNet and Google Facial Expression Comparison datasets.

Cost-effective real-time recognition for human emotion-age-gender using deep learning with normalized facial cropping preprocess

Because of technological advancement, human face recognition has been commonly applied in various fields. There are some HCI-related applications, such as camera-ready chatbot and companion robot, require gathering more information from user’s face. In this paper, we developed a system called EAGR for emotion, age, and gender recognition, which can perceive user’s emotion, age and gender based on the face detection. The EAGR system first applies normalized facial cropping (NFC) as a preprocessing method for training data before data augmentation, then uses convolution neural network (CNN) as three training models for recognizing seven emotions (six basics plus one neutral emotion), four age groups, and two genders. For better emotion recognition, the NFC will extract facial features without hair retained. On the other hand, the NFC will extract facial features with hair retained for better age and gender recognition. The experiments were conducted on these three training models of emotion, age and gender recognitions. The recognition performance results from the testing dataset, which has been normalized for tilted head by proposed binocular line angle correction (BLAC), showed that the optimal mean accuracy rates of real-time recognition for seven emotions, four age groups and two genders were 82.4%, 74.95%, and 96.65% respectively. Furthermore, the training time can be substantially reduced via NFC preprocessing. Therefore, we believe that EAGR system is cost-effective in recognizing human emotions, ages, and genders. The EAGR system can be further applied in social applications to help HCI service provide more accurate feedback from pluralistic facial classifications.

Detection of confusion behavior using a facial expression based on different classification algorithms

Confusion detection systems (CDSs) that need Noninvasive, mobile, and cost-effective methods use facial expressions as a technique to detect confusion. In previous works, the technology that the system used represents a major gap between this proposed CDS and other systems. This CDS depends on the Facial Action Coding System (FACS) that is used to extract facial features. The FACS shows the motion of the facial muscles represented by Action Units (AUs); the movement is represented with one facial muscle or more. Seven AUs are used as possible markers for detecting confusion that has been implemented in the form of a single vector of facial action; the AUs that have been used in this work are AUs 4, 5, 6, 7, 10, 12, and 23. The database used to calculate the performance of the proposed CDS is gathered from 120 participants (91males, 29 females), between the ages of 18-45. Four types of classification algorithms are used as individuals; these classifiers are (VG-RAM), (SVM), Logistic Regression and Quadratic Discriminant classifiers. The best success rate was found when using Logistic Regression and Quadratic Discriminant. This work introduces different classification techniques to detect confusion by collecting an actual database that can be used to evaluate the performance for every CDS employing facial expressions and selecting appropriate facial features.

Efficient video face recognition based on frame selection and quality assessment.

The article is considering the problem of increasing the performance and accuracy of video face identification. We examine the selection of the several best video frames using various techniques for assessing the quality of images. In contrast to traditional methods with estimation of image brightness/contrast, we propose to utilize the deep learning techniques that estimate the frame quality by using the lightweight convolutional neural network. In order to increase the effectiveness of the frame quality assessment step, we propose to distill knowledge of the cumbersome existing FaceQNet model for which there is no publicly available training dataset. The selected K-best frames are used to describe an input set of frames with a single average descriptor suitable for the nearest neighbor classifier. The proposed algorithm is compared with the traditional face feature extraction for each frame, as well as with the known clustering methods for a set of video frames.

Facial geometric feature extraction based emotional expression classification using machine learning algorithms.

Emotion plays a significant role in interpersonal communication and also improving social life. In recent years, facial emotion recognition is highly adopted in developing human-computer interfaces (HCI) and humanoid robots. In this work, a triangulation method for extracting a novel set of geometric features is proposed to classify six emotional expressions (sadness, anger, fear, surprise, disgust, and happiness) using computer-generated markers. The subject’s face is recognized by using Haar-like features. A mathematical model has been applied to positions of eight virtual markers in a defined location on the subject’s face in an automated way. Five triangles are formed by manipulating eight markers’ positions as an edge of each triangle. Later, these eight markers are uninterruptedly tracked by Lucas- Kanade optical flow algorithm while subjects’ articulating facial expressions. The movement of the markers during facial expression directly changes the property of each triangle. The area of the triangle (AoT), Inscribed circle circumference (ICC), and the Inscribed circle area of a triangle (ICAT) are extracted as features to classify the facial emotions. These features are used to distinguish six different facial emotions using various types of machine learning algorithms. The inscribed circle area of the triangle (ICAT) feature gives a maximum mean classification rate of 98.17% using a Random Forest (RF) classifier compared to other features and classifiers in distinguishing emotional expressions.

An efficient facial emotion recognition system using novel deep learning neural network-regression activation classifier

In the computer vision field, FER encompasses a significant place. It is being studied for a long period, and in recent decades, it has attained progress, but all is in vain, since, recognizing facial expression with high accuracy is still hard due to disparate facial expressions. To beat such difficulties, an efficient Facial Emotion Recognitions (FER) is proposed by utilizing a novel Deep Learning Neural Network-regression activation (DR) classifier. The proposed method has six phases, namely, pre-processing, facial point extraction, segmentation, feature extraction, feature selection, and classification. Initially the input image has been pre-processed using Gamma-HE technique and then facial points are extracted using Pyramid Histogram of Oriented Gradients (PHOG) based Supervised Descent (SMD) Method. The facial parts are segmented using Viola-Jones Algorithm (VJA) and then Local Tetra Pattern (LTrP), cluster shade, Inverse Divergent Moment (IDM), Local homogeneity, optimum probability, cluster prominence, dissimilarity, autocorrelation, and contrast features have been extracted. Modified Monarch Butterfly Optimization (MMBO) algorithm has been used to select necessary features from the extracted features. From the extracted facial points, the DR classifier classifies the emotions of the particular input image. A ‘2’ datasets were taken for analyzing the proposed system’s performance. Centred on the CK+ database, the proposed work attains 0.9885-accuracy, and centred on the JAFFE database, it has 0.9727-accuracy. Also, the investigational results proved that the proposed work trounces the existing systems centred on statistical metrics.

A new face detection method based on Faster RCNN

The Non-Maximum Suppression (NMS) method based on Faster RCNN uses hard threshold to identify candidate face frames which to be detected. In complex scenes with partial occlusion of the face and uneven lighting, the phenomenon of missing and false detection of the face is prone to occur. Aiming at this problem, this paper proposes a new face detection method to extract face features through CNN, and generates a large number of face candidate frames which to be detected by the Region Proposal Network (RPN); the hard threshold of NMS is improved by linear weighting method, and the face candidate frame is screened by the linearly weighted NMS. Comparison experiment results show that on the FDDB dataset, the new face detection method in this paper can effectively avoid the false detection and missed detection of multiple faces under partial occlusion and uneven illumination, also has high detection accuracy and robustness.

An improved age invariant face recognition using data augmentation

In spite of the significant advancement in face recognition expertise, accurately recognizing the face of the same individual across different ages still remains an open research question. Face aging causes intra-subject variations (such as geometric changes during childhood adolescence, wrinkles and saggy skin in old age) which negatively affects the accuracy of face recognition systems. Over the years, researchers have devised different techniques to improve the accuracy of age invariant face recognition (AIFR) systems. In this paper, the face and gesture recognition network (FG-NET) aging dataset was adopted to enable the benchmarking of experimental results. The FG-Net dataset was augmented by adding four different types of noises at the preprocessing phase in order to improve the trait aging face features extraction and the training model used at the classification stages, thus addressing the problem of few available training aging for face recognition dataset. The developed model was an adaptation of a pre-trained convolution neural network architecture (Inception-ResNet-v2) which is a very robust noise. The proposed model on testing achieved a 99.94% recognition accuracy, a mean square error of 0.0158 and a mean absolute error of 0.0637. The results obtained are significant improvements in comparison with related works.

HOG-ESRs Face Emotion Recognition Algorithm Based on HOG Feature and ESRs Method

As we all know, there are many ways to express emotions. Among them, facial emotion recognition, which is widely used in human–computer interaction, psychoanalysis of mental patients, multimedia retrieval, and other fields, is still a challenging task. At present, although convolutional neural network has achieved great success in face emotion recognition algorithms, it has a rising space in effective feature extraction and recognition accuracy. According to a large number of literature studies, histogram of oriented gradient (HOG) can effectively extract face features, and ensemble methods can effectively improve the accuracy and robustness of the algorithm. Therefore, this paper proposes a new algorithm, HOG-ESRs, which improves the traditional ensemble methods to the ensembles with shared representations (ESRs) method, effectively reducing the residual generalization error, and then combining HOG features with ESRs. The experimental results on the FER2013 dataset show that the new algorithm can not only effectively extract features and reduce the residual generalization error, but also improve the accuracy and robustness of the algorithm, the purpose of the study being achieved. The application of HOG-ESRs in facial emotion recognition is helpful to solve the symmetry of edge detection and the deficiency of related methods in an outdoor lighting environment.

Unconstrained Face Detection of Multiple Humans Present in the Video

To capture a stable picture through the digital camera is a challenging task in computer vision. The While identifying facial features such as misalignment, a parasitic light effect and a change in object position are the errors found in image processing. These errors get aggregated with images and cumulatively create distortion in the output video, which makes facial feature recognition more complicated in the video. In this paper, a solutions for unconstrained facial detection from digital image processing has been proposed, which fulfilled two requirements; first is a reliable method to extract the facial feature of the humans from a video and second is the estimation of 3D-image of human from the motion video. To meet these requirements, we develop a hybrid estimation method that combines the feature selection and extraction of facial features of the human from the video. Here we have extended the estimation of 2D to 3D unconstrained facial feature recognition. In the results, we found that the object in images is detected and we are able to develop the 3D sketch of human from the video. Further to validate the robustness of the proposed method, we have performed comprehensive testing on the huge dataset. The output of testing shows that the proposed method would be better to identify multiple facial features.

Face recognition system with feature normalization

ABSTRACT This study proposed an identity verification system that uses face recognition. The system features face detection as well as facial feature extraction and comparison methods. Early methods of face detection involved using specific approaches coupled with a classifier to extract features and detect faces. Although these methods can detect faces quickly, they generate a high false positive rate. Recent face detection methods based on a deep learning structure are extremely accurate but time-consuming. This study realized a face detection method based on the histogram of oriented gradient. The proposed method is not as accurate as deep learning; however, it is fast and can complete instant computing. Early methods of face recognition also involved using feature extraction methods coupled with a classifier to complete face recognition; however, these methods were not extremely accurate. The emergence of deep learning has facilitated greatly increasing the accuracy of face recognition. A deep learning-based method requires the entire deep learning structure to be retrained when a system needs to add a new user. This requirement is not feasible in actual applications. A researcher has therefore proposed the FaceNet method, which uses deep learning structure to extract eigenvectors and calculates the distance between eigenvectors as a measure of face similarity. Thus, the entire deep learning structure does not need to be retrained when a new user is added to the system. In this paper, FaceNet was used to extract the eigenvector of a face. However, the experiments of this study showed that facial features extracted using FaceNet are unevenly distributed in different dimensions, and using the calculated distance of the eigenvector as a measure of face similarity will yield inaccurate results. Therefore, this study proposed a facial feature normalization comparison method. The experimental results verified that the proposed method can achieve more than 98% accuracy and can be applied in practice.

A DenseUnet generative adversarial network for near-infrared face image colorization

A DenseUnet Generative Adversarial Network (GAN) is proposed to colorize near-infrared (NIR) face images. The GAN generator incorporates the advantages of both DenseNet and Unet structures. The DenseNet extracts facial features effectively by increasing the network depth, and the Unet keeps important facial details through skip-connection. The generator further integrates an optimized facial loss function designed by considering pixel loss, color loss and feature loss. The proposed network is evaluated against state-of-the-art grayscale image colorization GANs over two separate datasets. Both qualitative and quantitative comparisons demonstrate that with improved network structures and loss constrains the DenseUnet GAN can colorize NIR face images with natural color, minimal face shape distortion, and rich facial details.

Facial Expression Recognition With Machine Learning and Assessment of Distress in Patients With Cancer.

To estimate the effectiveness of combining facial expression recognition and machine learning for better detection of distress. 232 patients with cancer in Sichuan University West China Hospital in Chengdu, China. The Distress Thermometer (DT) and Hospital Anxiety and Depression Scale (HADS) were used as instruments. The HADS included scores for anxiety (HADS-A), depression (HADS-D), and total score (HADS-T). Distressed patients were defined by the DT cutoff score of 4, the HADS-A cutoff score of 8 or 9, the HADS-D cutoff score of 8 or 9, or the HADS-T cutoff score of 14 or 15. The authors applied histogram of oriented gradients to extract facial expression features from face images, and used a support vector machine as the classifier. The facial expression features showed feasible differentiation ability on cases classified by DT and HADS. Facial expression recognition could serve as a supplementary screening tool for improving the accuracy of distress assessment and guide strategies for treatment and nursing.

A Face Recognition System Using ACO-BPNN Model for Optimizing the Teaching Management System.

The basic idea of face recognition technology is to compare the matching degree between the standard face image marked with identity information and the static or dynamic face collected from the actual scene, which includes two main research contents: face feature extraction and face feature recognition. Traditional identification generally proves who we are through certificates, passwords, or certificates plus passwords. With the development of science and technology, face recognition technology will occupy an increasingly important position. Inspired by the human brain, the artificial neural network (ANN) is an information extraction system based on imitating the basic function and structure of the human brain and abstracted by physical and mathematical research methods. Based on the traditional BP neural network model, this paper proposes an ant colony algorithm-enabled BP neural network (ACO-BPNN) model and applies it to face recognition. Experimental results show that, similar to other face recognition techniques, the facial feature location needs to adapt to various changes of faces to the maximum extent, so the recognition and classification effect of the whole face feature extracted from the whole face image on the changes of such partial areas is not good, while the local feature extraction method based on ACO-BPNN can achieve a good recognition and classification effect.

An End-to-End Deep Model With Discriminative Facial Features for Facial Expression Recognition

Due to the complex challenges of the environment and emotion expressions, most facial expression recognition systems cannot achieve a high recognition rate. More discriminative features can describe facial expressions more accurately, so facial feature extraction is the key technology for facial expression recognition. In this article, an effective end-to-end deep model is proposed to improve the accuracy of face recognition. Considering the importance of data pre-processing (very few studies have focused on this process), first, a data enhancement method is proposed to locate the range of the face target and enhance the image contrast. Next, to obtain further discriminative features, a hybrid feature representation method is proposed, in which four typical feature extraction method are combined. After that, an effective deep model is designed to train and test the samples which can obtain the optimal parameters with less computation cost. Ablation study results show that the proposed hybrid feature representation method can help improve recognition accuracy. Finally, to comprehensively evaluate the performance of the proposed model, a series of experiments are conducted on three benchmark datasets. The recognition rate is achieved 94.5%, 98.6%, and 97.2% for FER2013, AR dataset, and CK+ dataset, respectively.

Feeling Recognition by Facial Expression Using Deep Learning

The visible appearance of the emotion state, personality, purpose, psychological feature activity and psychopathology of an individual is the Facial expression. This plays an outgoing role in social affairs. Automatic recognition of facial expressions will be a vital part of natural human-machine interfaces. It should even be applied in activity science and in clinical apply. Fellows in nurturing automatic facial features Recognition system must perform detection and placement of faces during a disordered scene, facial feature extraction and facial features classification. Emotion recognition by facial features utilizes a Deep Learning system, which is enforced victimization Convolution Neural Network (CNN). The CNN model of the project relies on LeNet design. Kaggle facial features dataset with seven facial features labels as fear, anger, happy, surprise, sad, neutral and disgust is employed during this project. The system achieved 60.37 accuracy.

Face recognition technology based on CNN, XGBoost, model fusion and its application for safety management in power system

The safety maintenance of operating site is important for safety management in power system. With the development of artificial intelligence (AI) and popularization of monitoring camera, face recognition technology is widely utilized in operating site of power system. To improve the capability for safety management of operating site in power system, a face recognition model based on convolutional neural network (CNN), eXtreme gradient boosting (XGBoost) and model fusion is built. Firstly, pre-processed images are input into CNN to obtain the recognition probabilities of various face and the extracted face features. Secondly, the extracted features of CNN are input into XGBoost to obtain the recognition probabilities of various face recognized by XGBoost. Finally, above two groups of probabilities are weighted by model fusion technology to obtain the final recognition probabilities of various face, and the final face recognition results are output. Simulation results show that CNN has better capability of feature extraction, and the proposed face recognition model has the highest recognition accuracy among those advanced face recognition models. In addition, this paper describes the application based on the proposed face recognition model in non-working personnel recognition, trajectory tracking of operators, etc, so as to greatly improve the safety management of operating site in power system, which not only ensures safety, but also reduces unnecessary management expenses.

Efficient Face Re-Identification through PSO Based Adaptive Deep Learning Models

Face plays a vital role in the Recognition or Re-Identification of a person. Therefore, it is significant to identify and extract the facial visual features that automatically lead to face identification-based classification. Facial features comprise different ways of detection, for instance, they could be located at corners or midpoints of the facial features that rely on multiple components such as eyes, lips, nose with different emotions and expressions used in face recognition. This paper introduced a robust and efficient deep learning model with the use of a transfer learning approach for PSO for extraction and selection of the best facial features. Deep learning models “Openface via PSO and introduced customized Inception-V3 model via PSO is used and present detailed comparative accuracy of both models in terms of classification recognition. For this, the paper presents seven different algorithms to evaluate the efficiency of the model with four different face databases. It is evident from the result; neural network classifier shows a gradual hike to calculate accuracy with the proposed PSO-based OpenFace deep learning approach. On the other hand, random forest and AdaBoost algorithm were observed most compatible with the customized PSO-based Inception-V3 model.