Facial Feature Extraction Research Articles

Hybrid facial expression recognition (FER2013) modelfor real-time emotion classification and prediction

Facial expression recognition is a vital research topic in most fields ranging from artificial intelligence and gamingto human-computer interaction (HCI) and psychology. This paper proposes a hybrid model for facial expressionrecognition, which comprises a deep convolutional neural network (DCNN) and a Haar Cascade deep learningarchitecture. The objective is to classify real-time and digital facial images into one of the seven facial emotioncategories considered. The DCNN employed in this research has more convolutional layers, ReLU activationfunctions, and multiple kernels to enhance filtering depth and facial feature extraction. In addition, a HaarCascade model was also mutually used to detect facial features in real-time images and video frames. Grayscaleimages from the Kaggle repository (FER2013) and then exploited graphics processing unit (GPU) computation toexpedite the training and validation process. Pre-processing and data augmentation techniques are applied toimprove training efficiency and classification performance. The experimental results show a significantly improvedclassification performance compared to state-of-the-art (SoTA) experiments and research. Also, compared to otherconventional models, this paper validates that the proposed architecture is superior in classification performancewith an improvement of up to 6%, totaling up to 70% accuracy, and with less execution time of 2,098.8 s

End-to-end Saliency Face Detection and Recognition

Face recognition is a long-lasting hot topic in compute vision. The face recognition system mainly includes face detection, alignment and feature extraction. In the forward task, the extracted features are used to measure the similarity between faces, and outputs whether those are same person or not or which person it is in the registered set. Typically, the three stages of recognition system training independently of each other have the following shortcomings: 1) redundant calculation of feature maps; 2) unable to end-to-end optimization; 3) detecting an extracting so much useless face. A lightweight model for saliency face detection and recognition that can be optimized end-to-end is proposed. While maintaining accuracy, it meets the real-time and memory limitation requirements in embedded devices or terminals.

Face recognition system based on the multi-resolution singular value decomposition fusion technique

This study proposes a Fusion, Feature-Level, Face Recognition System (FFLFRS) that is based on the Multi-Resolution, Singular Value Decomposition (MSVD) fusion technique. Face recognition in the FFLFRS is achieved via four processes: face detection, feature extraction, feature fusion, and face classification. In this system, the most significant face features (that is, the eyes, nose, and mouth) are first detected. Then, local and global features are extracted by the Local Binary Pattern (LBP) and Principal Component Analysis (PCA) extraction approaches. Afterwards, the extracted features are fused by the MSVD method and classified by the Artificial Neural Network (ANN). The proposed FFLFRS was verified on 10,000 face images drawn from the face images database of the Olivetti Research Laboratory (ORL). Face recognition performance of this system was contrasted with levels of performance of three state of the art, fusion-level, face recognition systems (FRSs) depending on the Frequency Partition (FP), Laplacian Pyramid (LP), and Covariance Intersection (CI) fusion methods. Ten-thousand images were employed to test the proposed model and assess its performance, which was evaluated in terms of changes in pose, illumination, and expression, besides low resolution and presence of occlusion. The face recognition results of the proposed FFLFRS are encouraging. This system proved to be effective in dealing with images having challenges to face recognition and it could achieve a recognition accuracy as high as 97.78%.

Semantic Human Face Analysis for Multi-level Age Estimation

Human face is one of the most widely used biometrics based on computer-vision to derive various useful information such as gender, ethnicity, age, and even identity. Facial age estimation has received great attention during the last decades because of its influence in many applications, like face recognition and verification, which may be affected by aging changes and signs which appear on human face along with age progression. Thus, it becomes a prominent challenge for many researchers. One of the most influential factors on age estimation is the type of features used in the model training process. Computer-vision is characterized by its superior ability to extract traditional facial features such as shape, size, texture, and deep features. However, it is still difficult for computers to extract and deal with semantic features inferred by human-vision. Therefore, we need somehow to bridge the semantic gap between machines and humans to enable utilization of the human brain capabilities of perceiving and processing visual information in semantic space. Our research aims to exploit human-vision in semantic facial feature extraction and fusion with traditional computer-vision features to obtain integrated and more informative features as an initial study paving the way to further augment the outperforming state-of-the-art age estimation models. A hierarchical automatic age estimation is achieved upon two consecutive stages: classification to predict (high-level) age group, followed by regression to estimate (low-level) exact age. The results showed noticeable performance improvements, when fusing semantic-based features with traditional vision-based features, surpassing the performance of traditional features alone.

An Instance and Feature-Based Hybrid Transfer Model for Fault Diagnosis of Rotating Machinery With Different Speeds

Recently, transfer learning approaches using pre-trained models to learn new tasks enable many real-world applications. However, when detecting rotating machinery faults under different conditions like rotating speeds, classical instance-based transfer (IBT) algorithms represented by generative adversarial networks (GANs) require the source and target domains to have similar feature spaces. Meanwhile, the other feature-based transfer (FBT) algorithms often face feature extraction difficulty and domain over-adaptation. To overcome these problems, a novel hybrid transfer model that combines both ideas of IBT and FBT algorithms is proposed in this paper. This hybrid model firstly introduces a cycle-consistent generative adversarial network (CycleGAN) to generate labeled pseudo samples. By adding an operating-condition discriminator and a fault-consistency discriminator to the CycleGAN, generated samples are easier to transfer operating conditions and retain the domain-invariant fault information. Secondly, generated and target samples are sent to a deep domain confusion (DDC) based network to adapt feature domain spaces and further predict the labels of target samples. Compared with the traditional IBT or FBT model, the proposed hybrid transfer model can combine both advantages of these two competitive models, broadening the transferable domain and improving the task accuracy. In addition, a pseudo-label based transition strategy for labeling intermediate domain samples is also proposed to help the transfer model perform better when unlabeled intermediate domain samples are available. Lab experiments verify that the proposed hybrid model and transition strategy can significantly improve the fault diagnosis accuracy of rotating machinery with different speeds.

Dynamic Feature Subset Selection for Occluded Face Recognition

Accurate recognition of person identity is a critical task in civil society for various application and different needs. There are different well-established biometric modalities that can be used for recognition purposes such as face, voice, fingerprint, iris, etc. Recently, face images have been widely used for person recognition, since the human face is the most natural and user-friendly recognition method. However, in real-life applications, some factors may degrade the recognition performance, such as partial face occlusion, poses, illumination conditions, facial expressions, etc. In this paper, we propose two dynamic feature subset selection (DFSS) methods to achieve better recognition for occluded faces. The proposed DFSS methods are based on resilient algorithms attempting to minimize the negative influence of confusing and low-quality features extracted from occluded areas by either exclusion or weight reduction. Principal Component Analysis and Gabor filtering based approaches are initially used for face feature extraction, then the proposed DFSS methods are dynamically enforced. This is leading to more effective feature representation and an improved recognition performance. To validate their effectiveness, multiple experiments are conducted and the performance of different methods is compared. The experimental work is carried out using AR database and Extended Yale Face Database B. The obtained results of face identification and verification show that both proposed DFSS methods outperform the standard (static) use of the whole number of features and the equal feature weights.

Eye of Devil: Face Recognition in Real World Surveillance Video with Feature Extraction and Pattern Matching

Abstract: People lost, people missing etc., these are the words we come across whenever there is any mass gathering events going on or in crowded areas. To solve this issue some traditional approaches like announcements are in use. One idea is to identify the person using face recognition and pattern matching techniques. There are several techniques to implement face recognition like extraction of facial features by using the position of eyes, nose, jawbone or skin texture analysis etc., By using these techniques a unique dataset can be created for each human. Here the photograph of the missing person can be used to extract these facial features. After getting the dataset of that individual, by using pattern matching techniques, there is a scope to find the person with same facial features in the crowd images or videos. Keywords: Face-Recognition, Image-Processing, Feature extraction, Video-Processing, Pattern-Matching.

Concealed Weapon Detection using Image Processing and Machine Learning

Abstract: This research paper presents a study of concealed weapon detection using image processing and machine learning. In order to attempt to replace the traditional method of detecting hidden weapons i.e. x-ray method with an automated and possibly a less error prone procedure, potential alternate techniques such as neural networks and image fusion have been studied and explored to identify the best possible solution. We propose a method to fuse Thermal/IR image with the conventional RGB image or HSV image in order to reduce image noise and retain all the critical features of the image to achieve both weapon detection and facial feature extraction. Keywords: Image fusion; concealed weapon; feature extraction; neural network; thermal imaging

Anti-jamming heart rate estimation using a spatial–temporal fusion network

Remote heart rate estimation based on face video has attracted increasing research attention. The previous methods are mostly based on ideal face features, where face videos are obtained under controllable experimental conditions, with uniform lighting and little noise interference. However, there is a big deviation from the real world scene. The lighting changes, head movements, and object occlusion are inevitable in complex scenes, which will lead to the failure of face positioning and feature extraction from video, and the instability of the facial signal required for heart rate estimation. In this paper, an anti-jamming network is proposed to improve the robustness of handling less-constrained scenarios. Specifically, a new spatial–temporal map generation mechanism is designed to enhance the spatial and temporal features representation by equivalent padding for low-quality video frame fragments. Meanwhile, a heart rate estimation stability module is built to evaluate the quality of the face signals and assign reasonable weights to video clips. Our approach significantly outperforms all current state-of-the-art methods on VIPL-HR dataset.

Feature Level Fusion Algorithm for Iris and Face

Some Bi-modal or multimodal recognition systems do not contain rich information needed for identification because information supplied to the biometric classifier are consolidated oncethe conclusions of the matching algorithm have been acquired. Feature based Fusion algorithm has the distinction of having richer information due to the integration of the extracted information before the application of the classifiers. Support Vector Machine over time has shown its unbeatable classification of the biometrics characteristics over other supervised learning classifiers due to its ability to minimize the structural risk simultaneously with bound on the margin complexity and by being solved using a quadratic optimization problem. Neural Network in contrast is a non-parametric estimator which is robust to errors in the training data used for classification and regression. Therefore in this research, algorithms for feature extraction of iris and face for recognition is designed; a recognition system using SVM and Multilayer Perceptron (MLP) is also designed based on the extracted features and the designed model is implemented using MATLAB

Gabor Log-Euclidean Gaussian and its fusion with deep network based on self-attention for face recognition

In this work, we proposed a face feature extraction method by Learning Gabor Log-Euclidean Gaussian with Whitening Principal Component Analysis (called LGLG-WPCA). The proposed method extracts raw features from the multivariate Gaussian in the transform domain of Gabor wavelet and uses WPCA to get robust features. Because the space of Gaussian is a Riemannian manifold, it is difficult to incorporate the learning mechanism into the model. To address this issue, Log-Euclidean approach is used to embed the multivariate Gaussian into the linear space, and then use WPCA to learn discriminative face features. LGLG-WPCA is good at extracting the detail features of face image. Furthermore, another outstanding advantage of LGLG is that its features can be effectively integrated with the high-level features of deep learning network for face recognition in more complex environments. We presented the feature fusing approaches for face recognition based on Self-attention Network (SAN) and achieved obvious performance improvement to the-state-of-the-art deep networks including SENet and FaceNet. Experiments show the proposed method is robust under adverse conditions such as varying poses, skin aging and uneven illumination, and it is suitable for face image under small-scale datasets in complex environments, such as network-based or video-based person searching or tracking.

High-Capacity Real-Time Face Retrieval Recognition Algorithm Based on Task Scheduling Model for the Treatment Area of Hospital

This paper presents an in-depth study of face detection, face feature extraction, and face classification from three important components of a high-capacity face recognition system for the treatment area of hospital and a study of a high-capacity real-time face retrieval and recognition algorithm for the treatment area of hospital based on a task scheduling model. Considering the real-time nature of our system, our face feature extraction network is modeled by DeepID, and the network is slightly improved by introducing a central loss verification signal to train a DeepID-like network model using central loss and use it to extract face features. To further investigate and optimize the schedulability analysis problem of the directed graph real-time task model, this paper proposes a rigorous and approximate response time analysis method for the directed graph real-time task model with an arbitrary time frame. Based on the theoretical results of the greatly additive algebra, it is shown that the coherent qualifying function is linearly periodic, i.e., the function can be represented by a finite nonperiodic part and an infinitely repeated periodic part, thus calculating the coherent qualifying function independent of the magnitude of the interval time. The algorithm for high-capacity real-time face retrieval and recognition in the treatment area of hospital based on the task scheduling model is further investigated, and a face database is established by using the PCA dimensionality reduction technique. Based on the internal architecture of the processor, image preprocessing and IP core packaging are implemented, and the hardware engineering of the high-capacity real-time face recognition system for hospital visits is built using the IP-based design concept. The performance tests of the face detection model and feature extraction network show that the face detection model has a significant reduction in false-positive rate, better fitting of border regression, and improved time performance. The face feature extraction network has no overfitting, and the features are highly discriminative with small feature extraction time consumption. The high-capacity real-time face recognition system for the treatment area of hospital combined with the optimized directed graph task scheduling model can approach 25 fps, which meets the real-time requirements, and the face recognition rate surpasses that of real people. It realizes the intelligence, self-help, and autonomy of medical services and satisfies the medical needs of users in all aspects.

Enhanced global and local face feature extraction for effective recognition of facial emotions

AbstractEmotion recognition is a challenging task in the field of human computer interaction. For a successful human emotion recognition system, a robust, discriminative, and sensitive feature extraction is an essential need. In this article, the extraction of global features is done by the proposed frequency decoded lifting wavelet pattern descriptor (FDLWP) and extraction of local features is done by the proposed local gradient difference zig‐zag pattern descriptor (LGDZP). The face parts are detected using viola jones algorithm and the selection of optimal face active regions is accomplished by the calculation of structural similarity index measure. Eventually the local spatial zig‐zag structure of the face region is utilized to attain LGDZP descriptor. The fusion of local and global features is accomplished using canonical correlation analysis. The classification made using bank of restricted Boltzmann machine (RBM) classifiers yields promising results for the proposed method. Furthermore, the proposed recognition method delivers a promising accuracy in varying illumination, occlusion, and noise. The accuracy of the method is analyzed by doing experiments with the databases such as JAFFE, CK+, MMI, Oulu‐CASIA, and SFEW. The obtained results of the proposed method yield better accuracy than the existing state of art methods in this field.

Typographic Portrait Generator Using Image Processing-Project Application

Abstract: Typography portrait is a portrait made up of words. The face of a person has different shades and distinct features. In order to make a typography portrait the distinct shades and features need to be clearly extracted. Existing systems focus only on the different shades of the face. They fill the face with word clouds without any regard for the distinct features of a face. This project will consider not only the shades of a face but also the features of a person’s face. The extraction of facial features will be done first and then text filling will be done accordingly. This project intends to create an application that will create a typography portrait having a much better visual appeal than the present systems. Keywords: Image Processing, Edge Detection, Image Segmentation, Face Recognition, Text Filling.

Quantum-inspired meta-heuristic algorithms with deep learning for facial expression recognition under varying yaw angles

In recent years, the increasing human–computer interaction has spurred the interest of researchers towards facial expression recognition to determine the expressive changes in human beings. The detection of relevant features that describe the expressions of different individuals is vital to describe human expressions accurately. The present work has employed the integrated concept of Local Binary Pattern and Histogram of Gradient for facial feature extraction. The major contribution of the paper is the optimization of the extracted features using quantum-inspired meta-heuristic algorithms of QGA (Quantum-Inspired Genetic Algorithm), QGSA (Quantum-Inspired Gravitational Search Algorithm), QPSO (Quantum-Inspired Particle Swarm Optimization), and QFA (Quantum-Inspired Firefly Algorithm). These quantum-inspired meta-heuristic algorithms utilize the attributes of quantum computing that ensure the adequate control of facial feature diversity with quantum measures and Q-bit superstition states. The optimized features are fed to the deep learning (DL) variant deep convolutional neural network added with residual blocks (DCNN-R) for the classification of expressions. The facial expressions are detected for the KDEF and RaFD datasets under varying yaw angles of –90∘, –45∘, 0∘, 45∘, and 90∘. The detection of facial expressions with varying angles is also a crucial contribution, as the features decrease with the increasing yaw angle movement of the face. The experimental evaluations demonstrate the superior performance of the QFA than other algorithms for feature optimization and hence the better classification of facial expressions.

Learning directly from synthetic point clouds for “in-the-wild” 3D face recognition

Point clouds-based networks have achieved great attention in 3D object classification, segmentation, and indoor scene semantic parsing, but its application to 3D face recognition is still underdeveloped owing to two main reasons: lack of large-scale 3D facial data and absence of deep neural network that can directly extract discriminative face representations from point clouds. To address these two problems, a PointNet++ based network is proposed in this paper to extract face features directly from point clouds facial scans and a statistical 3D Morphable Model based 3D face synthesizing strategy is established to generate large-scale unreal facial scans to train the proposed network from scratch. A curvature-aware point sampling technique is proposed to hierarchically down-sample feature-sensitive points which are crucial to pass and aggregate discriminative facial features deeply. In addition, a novel 3D face transfer learning method is proposed to ease the domain discrepancy between synthetic and ‘in-the-wild’ faces. Experimental results on two public 3D face benchmarks show that the network trained only on synthesized data can also be well generalized to ‘in-the-wild’ 3D face recognition. Our method achieves the state-of-the-art results by achieving an overall rank-1 identification rate of 99.46% and 99.65% on FRGCv2 and Bosphorus, respectively. Further, we evaluate on a self-collected dataset to demonstrate the robustness and application potential of our method.

Automatic classification of ASD children using appearance-based features from videos

Early diagnosis of Autism Spectrum Disorder (ASD) plays a crucial role in the intervention of ASD. Traditionally, an ASD child needs to be diagnosed by a psychiatrist in the hospital, which is expensive, time-consuming, and influenced by expertise. In this paper, we present an objective, convenient, and effective method for classifying ASD children from raw video sequences by integrating the appearance-based features from facial expressions, head pose, and head trajectory. To better extract facial expression features, we propose a novel attention-based facial expression recognition algorithm to focus on key face areas like eyebrows, mouth, etc. Moreover, we use accumulative histogram to individually extract temporal and spatial information from facial expression, head pose and head trajectory of the video sequence. After fusing these three kinds of features, we feed them to Long Short-Term Memory (LSTM) and achieve a classification accuracy of 96.7% on our self-collected ASD video dataset.

3D Face Recognition Based on an Attention Mechanism and Sparse Loss Function

Face recognition is one of the essential applications in computer vision, while current face recognition technology is mainly based on 2D images without depth information, which are easily affected by illumination and facial expressions. This paper presents a fast face recognition algorithm combining 3D point cloud face data with deep learning, focusing on key part of face for recognition with an attention mechanism, and reducing the coding space by the sparse loss function. First, an attention mechanism-based convolutional neural network was constructed to extract facial features to avoid expressions and illumination interference. Second, a Siamese network was trained with a sparse loss function to minimize the face coding space and enhance the separability of the face features. With the FRGC face dataset, the experimental results show that the proposed method could achieve the recognition accuracy of 95.33%.

Automatic Face Emotion Recognition With the Aid of Probability-Based Bird Swarm-Trained Neural Network

This paper intends to develop a novel FER model, which consists of four stages: (1) face detection, (2) feature extraction, (3) dimension reduction, and (4) classification. In this context, the face detection is done using Viola Jones method (VJ). It is the first object recognition model to offer better recognition rates in real-time. Further, features extraction techniques like local binary pattern (LBP) and discrete wavelet transform (DWT) are used for extracting the features from face detected images. Moreover, the dimension reduction of features is done using principal component analysis (PCA), which is an arithmetical process that exploits an orthogonal transformation to exchange a group of annotations of probably interrelated constraints. The classification procedure is performed using neural network (NN), with the new training algorithm called bird swarm algorithm, which is modified based on probability and hence termed as probability-based BSA (P-BSA).

Facial Feature Extraction for Emotion Classification Using Fuzzy C-Mean Clustering

Background: Automatic human emotion recognition system is an active area of research due to its wide applications in the field of Human Computer Interaction(HCI) systems, driver fatigue monitoring systems, surveillance systems, human assistance systems, smile detectors etc. Objective: The study presents a fuzzy based approach to extract facial features from input image and builds different classification models to classify the image into two emotion classes i.e. happy and neutral. The system has potential implications in the field of smile detection systems, customer experience analysis and patient monitoring systems. Methods: The proposed system determines the dimensional attributes (l-attribute and w-attribute) of mouth region extracted from the facial image using viola-jones algortithm. The feature set is generated by using a total of 136 images from JAFFE, NimStim and MUG dataset. The differentiating power of the attribures is then evaluated using five different classification models. Results: The accuracy, precision and recall is determined for each classification model. The results show good accuracy of 70% for the grayscale JAFFE and NimStim databases and 95% for the coloured MUG database. Conclusion: The mouth features calculated in the study are based on the geometric coordinates which eliminates the possibility of false distance measurements due to presence of noise or shadows.