Analysis Of Human Facial Expressions Research Articles

Feature extraction comparison for facial expression recognition using adaptive extreme learning machine

Facial expression recognition is an important part in the field of affective computing. Automatic analysis of human facial expression is a challenging problem with many applications. Most of the existing automated systems for facial expression analysis attempt to recognize a few prototypes emotional expressions such as anger, contempt, disgust, fear, happiness, neutral, sadness, and surprise. This paper aims to compare feature extraction methods that are used to detect human facial expression. The study compares the gray level co-occurrence matrix, local binary pattern, and facial landmark (FL) with two types of facial expression datasets, namely Japanese female facial expression (JFFE), and extended Cohn-Kanade (CK+). In addition, we also propose an enhancement of extreme learning machine (ELM) method that can adaptively select best number of hidden neurons adaptive ELM (aELM) to reach its maximum performance. The result from this paper is our proposed method can slightly improve the performance of basic ELM method using some feature extractions mentioned before. Our proposed method can obtain maximum mean accuracy score of 88.07% on CK+ dataset, and 83.12% on JFFE dataset with FL feature extraction.

A deep learning model for classifying human facial expressions from infrared thermal images

The analysis of human facial expressions from the thermal images captured by the Infrared Thermal Imaging (IRTI) cameras has recently gained importance compared to images captured by the standard cameras using light having a wavelength in the visible spectrum. It is because infrared cameras work well in low-light conditions and also infrared spectrum captures thermal distribution that is very useful for building systems like Robot interaction systems, quantifying the cognitive responses from facial expressions, disease control, etc. In this paper, a deep learning model called IRFacExNet (InfraRed Facial Expression Network) has been proposed for facial expression recognition (FER) from infrared images. It utilizes two building blocks namely Residual unit and Transformation unit which extract dominant features from the input images specific to the expressions. The extracted features help to detect the emotion of the subjects in consideration accurately. The Snapshot ensemble technique is adopted with a Cosine annealing learning rate scheduler to improve the overall performance. The performance of the proposed model has been evaluated on a publicly available dataset, namely IRDatabase developed by RWTH Aachen University. The facial expressions present in the dataset are Fear, Anger, Contempt, Disgust, Happy, Neutral, Sad, and Surprise. The proposed model produces 88.43% recognition accuracy, better than some state-of-the-art methods considered here for comparison. Our model provides a robust framework for the detection of accurate expression in the absence of visible light.

Implementasi Convolutional Neural Network untuk Klasifikasi Variasi Intensitas Emosi pada Dynamic Image Sequence

Facial emotion recognition (FER) is a research topic that focuses on the analysis of human facial expressions. There are many FER research has been conducted on single images or photo. Emotion analysis on single images has many disadvantages compared to dynamic image sequences or videos. This is due to human emotions or expressions within a certain time. The classification of emotions becomes complicated when considering different emotions. There are some people who are very expressive, there are some people who have low or moderate expressions. Predictions of emotion with variety intensities has decresed error due to data sets that provide only a few emotions intensities. Data annotation is a major problem in recognition fields that require a lot of time and effort to annotate new data. This study aims to find information about facial emotions with emotional intensity from subtle to sharp in a sequence images or videos. The dataset will be trained using Convolutional neural network by augmentation to add data annotations. The proposed method was evaluated using the public BP4D-Spontaneous dataset. The evaluation results show that the average emotion recognition in video sequences using the holdout method is 18%. Evaluation of the loss function parameter shows overfitting where the curve generalization gap is too high. The last evaluation is the evaluation of the emotion class between the real class and the prediction class in 14.28%. This shows that the classification of emotion recognition in dynamic image sequences is quite low.

Human facial expression analysis based on image granule LPP

This paper proposes a synthetic technique for human facial expression analysis based on locality preserving projections (LPP) and granular computing. The proposed method decreases the computational complexity of LPP and preserves the performance of LPP algorithm. In image processing, an image can be divided into various sizes of blocks, which are defined as image granules in this paper. The LPP algorithm is implemented to images with various image granules. By this strategy, the image dimension reduces quickly, which decreases the computational complexity. The experiments on three face databases are presented to show the performance of image granule LPP. Distribution of the facial pose and expression is shown and the computational complexities with different image granules are compared. Meanwhile, the order-preserving property of images is investigated by tracking the sequence of designated images. And the loss of image information is analyzed by image roughness, entropy and histogram. Furthermore, the parameter setting in LPP is discussed because of its non-ignorable affect on the experiment results. Finally, the method is applied to facial expression recognition.

Automatic landmark point detection and tracking for human facial expressions

Facial landmarks are a set of salient points, usually located on the corners, tips or mid points of the facial components. Reliable facial landmarks and their associated detection and tracking algorithms can be widely used for representing the important visual features for face registration and expression recognition. In this paper we propose an efficient and robust method for facial landmark detection and tracking from video sequences. We select 26 landmark points on the facial region to facilitate the analysis of human facial expressions. They are detected in the first input frame by the scale invariant feature based detectors. Multiple Differential Evolution-Markov Chain (DE-MC) particle filters are applied for tracking these points through the video sequences. A kernel correlation analysis approach is proposed to find the detection likelihood by maximizing a similarity criterion between the target points and the candidate points. The detection likelihood is then integrated into the tracker’s observation likelihood. Sampling efficiency is improved and minimal amount of computation is achieved by using the intermediate results obtained in particle allocations. Three public databases are used for experiments and the results demonstrate the effectiveness of our method.

A Neuro Fuzzy Approach for Facial Expression Recognition Using LBP Histograms

Facial expression analysis plays a significant role for human computer interaction. Automatic analysis of human facial expression is still a challenging problem with many applications. In this paper, we propose neuro-fuzzy based automatic facial expression recognition system to recognize the human facial expressions like happy, fear, sad, angry, disgust and surprise. Initially facial image is segmented into three regions from which the uniform Local Binary Pattern (LBP) texture features distributions are extracted and represented as a histogram descriptor. The facial expressions are recognized using Multiple Adaptive Neuro Fuzzy Inference System (MANFIS). The proposed system designed and tested with JAFFE face database. The proposed model reports 95.29% of classification accuracy.

Human Facial Expression Recognition using MANFIS Model

Facial expression analysis plays a significant role for human computer interaction. Automatic analysis of human facial expression is still a challenging problem with many applications. In this paper, we propose neuro-fuzzy based automatic facial expression recognition system to recognize the human facial expressions like happy, fear, sad, angry, disgust and surprise. Initially facial image is segmented into three regions from which the uniform Local Binary Pattern (LBP) texture features distributions are extracted and represented as a histogram descriptor. The facial expressions are recognized using Multiple Adaptive Neuro Fuzzy Inference System (MANFIS). The proposed system designed and tested with JAFFE face database. The proposed model reports 94.29% of classification accuracy.

Dynamics of facial expression: recognition of facial actions and their temporal segments from face profile image sequences

Automatic analysis of human facial expression is a challenging problem with many applications. Most of the existing automated systems for facial expression analysis attempt to recognize a few prototypic emotional expressions, such as anger and happiness. Instead of representing another approach to machine analysis of prototypic facial expressions of emotion, the method presented in this paper attempts to handle a large range of human facial behavior by recognizing facial muscle actions that produce expressions. Virtually all of the existing vision systems for facial muscle action detection deal only with frontal-view face images and cannot handle temporal dynamics of facial actions. In this paper, we present a system for automatic recognition of facial action units (AUs) and their temporal models from long, profile-view face image sequences. We exploit particle filtering to track 15 facial points in an input face-profile sequence, and we introduce facial-action-dynamics recognition from continuous video input using temporal rules. The algorithm performs both automatic segmentation of an input video into facial expressions pictured and recognition of temporal segments (i.e., onset, apex, offset) of 27 AUs occurring alone or in a combination in the input face-profile video. A recognition rate of 87% is achieved.