Japanese Female Facial Expression Dataset Research Articles

Feature extraction comparison for facial expression recognition using adaptive extreme learning machine

Facial expression recognition is an important part in the field of affective computing. Automatic analysis of human facial expression is a challenging problem with many applications. Most of the existing automated systems for facial expression analysis attempt to recognize a few prototypes emotional expressions such as anger, contempt, disgust, fear, happiness, neutral, sadness, and surprise. This paper aims to compare feature extraction methods that are used to detect human facial expression. The study compares the gray level co-occurrence matrix, local binary pattern, and facial landmark (FL) with two types of facial expression datasets, namely Japanese female facial expression (JFFE), and extended Cohn-Kanade (CK+). In addition, we also propose an enhancement of extreme learning machine (ELM) method that can adaptively select best number of hidden neurons adaptive ELM (aELM) to reach its maximum performance. The result from this paper is our proposed method can slightly improve the performance of basic ELM method using some feature extractions mentioned before. Our proposed method can obtain maximum mean accuracy score of 88.07% on CK+ dataset, and 83.12% on JFFE dataset with FL feature extraction.

Selective local binary pattern with convolutional neural network for facial expression recognition

<span lang="EN-US">Variation in images in terms of head pose and illumination is a challenge in facial expression recognition. This research presents a hybrid approach that combines the conventional and deep learning, to improve facial expression recognition performance and aims to solve the challenge. We propose a selective local binary pattern (SLBP) method to obtain a more stable image representation fed to the learning process in convolutional neural network (CNN). In the preprocessing stage, we use adaptive gamma transformation to reduce illumination variability. The proposed SLBP selects the discriminant features in facial images with head pose variation using the median-based standard deviation of local binary pattern images. We experimented on the Karolinska directed emotional faces (KDEF) dataset containing thousands of images with variations in head pose and illumination and Japanese female facial expression (JAFFE) dataset containing seven facial expressions of Japanese females’ frontal faces. The experiments show that the proposed method is superior compared to the other related approaches with an accuracy of 92.21% on KDEF dataset and 94.28% on JAFFE dataset.</span>

Facial emotion recognition using hybrid features-novel leaky rectified triangle linear unit activation function based deep convolutional neural network

Facial Expression Recognition (FER) is an important topic that is used in many areas. FER categorizes facial expressions according to human emotions. Most networks are designed for facial emotion recognition but still have some problems, such as performance degradation and the lowest classification accuracy. To achieve greater classification accuracy, this paper proposes a new Leaky Rectified Triangle Linear Unit (LRTLU) activation function based on the Deep Convolutional Neural Network (DCNN). The input images are pre-processed using the new Adaptive Bilateral Filter Contourlet Transform (ABFCT) filtering algorithm. The face is then detected in the filtered image using the Chehra face detector. From the detected face image, facial landmarks are extracted using a cascading regression tree, and important features are extracted based on the detected landmarks. The extracted feature set is then passed as input to the Leaky Rectified Triangle Linear Unit Activation Function Based Deep Convolutional Neural Network (LRTLU-DCNN), which classifies the input image expressions into six emotions, such as happiness, sadness, neutrality, anger, disgust, and surprise. Experimentation of the proposed method is carried out using the Extended Cohn-Kanade (CK+) and Japanese Female Facial Expression (JAFFE) datasets. The proposed work provides a classification accuracy of 99.67347% for the CK+ dataset along with 99.65986% for the JAFFE dataset.

Deep neural network for coded mask cryptographical imaging.

We proposed a novel cryptographic imaging scheme that is the combination of optical encryption and computational decryption. To prevent personal privacy from being spied upon amid the imaging formation process, in this study we applied a coded mask to optically encrypt the scene and utilized the deep neural network for computational decryption. For encryption, the sensor recorded a new representation of the original signal, not being distinguishable by humans on purpose. For decryption, we successfully reconstructed the image with the mean squared error equal to 0.028, and 100% for the classification through the Japanese Female Facial Expression dataset. By means of the feature visualization, we found that the coded mask served as a linear operator to synthesize the spatial fidelity of the original scene, but kept the features for the post-recognition process. We believe the proposed framework can inspire more possibilities for the unconventional imaging system.

Automatic facial recognition using VGG16 based transfer learning model

Detecting and analyzing emotions from human facial movement is a problem defined and developed over many years for the benefits it brings. Facial expression is a crux of the human-computer interaction (HCI) research area. Researchers are exploring its application in security, medical science, and to know the behavior of a person or community. In this paper, we have proposed a deep learning-based framework using transfer learning for facial expression. This approach uses the existing VGG16 model for a modified trained model and concatenates additional layers on it. VGG16 model is already trained on ImageNet, which has 1000 classes. After this, the model has been verified on CK+, JAFFE benchmark datasets. Extended Cohn-Kanade (CK+) and Japanese Female Facial Expression (JAFFE) are popular Facial Expression Dataset. The proposed model has shown 94.8% accuracy on CK+ and 93.7% on the JAFFE dataset and found superior to existing techniques. We have implemented proposed technique on Google Colab-GPU that has helped us to process these data.

Ensemble of Multi Feature Layers in CNN for Facial Expression Recognition using Deep Learning

Facial Expression Recognition is an important undertaking for the machinery to recognize different expressive alterations in individual. Emotions have a strong relationship with our behavior. Human emotions are discrete reactions to inside or outside occasions which have some importance meaning. Involuntary sentiment detection is a process to understand the individual’s expressive state to identify his intensions from facial expression which is also a noteworthy piece of non-verbal correspondence. In this paper we propose a Framework that combines discriminative features discovered using Convolutional Neural Networks (CNN) to enhance the performance and accuracy of Facial Expression Recognition. For this we have implemented Inception V3 pre-trained architecture of CNN and then applying concatenation of intermediate layer with final layer which is further passing through fully connected layer to perform classification. We have used JAFFE (Japanese Female Facial Expression) Dataset for this purpose and Experimental results show that our proposed method shows better performance and improve the recognition accuracy.

Continuous emotion estimation of facial expressions on JAFFE and CK+ datasets for human–robot interaction

Human–robot interaction was always based on estimation of human emotions from human facial expressions, voice and gestures. Human emotions were always categorized in a discretized manner, while we estimate facial images from common datasets for continuous emotions. Linear regression was used in this study which numerically quantizes human emotions as valence and arousal by displaying the raw images on the two-respective coordinate axis. The face image datasets from the Japanese female facial expression (JAFFE) dataset and the extended Cohn–Kanade (CK+) dataset were used in this experiment. Human emotions for the above-mentioned datasets were interpreted by 85 participants who were used in the experimentation. The best result from a series of experiments shows that the minimum of root mean square error for the JAFFE dataset was 0.1661 for valence and 0.1379 for arousal. The proposed method has been compared with previous methods such as songs, sentences, and it is observed that the proposed method for common datasets testing showed an outstanding emotion estimation performance.

Two-channel convolutional neural network for facial expression recognition using facial parts

This paper proposes the design of a facial expression recognition system based on the deep convolutional neural network by using facial parts. In this work, a solution for facial expression recognition that uses a combination of algorithms for face detection, feature extraction and classification is discussed. The proposed method builds a two-channel convolutional neural network model in which facial parts are used as inputs: the extracted eyes are used as inputs to the first channel, while the mouth is the input into the second channel. Feature information from both channels converges in a fully connected layer which is used to learn global information from these local features and is then used for expression classification. Experiments are carried out on the Japanese female facial expression dataset and the extended Cohn-Kanada dataset to determine the expression recognition accuracy for the proposed facial expression recognition system based on facial parts. The results achieved shows that the system provides state of art classification accuracy with 97.6% and 95.7% respectively when compared to other methods.

Emotion recognition from facial expressions using hybrid feature descriptors

Here, a hybrid feature descriptor-based method is proposed to recognise human emotions from their facial expressions. A combination of spatial bag of features (SBoFs) with spatial scale-invariant feature transform (SBoF-SSIFT), and SBoFs with spatial speeded up robust transform are utilised to improve the ability to recognise facial expressions. For classification of emotions, K-nearest neighbour and support vector machines (SVMs) with linear, polynomial, and radial basis function kernels are applied. SBoFs descriptor generates a fixed length feature vector for all sample images irrespective of their size. Spatial SIFT and SURF features are independent of scaling, rotation, translation, projective transforms, and partly to illumination changes. A modified form of bag of features (BoFs) is employed by involving feature's spatial information for facial emotion recognition. The proposed method differs from conventional methods that are used for simple object categorisation without using spatial information. Experiments have been performed on extended Cohn–Kanade (CK+) and Japanese female facial expression (JAFFE) data sets. SBoF-SSIFT with SVM resulted in a recognition accuracy of 98.5% on CK+ and 98.3% on JAFFE data set. Images are resized through selective pre-processing, thereby retaining only the information of interest and reducing computation time.

Using the genetic algorithm to enhance nonnegative matrix factorization initialization

AbstractNonnegative matrix factorization (NMF) algorithms have been utilized in a wide range of real applications; however, the performance of NMF is highly dependent on three factors including: (1) choosing a problem dependent cost function; (2) using an effective initialization method to start the updating procedure from a near‐optimal point; and (3) determining the rank of factorized matrices prior to decomposition. Due to the nonconvex nature of the NMF cost function, finding an analytical‐based optimal solution is impossible. This paper is aimed at proposing an efficient initialization method to modify the NMF performance. To widely explore the search space for initializing the factorized matrices in NMF, the island genetic algorithm (IGA) is employed as a diverse multiagent search scheme. To adapt IGA for NMF initialization, we present a specific mutation operator. To assess how the proposed IGA initialization method efficiently enhances NMF performance, we have implemented state‐of‐the‐art initialization methods and applied to the Japanese Female Facial Expression dataset to recognize the facial expression states. Experimental results demonstrate the superiority of the proposed approach to the compared methods in terms of relative error and fast convergence.

Facial Expression Recognition in JAFFE Dataset Based on Gaussian Process Classification

The Gaussian process (GP) approaches to classification synthesize Bayesian methods and kernel techniques, which are developed for the purpose of small sample analysis. Here we propose a GP model and investigate it for the facial expression recognition in the Japanese female facial expression dataset. By the strategy of leave-one-out cross validation, the accuracy of the GP classifiers reaches 93.43% without any feature selection/extraction. Even when tested on all expressions of any particular expressor, the GP classifier trained by the other samples outperforms some frequently used classifiers significantly. In order to survey the robustness of this novel method, the random trial of 10-fold cross validations is repeated many times to provide an overview of recognition rates. The experimental results demonstrate a promising performance of this application.