Cohn-Kanade Facial Expression Database Research Articles

Facial expression recognition using digital signature feature descriptor

Facial feature extraction is the most crucial part for efficient representation of facial images. For facial expression recognition, efficiency of recognition mainly depends on discriminative nature of features that describe optical changes of facial expressions. Facial expressions are very dynamic in nature. The dynamics of facial muscle changes is required to be captured and encoded accurately. Facial expressions are contractions and expansions of facial muscles which results in high-frequency edges on different part of facial image. A novel feature extraction framework called digital signature based on high-frequency edges in combination with LBP histogram features is developed, and the proposed methodology is used for facial expression recognition (FER). Digital signature descriptor of facial dynamic is obtained by projecting edge pixels vertically and horizontally. Digital signature uniquely and completely describes the facial expressions. Support vector method classifier is used to classify six basic expressions based on one-against-all strategy of classification. The validity of proposed algorithm is tested on standard widely used Cohn–Kanade Facial Expression Database (CKFED), Taiwanese Facial Expression Database (TFED) and Japanese Female Facial Expression databases (JFED). Experimental results show that the efficiency of expression recognition of proposed novel technique is 96.25% on CKFED which is higher than other existing methods.

Rancang Bangun Aplikasi Face Tracking dan Filter Berdasarkan Raut Wajah Menggunakan Algoritma Fisher-Yates Berbasis iOS

Expressions of facial expressions in addition to providing important emotional indicators, are very important objects in our daily lives too. Real-time video processing on mobile devices is a hot topic and has a very broad application. Photos that have used the filter have 21% more possibilities to be seen and 45% more likely to be commented on by photo consumers. The use of the Fisher-Yates algorithm is used as a filter scrambler for each facial expression emotion. The application is made for the iOS operating system with the Swift programming language that utilizes the Core ML and Vision framework. Custom Vision is used as a tool for creating and training models. In making a model, this study uses a dataset from Cohn-Kanade AU-Coded Facial Expression Database and Karolinska Directed Emotional Faces. Custom Vision can provide performance result training and provide precision and recall values ​​for data that has been trained. The facial expression match with the model is determined by the confidence level value. The results of trials with Hedonic Motivation System Adoption Model method produce a percentage of pleasure in using the application (joy) of 79.39% of the users agree that the application provides joy. 

MQSMER: a mixed quadratic shape model with optimal fuzzy membership functions for emotion recognition

The traditional geometrical-based approaches used in facial emotion recognition fail to capture the uncertainty present in the quadrilateral shape of emotions under analysis, which reduces the recognition accuracy rate. Furthermore, these approaches require extensive computational time to extract the facial features and to train the models. This article proposes a novel geometrical fuzzy-based approach to accurately recognize facial emotions in images in less time. The four corner vertices of the mouth are the most important features to recognize facial emotions and can be extracted without the need of a reference face. These extracted features can then be used to define the quadrilateral shape, and the associated degree of impreciseness in the shape can be accessed using the proposed geometric fuzzy membership functions. Hence, four fuzzy features are derived from the membership functions and given to classifiers for emotion evaluations. In our tests, the fuzzy features achieved an accuracy rate of 96.17% in the Japanese Female Facial Expression database, and 98.32% in the Cohn-Kanade Facial Expression database, which are higher than the ones achieved by other common up-to-date methods. In terms of computational time, the proposed method required an average of 0.375 s to build the used model in a common PC.

Saliency-based framework for facial expression recognition

This article proposes a novel framework for the recognition of six universal facial expressions. The framework is based on three sets of features extracted from a face image: entropy, brightness, and local binary pattern. First, saliency maps are obtained using the state-of-the-art saliency detection algorithm “frequency-tuned salient region detection”. The idea is to use saliency maps to determine appropriate weights or values for the extracted features (i.e., brightness and entropy). We have performed a visual experiment to validate the performance of the saliency detection algorithm against the human visual system. Eye movements of 15 subjects were recorded using an eye-tracker in free-viewing conditions while they watched a collection of 54 videos selected from the Cohn-Kanade facial expression database. The results of the visual experiment demonstrated that the obtained saliency maps are consistent with the data on human fixations. Finally, the performance of the proposed framework is demonstrated via satisfactory classification results achieved with the Cohn-Kanade database, FG-NET FEED database, and Dartmouth database of children’s faces.

Facial expression recognition based on eigenspaces and principle component analysis

Facial expression detection or emotion recognition is one of the rising fields of research on intelligent systems. Emotion plays a significant role in non-verbal communication. An efficient face and facial feature detection algorithms plays as important role in indentifying of an emotion of a person at a particular moment. In this work, the authors implemented a system that recognises a person's facial expressions from the input images, using the algorithm of eigenspaces and principle component analysis (PCA). Eigenspaces are the face images which are projected onto a feature space that encodes the variation among known face images. PCA is used in this paper to make dimensional reduction of images in order to obtain a reduced representation of face images. The implementation is applied on three different facial expressions databases, extended Cohn-Kanade facial expression database, Japanese female facial expression database and self-made database in order to find out the effectiveness of the proposed method.

Facial expression recognition based on eigenspaces and principle component analysis

Facial expression detection or emotion recognition is one of the rising fields of research on intelligent systems. Emotion plays a significant role in non-verbal communication. An efficient face and facial feature detection algorithms plays as important role in indentifying of an emotion of a person at a particular moment. In this work, the authors implemented a system that recognises a person's facial expressions from the input images, using the algorithm of eigenspaces and principle component analysis (PCA). Eigenspaces are the face images which are projected onto a feature space that encodes the variation among known face images. PCA is used in this paper to make dimensional reduction of images in order to obtain a reduced representation of face images. The implementation is applied on three different facial expressions databases, extended Cohn-Kanade facial expression database, Japanese female facial expression database and self-made database in order to find out the effectiveness of the proposed method.

Statistical Assessment of PCA/SVD and FFT-PCA/SVD on Variable Facial Expressions

Face recognition is a dedicated process in the human brain. Automatic face recognition is rewarding since an efficient and resilient recognition system is useful in many application areas. Recent face recognition algorithms are still faced with the challenge of recognizing face image under variable environmental constraints. This paper presents a statistical evaluation of the performance of two face recognition algorithms namely, Principal Component Analysis with Singular Value Decomposition (PCA/SVD) and Principal Component Analysis with Singular Value Decomposition using Fast Fourier Transform for preprocessing (FFT-PCA/SVD) on variable facial expressions (Angry, Disgust, Fear, Happy, Sad and Surprise) along with their neutral expressions. We considered 42 individuals from Cohn Kanade Facial Expressions database, Japanese Female Facial Expressions (JAFFE) and a created Ghanaian Face database for recognition runs. Multivariate statistical methods were used in the assessment of the face recognition algorithms. GNU Octave was used to perform all numerical runs and statistical evaluation of the recognition algorithms. The results of the statistical evaluation show that, FFT-PCA/SVD is comparatively consistent (Low variation) and efficient (Higher recognition rate) than PCA/SVD algorithm in the recognition of variable facial expressions. The paper also proposes Fast Fourier Transform as a viable noise removal mechanism that should be adopted during image preprocessing.

Facial expression recognition based on characteristics of block LGBP and sparse representation

Aiming at the difficulty of distinguishing texture feature in facial expressions recognition, this paper put forward a LGBP facial expression recognition algorithm based on the character blocking and sparse representation. The main contents are training the block images of different categories expr ession images and extracting the LGBP features of each sub-block. We construct a over-complete dictionary from which we get the discrepancy vector of each sub-block using sparse representation. Through finding the minimum residual vectors to achieve the recognition of different facial expressions. To some extent, the experimental results based on the JAFFF and Cohn-kanade facial expression database show that this algorithm can effectively overcome the influence of texture feature differences and have higher recognition rate.

Research on Locality Preserving Discriminant Projection Algorithm Based on Gabor for Face Expression Recognition

For the problem of features extraction and dimensionality reduction of expression recognition, the paper proposes Gabor Locality Preserving Discriminant Projection (GLPDP) algorithm, which is based on Gabor Wavelet. Firstly, we use Gabor wavelet transform to have an expression feature extraction. Secondly, we improved the locality preserving projection (LPP) algorithm, introducing scatter difference in the LPP objective function to increase divergence constraints among the sample classes and extracts more discriminated features while having the dimensionality reduction. Finally, we use the nearest neighbor classifier to have a classification for expression category. The effectiveness of the proposed methods is validated through the experimental results on JAFFE and Cohn-Kanade Facial expression databases.

Using multiple steerable filters and Bayesian regularization for facial expression recognition

Facial expression recognition has recently become a challenging research area. Its applications include human–computer interfaces, human emotion analysis, and medical care and cure.In this paper, we present a new challenging method to recognize seven universal emotional expressions, which are happiness, neutral, angry, disgust, sadness, fear and surprise. In our approach, we identify the user׳s facial expressions from the input images, using statistical features extracted from the steerable pyramid decomposition, and classified with a Bayesian regularization neural network. The evaluation of the proposed approach in terms of recognition accuracy is achieved using two universal databases, the Japanese Female Facial Expression database and the Cohn–Kanade facial expression database. The overall accuracy rate reaches 93.33% for the first database and is about 98.13% for the second one. These results show the effectiveness of the steerable proposed algorithm.

Facial Expression Recognition Based on Linear Discriminant Locality Preserving Analysis Algorithm

The traditional LDA algorithm is used to seek the optimal distinguishing vector spaces, but it only considers the global structure of samples. The LPP algorithm can preserve the local structure of data very well, however, its projection direction is likely to cause the class overlap when applied to the facial expression recognition which is one of the multiple-classification problems. To resolve the class overlap problem, this paper presents a new Linear Discriminant Locality Preserving Analysis (LDLPA) algorithm which takes full advantage of LDA and LPP for facial expression recognition. It utilizes the strong discriminative superiority of LDA to find the optimal distinguish vector space and joins the thinking of LPP to preserve the neighborhood relationship among the data points. By reconstructing the between-class scatter and minimizing within-class scatter of LDLPA objective function, it makes the sample points of different classes be away from each other and the sample points of same classes be close to each other to get the optimal discriminative features. Finally, the validity of LDLPA is verified by the experiments on JAFFE and Cohn-Kanade facial expression databases.

Geometric Feature-Based Facial Expression Recognition in Image Sequences Using Multi-Class AdaBoost and Support Vector Machines

Facial expressions are widely used in the behavioral interpretation of emotions, cognitive science, and social interactions. In this paper, we present a novel method for fully automatic facial expression recognition in facial image sequences. As the facial expression evolves over time facial landmarks are automatically tracked in consecutive video frames, using displacements based on elastic bunch graph matching displacement estimation. Feature vectors from individual landmarks, as well as pairs of landmarks tracking results are extracted, and normalized, with respect to the first frame in the sequence. The prototypical expression sequence for each class of facial expression is formed, by taking the median of the landmark tracking results from the training facial expression sequences. Multi-class AdaBoost with dynamic time warping similarity distance between the feature vector of input facial expression and prototypical facial expression, is used as a weak classifier to select the subset of discriminative feature vectors. Finally, two methods for facial expression recognition are presented, either by using multi-class AdaBoost with dynamic time warping, or by using support vector machine on the boosted feature vectors. The results on the Cohn-Kanade (CK+) facial expression database show a recognition accuracy of 95.17% and 97.35% using multi-class AdaBoost and support vector machines, respectively.

Semi-supervised dimensionality reduction using estimated class membership probabilities

In solving pattern-recognition tasks with partially labeled training data, the semi-supervised dimensionality reduction method, which considers both labeled and unlabeled data, is preferable for improving the classification and generalization capability of the testing data. Among such techniques, graph-based semi-supervised learning methods have attracted a lot of attention due to their appealing properties in discovering discriminative structure and geometric structure of data points. Although they have achieved remarkable success, they cannot promise good performance when the size of the labeled data set is small, as a result of inaccurate class matrix variance approximated by insufficient labeled training data. In this paper, we tackle this problem by combining class membership probabilities estimated from unlabeled data and ground-truth class information associated with labeled data to more precisely characterize the class distribution. Therefore, it is expected to enhance performance in classification tasks. We refer to this approach as probabilistic semi-supervised discriminant analysis (PSDA). The proposed PSDA is applied to face and facial expression recognition tasks and is evaluated using the ORL, Extended Yale B, and CMU PIE face databases and the Cohn-Kanade facial expression database. The promising experimental results demonstrate the effectiveness of our proposed method.

Facial Expressions Recognition Using Eigenspaces

A challenging research topic is to make the Computer Systems to recognize facial expressions from the face image. A method of facial expression recognition, based on Eigenspaces is presented in this study. Here, the authors recognize the user’s facial expressions from the input images, using a method that was customized from eigenface recognition. Evaluation was done for this method in terms of identification correctness using two different Facial Expressions databases, Cohn-Kanade facial expression database and Japanese Female Facial Expression database. The results show the effectiveness of proposed method.

Robust Facial Expression Recognition via Compressive Sensing

Recently, compressive sensing (CS) has attracted increasing attention in the areas of signal processing, computer vision and pattern recognition. In this paper, a new method based on the CS theory is presented for robust facial expression recognition. The CS theory is used to construct a sparse representation classifier (SRC). The effectiveness and robustness of the SRC method is investigated on clean and occluded facial expression images. Three typical facial features, i.e., the raw pixels, Gabor wavelets representation and local binary patterns (LBP), are extracted to evaluate the performance of the SRC method. Compared with the nearest neighbor (NN), linear support vector machines (SVM) and the nearest subspace (NS), experimental results on the popular Cohn-Kanade facial expression database demonstrate that the SRC method obtains better performance and stronger robustness to corruption and occlusion on robust facial expression recognition tasks.

A realistic dynamic facial expression transfer method

We present a novel approach for synthesizing the dynamic facial expressions of the source subject and transferring the dynamic expression to the target subject. The synthesized animation of the target subject preserves both the facial appearance of the target subject and expression deformation of the source subject. We use active appearance model to separate and align the shapes and texture of the multi-expression facial images. The dynamic facial expressions of the source subject are obtained by the nonlinear TensorFace trained on a small sample size. Through interpolating the aligned sequential shapes of different expressions, we obtain the smooth shape variations under different expressions, according to which we warp the neutral faces to other expressions. However, the warped expressions are missing of the expression details. We transfer the facial detail obtained by nonlinear TensorFace to the warped dynamic expression faces with the proposed strategy. Experiments on the extended Cohn-Kanade (CK+) facial expression database show that our results have higher perceptual quality than state-of-the-art methods.

Recognition of Human Facial Expression in a Video Image using the Active Appearance Model

Abstract —Tracking human facial expression within a video image has many useful applications, such as surveillance and teleconferencing, etc. Initially, the Active Appearance Model (AAM) was proposed for facial recognition; however, it turns out that the AAM has many advantages as regards continuous facial expression recognition. We have implemented a continuous facial expression recognition system using the AAM. In this study, we adopt an independent AAM using the Inverse Compositional Image Alignment method. The system was evaluated using the standard Cohn-Kanade facial expression database, the results of which show that it could have numerous potential applications. Keywords —Active Appearance Model, Facial Expression Recognition, Image Alignment Method 1. I NTRODUCTION Facial expression recognition is a crucial method of inferring human emotions. Facial emo-tions are basically categorized into six facial expressions (surprise, fear, sadness, anger, disgust, happiness). The process flow of the present system is shown in Figure 1. Facial-expression im-ages are captured from a web camera. The active appearance model includes various parameters of shape and appearance. With these images, an AAM

Facial expression recognition using intelligent optical neural networks

The goal of this paper is automating facial expression analysis in facial images and image sequences using intelligent optical neural networks. Humans detect and interpret faces and facial expressions in a scene with little or no effort. Still, development of an automated system that accomplishes this task is rather difficult. A system that performs these operations accurately and in real time would form a big step in achieving a human-like interaction between man and machine. Automating facial expression analysis could bring facial expressions into man-machine interaction as a new modality and make the interaction tighter and more efficient. This paper proposes an optical neural network in detection of the facial expressions in images by extracting the gabor texture features. The Karhunen-Loeve Transform identifies the facial expression of the detected face. Results are analysed using the Cohn-Kanade facial expression database and the classification rate is proved higher compared to the approaches found ...

Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions

Dynamic texture (DT) is an extension of texture to the temporal domain. Description and recognition of DTs have attracted growing attention. In this paper, a novel approach for recognizing DTs is proposed and its simplifications and extensions to facial image analysis are also considered. First, the textures are modeled with volume local binary patterns (VLBP), which are an extension of the LBP operator widely used in ordinary texture analysis, combining motion and appearance. To make the approach computationally simple and easy to extend, only the co-occurrences of the local binary patterns on three orthogonal planes (LBP-TOP) are then considered. A block-based method is also proposed to deal with specific dynamic events such as facial expressions in which local information and its spatial locations should also be taken into account. In experiments with two DT databases, DynTex and Massachusetts Institute of Technology (MIT), both the VLBP and LBP-TOP clearly outperformed the earlier approaches. The proposed block-based method was evaluated with the Cohn-Kanade facial expression database with excellent results. The advantages of our approach include local processing, robustness to monotonic gray-scale changes, and simple computation.

Recognition of facial expressions and measurement of levels of interest from video

This paper presents a spatio-temporal approach in recognizing six universal facial expressions from visual data and using them to compute levels of interest. The classification approach relies on a two-step strategy on the top of projected facial motion vectors obtained from video sequences of facial expressions. First a linear classification bank was applied on projected optical flow vectors and decisions made by the linear classifiers were coalesced to produce a characteristic signature for each universal facial expression. The signatures thus computed from the training data set were used to train discrete hidden Markov models (HMMs) to learn the underlying model for each facial expression. The performances of the proposed facial expressions recognition were computed using five fold cross-validation on Cohn-Kanade facial expressions database consisting of 488 video sequences that includes 97 subjects. The proposed approach achieved an average recognition rate of 90.9% on Cohn-Kanade facial expressions database. Recognized facial expressions were mapped to levels of interest using the affect space and the intensity of motion around apex frame. Computed level of interest was subjectively analyzed and was found to be consistent with "ground truth" information in most of the cases. To further illustrate the efficacy of the proposed approach, and also to better understand the effects of a number of factors that are detrimental to the facial expression recognition, a number of experiments were conducted. The first empirical analysis was conducted on a database consisting of 108 facial expressions collected from TV broadcasts and labeled by human coders for subsequent analysis. The second experiment (emotion elicitation) was conducted on facial expressions obtained from 21 subjects by showing the subjects six different movies clips chosen in a manner to arouse spontaneous emotional reactions that would produce natural facial expressions.