Person-independent Facial Expression Research Articles

Gaussian-filtered Local Difference Pattern with kernel representation for person-independent facial expression recognition robust to noise and resolution

Gaussian-filtered Local Difference Pattern with kernel representation for person-independent facial expression recognition robust to noise and resolution

DRCP: Dimensionality Reduced Chess Pattern for Person Independent Facial Expression Recognition

Automatic Facial Expression Recognition (FER) has become essential today as it has many applications in real time such as animation, driver mood detection, lie detection, and clinical psychology. The effectiveness of FER systems mainly depends on the extracted features. For extracting distinctive features with low dimensions, a new local texture-based image descriptor named Dimensionality Reduced Chess Pattern (DRCP) is proposed for recognizing facial expressions in a person independent scenario. DRCP, an improvement over Chess Pattern (CP), is mainly proposed for effectively reducing the feature vector length of CP. For feature extraction, DRCP also considers the movements of chessmen in a [Formula: see text] neighborhood, as like CP. As a part of feature extraction through DRCP, apart from the center pixel, the remaining 24 pixels are arranged into four groups in such a manner that each group contains the pixels corresponding to three chessmen. From each group, one feature is extracted and thus corresponding to four groups, four features are extracted in a [Formula: see text] neighborhood. The extracted features are fed into multi-class Support Vector Machine (SVM) for expression recognition. The experiments are performed on five “in the lab” datasets (MUG, TFEID, JAFFE, CK[Formula: see text] and KDEF) and on two “in the wild” datasets (RAF and SFEW) in person independent setup to simulate a real world scenario.

Person-independent facial expression recognition based on local directional compact pattern

This paper proposes a novel descriptor for facial expression recognition called the Local Directional Compact Pattern (LDCP), which encodes the prominent information of local facial textures in a simple and compact way. We obtain edge responses for the local neighborhood of each pixel in four different directions by convolving the symmetric compass masks with the face image. LDCP uses direction, sign, and position information of the two top edge responses to generate a more distinctive code than existing methods that allows us to distinguish different textures with similar gradient directions. Unlike other methods in which the whole face is uniformly divided into several blocks to obtain a global feature vector by concatenating the histogram of each block, we select emotion-related blocks from the feature map to obtain the histogram-based feature vector of each block, which have a different contribution to exhibiting facial expressions. Again, we assign a weight to each block to classify facial expressions using a robust kernel representation algorithm. We conduct our experiments on the CK+ , FACES, MMI, and JAFFE datasets to compare our LDCP descriptor performance with 20 existing descriptors in terms of the number of bits, base length, feature extraction time, and recognition rate. In addition, we compare our proposed method with the recent state-of-the-art methods in different testing strategies.

Person-independent facial expression recognition based on the fusion of HOG descriptor and cuttlefish algorithm

Person-independent facial expression recognition based on the fusion of HOG descriptor and cuttlefish algorithm

Chess pattern with different weighting schemes for person independent facial expression recognition

Chess pattern with different weighting schemes for person independent facial expression recognition

New framework for person-independent facial expression recognition combining textural and shape analysis through new feature extraction approach

Automatic facial expression recognition (FER) has been extensively studied owing to its wide range of applications, such as in e-learning platforms used to automatically collect the feedback of students regarding a particular content and to help children with autism have a better understanding of their environment. Owing to the advances made in the fields of machine learning and computational devices, researchers are developing more accurate and robust facial expression recognition frameworks. In this paper, we propose a completely new framework for person-independent FER based on combining textural and shape features from 49 detected landmarks in an input facial image. The shape information is extracted using the histogram of oriented gradients (HOG) applied on a binary patch generated by interpolating the locations of the 49 detected landmarks. The textural information is computed from 49 sub-images, each centered on one landmark, using a new handcrafted descriptor that we also propose herein and is referred to as Orthogonal and Parallel-based Directions Generic Quad Map Binary Patterns (OPD-GQMBP). OPD-GQMBP encodes the relevant information based on the orthogonality and parallelism of the geometries to select the prominent pixels within a n×n neighborhood. The proposed framework outperforms many previous state-of-the-art methods including deep-learning-based approaches on five widely used benchmarks: CK+, KDEF, JAFFE, Oulu-Casia VIS, and RaFD, through the Leave-One-Subject-Out evaluation protocol. In addition, the superiority of the OPD-GQMBP descriptor is fairly proven against 10 deep features (e.g., VGG, ResNets, DenseNet, GoogeLeNet, and Inception) and 12 recent and powerful LBP variants.

Person-Independent Facial Expression Recognition Based on Improved Local Binary Pattern and Higher-Order Singular Value Decomposition

The recognition rate of person-independent facial expression is generally not high, which limits the practical application of facial expression recognition. Aiming at this problem, this paper analyzes the reasons for the low recognition rate of person-independent facial expression, and proposes a recognition algorithm of person-independent facial expression based on improved LBP (Local Binary Pattern) and HOSVD (Higher-Order Singular Value Decomposition). The algorithm has the following contributions of facial expression recognition framework. In the stage of facial expression feature extraction, the transient features extracted by LDP(Local Directional Pattern) and the persistent features extracted by CBP(Centralized Binary Pattern) are integrated to improve the discrimination of facial expression features. Moreover, in the stage of facial expression classification and recognition, the traditional nearest neighbor classification is changed into k-nearest neighbor pre-classification, and the regional energy calculated by HOSVD is used to determine the similarity of two images for secondary classification. Finally, in the extended Cohn-Kanade dataset and Oulu-CASIA NIR&VIS facial expression database, the theoretical analysis and experimental results show that the method has better recognition effect for solving the problem of person-independent facial expression recognition.

Person-independent facial expression recognition method based on improved Wasserstein generative adversarial networks in combination with identity aware

Since the distinction between two expressions is fairly vague, usually a subtle change in one part of the human face is enough to change a facial expression. Most of the existing facial expression recognition algorithms are not robust enough because they rely on general facial features or algorithms without considering differences between facial expression and facial identity. In this paper, we propose a person-independent recognition method based on Wasserstein generative adversarial networks for micro-facial expressions, where a facial expression recognition network and a facial identity recognition network are established to improve the accuracy and robustness of facial expression recognition via inhibition of intra-class variation. Extensive experimental results demonstrate that 90% average recognition accuracy of facial expression has been reached on a mixed dataset composed of CK+, Multi-PIE, and JAFFE. Moreover, our method achieves 96% accuracy of person-independent recognition on CK+. A 4.5% performance gain is achieved with the novel identity-inhibited expression feature. The proposed algorithm in this paper has been successfully applied to Haikang Visual Integrated Management Platform (iVMS-8700). At present, it runs well and can effectively recognize facial expressions.

Local directional-structural pattern for person-independent facial expression recognition

Existing popular descriptors for facial expression recognition often suffer from inconsistent feature description, experiencing poor accuracies. We present a new local descriptor, local directional-structural pattern (LDSP), in this work to address this issue. Unlike the existing local descriptors using only the texture or edge information to represent the local structure of a pixel, the proposed LDSP utilizes the positional relationship of the top edge responses of the target pixel to extract more detailed structural information of the local texture. We further exploit such information to characterize expression-affiliated crucial textures while discarding the random noisy patterns. Moreover, we introduce a globally adaptive thresholding strategy to exclude futile flat patterns. Hence, LDSP offers a stable description of facial expressions with the explicit representation of the expression-affiliated features along with the exclusion of random futile textures. We visualize the efficacy of the proposed method in three folds. First, the LDSP descriptor possesses a moderate code-length owing to the exclusion of the futile patterns, yielding less computation time than other edge descriptors. Second, for person-independent expression recognition in benchmark datasets, LDSP demonstrates higher accuracy than existing descriptors and other state-of-the-art methods. Third, LDSP shows better performance than other descriptors against noise and low resolution, exhibiting its robustness under such uneven conditions.

Subject adaptation using selective style transfer mapping for detection of facial action units

A key assumption in some learning methods in intelligent systems is that the probability distribution of both test and training data is the same. However, the identical distribution assumption does not hold true for person-independent facial expression recognition. This is because the appearance of an expression may significantly vary for different people. Therefore, domain adaptation methods have been proposed to bring the performance of a person-independent system closer to a person-dependent one. Mismatched conditions between training data and new subject data vary based on the individual. Selective style transfer mapping (SSTM) is an instance-transfer method that will not require re-training classifier and can be classifier independent. This method is proposed to increase the generalization ability of action unit detection through the selection of style transfer mapping type (linear or nonlinear) for different persons. We also propose a rapid SSTM that uses the neutral vectors from a particular person (a small amount of data) to improve action unit detection. Rapid SSTM is also first method for style transfer using a SSTM framework without the need of full labelled target data. The F1 score of selective style transfer mapping for action unit detection in the UNBC-McMaster database is 81.15, which is a significant (P < 0.05) improvement over the style transfer mapping, which is 60.30. The results also show that our approach can effectively perform the task of action unit detection with better generalization of the subjects in the other database. The training database was CK+, but the test database was UNBC-McMaster.

Video‐based facial expression recognition by removing the style variations

This study examines the performance of person-independent facial expression recognition improved by adapting the system to a given person. The proposed method transfers the style of particular subjects to the semi-style-free space. There is no need to change the person-independent classifier in order to improve the performance. The style transfer mapping (STM) has been proposed in image-based classification. The challenges of employing this technique in video-based facial expression recognition are: estimating STM from image sequences of each subject (adaptation data) and projecting new sequential data of each subject in semi-style-free space. A mixture of ‘binary support vector machines’ and ‘hidden Markov models’ were employed to overcome these challenges. Moreover, virtual samples generated by using the person's neutral samples were used to estimate STM. Experimental results on the CK+ database confirm the efficiency of the proposed method in recognition rate improvement.

Human’s Facial Parts Extraction to Recognize Facial Expression

Real-time facial expression analysis is an important yet challenging task in human computer interaction. This paper proposes a real-time person independent facial expression recognition system using a geometrical feature-based approach. The face geometry is extracted using the modified active shape model. Each part of the face geometry is effectively represented by the Census Transformation (CT) based feature histogram. The facial expression is classified by the SVM classifier with exponential chi-square weighted merging kernel. The proposed method was evaluated on the JAFFE database and in real-world environment. The experimental results show that the approach yields a high recognition rate and is applicable in real-time facial expression analysis.

Person-independent facial expression analysis by fusing multiscale cell features

Automatic facial expression recognition is an interesting and challenging task. To achieve satisfactory accuracy, deriving a robust facial representation is especially important. A novel appearance-based feature, the multiscale cell local intensity increasing patterns (MC-LIIP), to rep- resent facial images and conduct person-independent facial expression analysis is presented. The LIIP uses a decimal number to encode the tex- ture or intensity distribution around each pixel via pixel-to-pixel intensity comparison. To boost noise resistance, MC-LIIP carries out comparison computation on the average values of scalable cells instead of individual pixels. The facial descriptor fuses region-based histograms of MC-LIIP features from various scales, so as to encode not only textural microstruc- tures but also the macrostructures of facial images. Finally, a support vec- tor machine classifier is applied for expression recognition. Experimental results on the CKþ and Karolinska directed emotional faces databases show the superiority of the proposed method. © 2013 Society of Photo-Optical

Person Independent Facial Expression Detection using MBWM and Multiclass SVM

Facial expression analysis is an attractive, challenging and important field of study in facial analysis. It’s important applications include many areas such as human–computer interaction, human emotion analysis, biometric authentication, exhaustion detection and data-driven animation. For successful facial expression recognition, the first step is to arrive at an appropriate facial representation from original face image which is a crucial step. This paper, empirically evaluate facial representation using statistical features from the Local Binary Patterns, Simplified local binary mean and Mean based weight matrix for person-independent facial expression recognition. Multiclass SVM is applied systematically for classification. The Japanese female database JAFFE is used for the experiment. Extensive experiments shows that statistical features derived from LBP are effective and efficient for facial expression recognition. Further improved and best results are obtained with SLBM and MBWM features extracted using Multiclass Support Vector Machine classifiers.

Localized discriminative scale invariant feature transform based facial expression recognition

This paper presents a discriminative scale invariant feature transform (D-SIFT) based feature representation for person-independent facial expression recognition. Keypoint descriptors of the SIFT features are used to construct distinctive facial feature vectors. Kullback Leibler divergence is used for the initial classification of the localized facial expressions and weighted majority voting based classifier is employed to fuse the decisions obtained from localized rectangular facial regions to generate the overall decision. Experiments on the Bosphorus and BU-3DFE databases illustrate that the D-SIFT is effective and efficient for facial expression recognition.

3D facial expression recognition using SIFT descriptors of automatically detected keypoints

Methods to recognize humans’ facial expressions have been proposed mainly focusing on 2D still images and videos. In this paper, the problem of person-independent facial expression recognition is addressed using the 3D geometry information extracted from the 3D shape of the face. To this end, a completely automatic approach is proposed that relies on identifying a set of facial keypoints, computing SIFT feature descriptors of depth images of the face around sample points defined starting from the facial keypoints, and selecting the subset of features with maximum relevance. Training a Support Vector Machine (SVM) for each facial expression to be recognized, and combining them to form a multi-class classifier, an average recognition rate of 78.43% on the BU-3DFE database has been obtained. Comparison with competitor approaches using a common experimental setting on the BU-3DFE database shows that our solution is capable of obtaining state of the art results. The same 3D face representation framework and testing database have been also used to perform 3D facial expression retrieval (i.e., retrieve 3D scans with the same facial expression as shown by a target subject), with results proving the viability of the proposed solution.

Person-independent facial expression recognition based on expression subspace multi-classifiers integration

To the problem that the average recognition rate of person-independent facial expression is not high(about 65%),a new method of facial expression recognition,based on expression subspace and multi-classifiers integration,was proposed.In the training set 1,the features of global face region,eyes(include eyebrows) region and mouth region were respectively extracted and decomposed by Local Binary Pattern(LBP) and Higher Order Singular Value Decomposition(HOSVD),and the corresponding expression subspaces were built.Then the facial images of the training set 2 were trained by Support Vector Machine(SVM) in the expression subspaces and the parameters of fuzzy rule system were conducted.Finally,the expression subspaces and the multi-classifiers ensemble were combined to classify the expressions in test set.The experiments were conducted on JAFFE database and the average recognition rate was 71.43%.The experimental results show that the proposed method effectively reduces the influence caused by facial shape feature and facial expression manner,and it has better recognition rate.

Real-time facial expression recognition using STAAM and layered GDA classifier

This paper proposes a real-time person independent facial expression recognition in two parts: one is a model fitting part using a proposed stereo active appearance model (STAAM) and another is a person independent facial expression recognition using a layered generalized discriminant analysis (GDA) classifier. The STAAM fitting algorithm uses multiple calibrated perspective cameras to compute the 3D shape and rigid motion parameters. The use of calibration information reduces the number of model parameters, restricts the degree of freedom in the model parameters, and increases the accuracy and speed of fitting. The STAAM uses a modified simultaneous update fitting method that reduces the fitting computation greatly. Also, the layered GDA classifier combines 3D shape and 2D appearance to improve the recognition performance of person independent facial expressions. Experimental results show that (1) the STAAM shows a better fitting stability than the existing multiple-view AAM (MVAAM), (2) the modified simultaneous update algorithm accelerates the AAM fitting speed, and (3) the combination of the 3D shape and 2D appearance features using a layered GDA classifier improves the performance of facial expression recognition greatly.

Facial expression recognition based on Local Binary Patterns: A comprehensive study

Automatic facial expression analysis is an interesting and challenging problem, and impacts important applications in many areas such as human–computer interaction and data-driven animation. Deriving an effective facial representation from original face images is a vital step for successful facial expression recognition. In this paper, we empirically evaluate facial representation based on statistical local features, Local Binary Patterns, for person-independent facial expression recognition. Different machine learning methods are systematically examined on several databases. Extensive experiments illustrate that LBP features are effective and efficient for facial expression recognition. We further formulate Boosted-LBP to extract the most discriminant LBP features, and the best recognition performance is obtained by using Support Vector Machine classifiers with Boosted-LBP features. Moreover, we investigate LBP features for low-resolution facial expression recognition, which is a critical problem but seldom addressed in the existing work. We observe in our experiments that LBP features perform stably and robustly over a useful range of low resolutions of face images, and yield promising performance in compressed low-resolution video sequences captured in real-world environments.

Affective interaction based on person-independent facial expression space

This paper proposes a person-independent facial expression space (PIFES) to analyze and synthesize facial expressions based on supervised locality preserving projections (SLPP), which aligns different subjects and different intensities of facial expressions on one generalized expression manifold. Interactive curves of different patterns are generated according to the input facial expression image sequence, and target responsive expression images are synthesized for different emotions. In order to synthesize arbitrary expressions for a new person with natural details, a novel approach based on local geometry preserving between the input face image and the target expression image is proposed. Experimental results clearly demonstrate the efficiency of the proposed algorithm.