Edge-enhanced bi-dimensional empirical mode decomposition-based emotion recognition using fusion of feature set

Abstract

Emotion recognition has been of great interest in psychology, machine intelligence, human–machine interaction and biomedical fields. This paper proposes a novel soft computing technique for facial emotion recognition by introducing edge- enhanced bidimensional empirical mode decomposition (EEBEMD) as a feature extraction tool for facial emotion recognition. Facial images are subjected to optimized cost function-based self-guided edge enhancement algorithm. BEMD has been applied on the edge- enhanced facial images, and the first four intrinsic mode functions (IMFs) and the residue have been calculated. On the basis of an empirical analysis, the first IMF is selected for further analysis. A proposed fusion model that consists of selected features from the gray-level co-occurrence matrix, the histogram of oriented gradients and the local ternary pattern of the IMF response is fed to a recursive feature elimination-based algorithm to select the appropriate feature subsets for classification. These feature vectors have been trained in three machine learning algorithms namely multi-class SVM, ELM with RBF kernel and k-NN classifier independently. The IMFs have been subjected to principal component analysis and linear discriminant analysis (LDA) algorithm successively for dimensionality reduction, and the facial images with different emotions have been clustered in different zones in the LDA subspace. The proposed method demonstrates promising accuracy when tested on the JAFFE database, Cohn–Kanade database and the eNTERFACE database.