Emotion Classification of Facial Thermal Images using Sparse Coded Filters

Abstract

In affective computing, identifying the true emotion of an individual is still a significant concern. Under real-life conditions, muscle movements are found to be unreliable to identify the behavior of a person. Researchers have used thermal modality to recognize real emotions; however, features used were originally handcrafted for visible modality and were directly adopted for thermal modality. Since visible and thermal images are built with different principles and have distinct characteristics, adopted features do not perform well. This paper presents an algorithm to classify six basic emotions from thermal facial images. The primary aim is to find the thermal modality-specific filters using a subset of the NVIE dataset. For this, an optimal set of local region-specific filters are generated using convolutional sparse coding. The optimal set of filters is used for feature extraction in which the idea of a histogram-based feature descriptor known as binarized statistical image features (BSIF) is used. Further, a supervised dimensionality reduction algorithm acknowledging the correlation between classes is employed based on the idea of discriminant correlation analysis (DCA). Finally, six emotions are classified using a linear support vector machine (SVM). The improved accuracy validates the performance of the proposed method with respect to previous works.