FACIAL EMOTION DETECTION OF THERMAL AND DIGITAL IMAGES BASED ON MACHINE LEARNING TECHNIQUES

Abstract

The aim of the study is (i) to determine temperature distribution for various emotions from the facial thermal images; (ii) to extract statistical features from the facial region using GLCM feature extraction technique and to classify the emotions using machine learning classifiers such as SVM and Naïve Bayes; (iii) to develop the custom CNN model for the classification of various emotions and compare its performance with machine learning classifiers. Fifty normal subjects were considered for the study to analyze the facial emotions using thermal and digital images. The four different emotions, such as happy, angry, neutral and sad, were obtained with a total image of 200 thermal and 200 digital images. Ten statistical features were extracted using the GLCM method from both thermal and digital images and fed into the machine learning classifiers. After data augmentation, the images are fed into the custom CNN model for the classification of various emotions. The SVM classifier produced an accuracy of 80% in thermal images and 76.5% in digital images compared to the Naive Bayes classifier. The developed CNN model improved the classification accuracy to 94.3% and 90.3% for thermal and digital image, respectively, for the multi-class classification of facial emotions. The CNN model implemented using thermal images provided better classification accuracy than digital images in facial emotion recognition. Hence, it was proved that thermal imaging techniques resulted in better performance in predicting facial emotion than digital images.