Facial expression and EEG signal based classification of emotion

Abstract

The paper provides a novel approach to emotion recognition from facial expression and Electro Encephalograph (EEG) signal of subjects. Five subjects are requested to watch particular videos for arousing five different emotions in their mind. The facial expressions and EEG signal of subjects are recorded by a good quality camera and EEG machine respectively while watching the movie clips. Facial features include mouth-opening, eye-opening, eyebrow-constriction, and EEG features include, 132 number of Wavelet coefficients, 16 numbers of Kalman Filter coefficients and power spectral density, are then extracted from the facial expression and EEG signal frames. Then these huge numbers of features are reduced by Principle Component Analysis (PCA) and feature vector is constructed for 5 different emotions. A linear Support Vector Machine classifier is used to classify the extracted feature vectors into different emotion classes. Experimental results confirm that the recognition accuracy of emotion up to a level of 97% is maintained, even when the mean and standard deviation of noise are as high as 5% and 20% respectively over the individual features.