Accurate and robust facial expression recognition system using real-time YouTube-based datasets

Abstract

This paper presents an accurate and robust real-time FER system. In this system, an unsupervised technique based on active contour (AC) model is adopted in order to detect and extract the human faces automatically from the facial expression frames. In this model, the combination of two energy functions like Chan-Vese (CV) energy and Bhattacharyya distance functions were exploited that not only minimize the dissimilarities within the object (face) but also maximize the distance between the object (face) and background. Moreover, we extracted the facial features by proposing a new feature extraction method in order to solve the limitations of the previous works of the feature extraction. Similarly, in this system, we also proposed the usage of a robust non-linear feature selection method called stepwise linear discriminant analysis (SWLDA) that focuses on selecting localized features from facial expression images and discriminating their classes based on regression values (i.e., partial F-test). Finally, the system has been trained by employing hidden Markov model (HMM) to label the expressions. Unlike most of the previous works that were evaluated using a single dataset in a controlled environment, the performance of the proposed system have been assessed by employing three different spontaneous datasets that have been collected in naturalistic environments. 10–fold cross validation rule has been exploited for the whole experiments. In last, a set of experiments were also performed to assess the effectiveness of each module of the proposed approaches separately. The proposed system achieved weighted average recognition rate (95%) across three different YouTube-based datasets against the existing state-of-the-art methods.