Facial Expression Recognition Based on Ensemble Extreme Learning Machine with Eye Movements Information

Abstract

Facial expression recognition has become a very active research in computer vision, behavior interpretation of emotions, human computer interaction, cognitive science and intelligent control. Traditional facial expression analysis methods mainly focuses on the facial muscle movement and basic expression features of face image. In this paper, we propose a novel method for facial expression recognition based on ensemble extreme learning machine with eye movements information. Here, the eye movements information is regarded as explicit clue to improve the performance of facial expression recognition. Firstly, we extract eye movements features from eye movements information which recorded by Tobii eye tracker. The histogram of orientation gradient (HOG) features are simultaneously obtained from the face images by dividing it into a number of small cells. Secondly, we combine the eye movements features together with the HOG features of face images by using a tensor kernel. Finally, the fusion features are trained by ensemble extreme learning machine and a bagging algorithm is explored for producing the results. Extensive experiment on the two widely available datasets of facial expressions demonstrate that our proposal effectively improves the accuracy and efficiency of face expression recognition and achieve performance at extremely high speed.