Facial Emotion Recognition Based on Eye and Mouth Regions

Abstract

Machine analysis of facial emotion recognition is a challenging and an innovative research topic in human-computer intelligent interaction nowadays. The eye and the mouth regions are most essential components for facial emotion recognition. Most of the existing approaches have not utilized the eye and the mouth regions for high recognition rate. This paper proposes an approach to overcome this limitation using the eye and the mouth region-based emotion recognition using reinforced local binary patterns (LBP). The local features are extracted in each frame by using Gabor wavelet with selected scale and orientations. This feature is passed on to the ensemble classifier for detecting the location of the face region. From the signature of each pixel on the face, the eye and the mouth regions are detected using ensemble classifier. The eye and the mouth features are extracted using reinforced LBP. Multi-class Adaboost algorithm is used to select and classify these discriminative features for recognizing the emotion of the face. The developed methods are deployed on the RML, CK and FERA 2011 databases, and they exhibit significant performance improvement owing to their novel features when compared to the existing techniques.