Facial expression recognition using depth information and spatiotemporal features

Abstract

Facial expression recognition is considered to be one of the very important topics in image processing, pattern recognition, and computer vision due to its broad applications such as human-computer interaction, behavior analysis, and image understanding. In this work, a novel approach is proposed to recognize some facial expressions from time-sequential depth videos. First of all, efficient Local Directional Pattern (LDP) features are obtained from the time-sequential depth faces that are further augmented with the optical flow features. The augmented features are classified by Generalized Discriminant Analysis (GDA) to make the features more robust and finally, the spatiotemporal features are fed into Hidden Markov Models (HMMs) to train and recognize different facial expressions successfully. The depth information-based proposed facial expression recognition approach is compared to the conventional approaches where the proposed one outperforms others.