Facial expression recognition from image based on hybrid features understanding

Abstract

Facial expression recognition (FER) plays an important role in the applications of human computer interaction. Given the wide use of convolutional neural networks (CNNs) in automatic video and image classification systems, higher-level features can be automatically learned from hierarchical neural networks with big data. However, learning CNNs require large amount of training data for adequate generalization, while the Scale-invariant feature transform (SIFT) does not need large training samples to generate useful feature. In this paper, we propose a new hybrid feature representation for the recognition of facial expressions from a single image frame that uses a combination of SIFT and deep-learning feature of different level extracted from the CNN model, then adopt the combined features and classify the expression by support vector machines (SVM). The performance of the proposed method has been validated on public CK+ databases. To evaluate the generalization ability of our method, we also performed an experiment on a cross-database environment. Experimental results show that the proposed approach can achieve better classification rates compared with state-of-art CNN methods, which indicate the considerable potential of combining shallow feature with deep-learning feature.