Facial expression recognition using HOG and LBP features with convolutional neural network

Abstract

In computer vision, automatic facial expression recognition (FER) continued a difficult and interesting topic. The majority of extant techniques are based on traditional features descriptors such as local binary pattern (LBP) and histogram of oriented gradient (HOG), in which the classifier's hyperparameters are tailored to produce the best recognition accuracies across a single database or a small set of similar databases. This paper integrates the power of deep learning techniques with the LBP and HOG. The LBP and HOG are estimated from each image in the dataset. The resulting dataset is applied to a convolutional neural network (CNN). The architecture of this CNN constitutes three convolutional layers and three max-pooling layers. The output layers involve BatchNormalization, three dense layers, and two dropout layers. The proposed architecture is validated on the extended cohn-kanade dataset (CK+). We obtain improvement in the accuracy of the CNN model from 0.9593 to 0.967 and 0.975 after using the LBP and HOG respectively.