A Review of Local Feature Algorithms and Deep Learning Approaches in Facial Expression Recognition with Tensorflow and Keras

Abstract

In facial expression identification classification and lower processing times are key in choosing the algorithms to use in the facial detection, preprocessing, feature extraction or classification step. Facial expression recognition is based on deep learning, feature and holistic algorithms. Feature based algorithms like local binary patterns, local directional patterns (LDP) extract features from various facial components like nose, mouth or ears into a histogram. Deep learning involves using convolutional neural networks for image analysis with several hidden layers as opposed to artificial neural or shallow networks. The most popular models are AlexNet, VGG-Face and GoogleNet. The study evaluates computational accuracy and efficiency of deep learning algorithms and compares them to local feature based algorithms. The FER2013, Yale Faces, AT&T Database of Faces, JAFFE and CK+ datasets were used for analysis. Popular frameworks deep learning frameworks called Keras and Tensorflow backends are used to classify data and give better accuracy than a variant of local binary patterns. The processing time is shorter for feature based algorithms than the deep learning algorithms. To improve time on the deep learning approaches the study used pre-trained models to achieve greater accuracy with low execution times as well. A combination of preprocessed multi block binary patterns, PCA, multilayer perceptron, support vector machines and extra trees classifier gave competitive results to the superior established convolutional network for small datasets within a percentage range. Preprocessing used canny edge detection and histogram equalization.