Facial Expression Recognition using Local Directional Pattern variants and Deep Learning

Abstract

Automated facial expressions has been used with success in medical, industrial security, gaming and aviation security as well as marketing systems. The study compares and analyses synergy of a Local Binary Pattern variant and Convolutional Neural Networks (CNNs / ConvNets) in facial expression recognition. Major emotional behavioural states include fear, anger, neutrality, happiness and sadness. Local Directional Patterns are used in facial edge detection on local features in grey scales. The study applies LDP feature extraction and uses deep learning CNN algorithms to recognise facial expressions of targeted facial databases. The study uses Convolutional Neural Networks (CNNs / ConvNets) on a dataset already trained by LDP Feature Extractor. Local Directional Pattern algorithm is based on edge detection Kirsh Algorithm. The CK+ and Googleset facial expression databases are used in this study. Convolutional Neural Networks used the extracted feature histograms for training. Performance accuracy is used as measure of the study. A hybrid of Local Directional Patterns, local binary pattern variants and an ensemble voting classifier gave an accuracy which was within one percentage point less than convolutional neural networks alone with very quick processing times of sub minute. A hybrid of feature extraction using LDP and deep learning CNN(LDGPNet) algorithm's accuracy was less than 1 percentage point better than convolutional neural networks alone albeit with quicker processing time. For modest and higher budgets, the study recommends LDGPNet using the Local Directional Pattern feature extractor, Gabor Filters and Convolutional Neural Networks. The implementation resulted in reduced processing time, improved edge detection and slightly higher accuracy to Convolutional Neural Networks. For less budgets, the study recommends the local directional pattern, local binary pattern and ensemble voting classifier hybrid oering fastest processing time, and slightly less accuracy times within 1 to 2 percentage points of convolutional neural networks and LDGBNet.