Facial expression recognition with fused handcraft features based on pixel difference local directional number pattern

Abstract

Facial expression recognition (FER) has been an active research area in recent years, which plays a vital role in national security and human-computer interaction. Due to the lacking of sufficient expression features and facial images, it is challenging to automatically recognize facial expression with high accuracy. In this paper, we propose a fusion handcraft feature method to improve FER from images. Firstly, a new texture feature extraction method PD-LDN (Pixel Difference Local Directional Number pattern) is proposed, which can extract more local information, reduce noise disturbance and feature dimension. Secondly, the handcrafted features including PD-LDN texture features, geometric features, and BOVW (Bag of Visual Words) semantic features are connected in parallel to an improved autoencoder network for fusion. Finally, the fused features are input into the softmax classifier for recognizing facial expression. We conduct extensive experiments on JAFFE and CK+datasets. Our proposed method shows superior performance than the state-of-the-art approaches on recognizing facial expressions.