Facial expression recognition for low resolution images using convolutional neural networks and denoising techniques

Pavan Nageswar Reddy Bodavarapu,P V V S Srinivas

doi:10.17485/ijst/v14i12.14

Abstract

Background/Objectives: There is only limited research work is going on in the field of facial expression recognition on low resolution images. Mostly, all the images in the real world will be in low resolution and might also contain noise, so this study is to design a novel convolutional neural network model (FERConvNet), which can perform better on low resolution images. Methods: We proposed a model and then compared with state-of-art models on FER2013 dataset. There is no publicly available dataset, which contains low resolution images for facial expression recognition (Anger, Sad, Disgust, Happy, Surprise, Neutral, Fear), so we created a Low Resolution Facial Expression (LRFE) dataset, which contains more than 6000 images of seven types of facial expressions. The existing FER2013 dataset and LRFE dataset were used. These datasets were divided in the ratio 80:20 for training and testing and validation purpose. A HDM is proposed, which is a combination of Gaussian Filter, Bilateral Filter and Non local means denoising Filter. This hybrid denoising method helps us to increase the performance of the convolutional neural network. The proposed model was then compared with VGG16 and VGG19 models. Findings: The experimental results show that the proposed FERConvNet_HDM approach is effective than VGG16 and VGG19 in facial expression recognition on both FER2013 and LRFE dataset. The proposed FERConvNet_HDM approach achieved 85% accuracy on Fer2013 dataset, outperforming the VGG16 and VGG19 models, whose accuracies are 60% and 53% on Fer2013 dataset respectively. The same FERConvNet_HDM approach when applied on LRFE dataset achieved 95% accuracy. After analyzing the results, our FERConvNet_HDM approach performs better than VGG16 and VGG19 on both Fer2013 and LRFE dataset. Novelty/Applications: HDM with convolutional neural networks, helps in increasing the performance of convolutional neural networks in Facial expression recognition. Keywords: Facial expression recognition; facial emotion; convolutional neural network; deep learning; computer vision

Highlights

The raw data consists of noise like random variation of brightness or color information, removing noise from the images drastically improves the performance of the facial emotion recognition models
The results show that this approach performs better than convolutional neural networks based on micro-expression recognition(9)
low resolution facial expression (LRFE) dataset contains 6000 images of facial expression belonging to seven emotions (Happy, Sad, Surprise, Neutral, Fear, Disgust, Angry), which are collected from various sources

Summary

Introduction

The raw data consists of noise like random variation of brightness or color information, removing noise from the images drastically improves the performance of the facial emotion recognition models. To eliminate noise from images there are many denoising techniques such as gaussian blur, bilateral filter, non-local means filtering. A bilateral Filter decreases the noise by preserving the edges by replacing the intensity of pixels with a weighted average of intensity from surrounding pixels(2). Non Local Means Filtering averages neighbors with similar neighborhoods, with much greater clarity and smaller extent loss of detail post filtering. The limitation of this technique is, efficiency is slightly lower when compared to traditional techniques. To speed up the execution many techniques were designed, one such technique is the fast Fourier transform, it determines the similarity between two pixels by speeding up the algorithm by a factor of 50 and maintains the quality of the result(3)

Objectives

Methods

Results