Hybrid Deep Neural Network for Facial Expressions Recognition

Wijdan Rashid Abdulhussien,Nidhal K El Abbadi,Abdul M Gaber

doi:10.52549/ijeei.v9i4.3425

Abstract

Facial expressions are critical indicators of human emotions where recognizing facial expressions has captured the attention of many academics, and recognition of expressions in natural situations remains a challenge due to differences in head position, occlusion, and illumination. Several studies have focused on recognizing emotions from frontal images only, while in this paper wild images from the FER2013 dataset have been used to make a more generalizing model with the existence of its challenges, it is among the most difficult datasets that only got 65.5 % accuracy human-level. This paper proposed a model for recognizing facial expressions using pre-trained deep convolutional neural networks and the technique of transfer learning. this hybrid model used a combination of two pre-trained deep convolutional neural networks, training the model in multiple cases for more efficiency to categorize the facial expressions into seven classes. The results show that the best accuracy of the suggested models is 74.39% for the hybrid model, and 73.33% for Fine-tuned the single EfficientNetB0 model, while the highest accuracy for previous methods was 73.28%. Thus, the hybrid and single models outperform other state of art classification methods without using any additional, the hybrid and single models ranked in the first and second position among these methods. Also, The hybrid model has even outperformed the second-highest in accuracy method which used extra data. The incorrectly labeled images in the dataset unfairly reduce accuracy but our best model recognized their actual classes correctly.

Highlights

One of the strongest, natural, and global signals that reflect human emotions and state is facial expression
THE EXPERIMENTAL RESULTS The parameters that used for training all models are listed in Table 1, the following subsections explain the results obtained from training two pre-trained models separately, and that get from the proposed combining models trained in two cases
The pre-trained models used in this work are EfficientNetB0 and MobileNetV2 models, the single pre-trained models are trained by using a new classifier and fine-tuning the features extraction part.the proposed hybrid model contains a mix of two single pre-trained models which are EfficientNetB0 and MobileNetV2 models to categorize facial expression into seven categories

Summary

Introduction

Natural, and global signals that reflect human emotions and state is facial expression. Machines can give customized services based on human reaction data Many systems, such as personalized suggestions, virtual reality, customer satisfaction, etc., rely on the ability to identify facial expressions quickly and accurately.In recent years, due to the rapid growth in artificial intelligence, automatic facial expression recognition (FER) has an increasing interest among researchers in the field of computer vision, psychology, and pattern recognition [1], facial expression's classification has become a fundamental portion of computer systems and the quick interaction between humans and computers. Many other parameters have a significant effect on the recognition performance, some of them are a variation of personal attributes that can cause high inter-subject variations, in addition to age, gender, ethnic backgrounds, and pose variation [4]

Results

Discussion

Conclusion