Abstract

Facial Expression Recognition (FER) is a challenging task that improves natural human-computer interaction. This paper focuses on automatic FER on a single in-the-wild (ITW) image. ITW images suffer real problems of pose, direction, and input resolution. In this study, we propose a pyramid with super-resolution (PSR) network architecture to solve the ITW FER task. We also introduce a prior distribution label smoothing (PDLS) loss function that applies the additional prior knowledge of the confusion about each expression in the FER task. Experiments on the three most popular ITW FER datasets showed that our approach outperforms all the state-of-the-art methods.

Highlights

  • Non-verbal communication plays an essential role in person-person communication

  • EXPERIMENTAL RESULTS We report the experimental results for the Real-world Affective Faces Database (RAF-DB), facial expression recognition (FER)+, and AffectNet datasets

  • On the RAF-DB dataset, our accuracy is better by about 2% in weighted accuracy (WA) metric and 4.05% in unweighted accuracy (UA) metric, compared to the state-of-the-art results

Read more

Summary

INTRODUCTION

Non-verbal communication plays an essential role in person-person communication. These non-verbal signals can add clues, additional information, and meaning to spoken (verbal) communication. The network trained with this input size works poorly with the same images but on a different scale. Dong et al introduced the Super-Resolution Convolutional Neural Network (SRCNN), a deep CNN model that works on low-resolution and high-resolution feature maps and generated a high-resolution image [23]. We propose a Pyramid with Super-Resolution (PSR) network architecture to deal with the different-image-size problem for the ITW FER task. The main idea of this block is to view the input image on different scale from small to large. By learning how to resample the image size, we assume that this block can add useful information to this particular task, and thereby increases the accuracy of the prediction model

LOW AND HIGH-LEVEL FEATURE EXTRACTOR
FULLY CONNECTED BLOCK AND CONCATENATION BLOCK
EXPERIMENTS AND RESULTS
CONCLUSION

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call

Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.