Emotion Recognition from Facial Expressions Using Videos and Prototypical Network for Human–Computer Interaction

Abstract

Emotions play a vital part in a person’s day-to-day life. In this work, a method for emotion recognition from videos is proposed which uses prototypical network and Long Short-Term Memory (LSTM). The method is named Prototypical network and LSTM for Emotion Recognition in Videos (PLERV). The method first finds the classification of selected frames in a video, and this is achieved using the metric meta-learning approach prototypical network. A sequence of classification is generated from the selected frames in a video, which is in turn given to Long Short-Term Memory (LSTM) model. LSTM model will provide the emotion classification of the video. Use of meta-learning algorithm enabled better generalization with lesser samples. Experiments are done using the BU-4DFE dataset for the emotions such as Happy, Surprise, Angry, Fear, Sad, and Disgust which gave an accuracy of 89%.