Multi-modal Emotion Recognition Using Neural Networks: Deploy on Cloud

Abstract

This paper explores recent developments in emotion recognition, focusing on methodologies employing Multilayer Perceptron (MLP) classifiers. Emotion recognition is crucial for human-machine interaction across various domains, including healthcare, surveillance systems, and intelligent learning environments. There are many deep learning models that have been employed for emotion detection, this emphasizes the effectiveness of MLP classifiers. The project at hand centers on predicting emotions such as happiness, calmness, fear, and disgust solely utilizing MLP classifiers. The system is trained on audio data, with recordings matched in frequency, time, and speed to the raw data used for training. Additionally, Convolutional Neural Network (CNN) models are used here to analyze facial expressions in real-time video streams by providing neat and complete approach to emotion recognition. Upon deployment on the AWS cloud platform, the system provides real-time emotion predictions in textual format via a web interface created using Flask. Furthermore, the system offers the capability to analyze emotions from live video streams, presenting continuous feedback on emotions through a face-outlined box updated every three seconds. This functionality serves to confirm individuals' emotions and provide self-awareness regarding their emotional states. This project focuses on giving efficiency by using MLP classifiers in conjunction with CNN models for emotion recognition tasks, thereby contributing to the advancement of human-machine interaction. Key Words: MLP Classifier, CNN, AWS Cloud, Flask, emotion, audio, video