Deep Learning based Emotion Recognition IoT System

Abstract

Recently, IoT systems are wildly used in various fields, and received the most attention in the research field of healthcare system. This study aims to realize an emotion recognition IoT system for helping check the people feeling and health status. In edge side, the designed system applies a motion sensor, and a RGB camera which were equipped on a compact edge device, Raspberry Pi, for detecting human motion and take the people image. When motion sensor detect motion, the RGB camera is started and image of the people is taken. Then, the face area in the image are cropped from the people image by Haar feature-based cascade classifiers. When the face is detected, deep learning model of MobileNet is applied for human identification, and the face image is sent to server for emotion recognition by deep learning. In terms of human identification models, MobileNet, which was equipped on the Raspberry Pi, has been trained previously on a GPU machine. Emotion recognition deep learning model is also trained by GPU machine and equipped on a server machine. The MobileNet is also selected as emotion recognition deep learning model, according the compassion of nine state-of-the-art deep learning models. In the experimental results, the motion sensor and RGB camera work well, and the human identification accuracy almost achieves 100%. Furthermore, face images are transferred from edge to cloud correctly, and emotion recognition also achieves a better accuracy. The experimental results also proved the effectiveness of proposed system.