Healthcare IoT-Based Affective State Mining Using a Deep Convolutional Neural Network

Abstract

Human effects are complex phenomena, which are studied for pervasive healthcare and well-being. The legacy pen and paper-based affective state determination methods are limited in their scientific explanation of causes and effects. Therefore, due to advances in intelligence technology, researchers are trying to apply some advanced artificial intelligence (AI) methods to realize individuals’ affective states. To recognize, realize, and predict a human’s affective state, domain experts have studied facial expressions, speeches, social posts, neuroimages, and physiological signals. However, with the advancement of the Internet of Medical Things (IoMT) and wearable computing technology, on-body non-invasive medical sensor observations are an effective source for studying users’ effects or emotions. Therefore, this paper proposes an IoMT-based emotion recognition system for affective state mining. Human psychophysiological observations are collected through electromyography (EMG), electro-dermal activity (EDA), and electrocardiogram (ECG) medical sensors and analyzed through a deep convolutional neural network (CNN) to determine the covert affective state. According to Russell’s circumplex model of effects, the five basic emotional states, i.e., happy, relaxed, disgust, sad, and neutral, are considered for affective state mining. An experimental study is performed, and a benchmark dataset is used to analyze the performance of the proposed method. The higher classification accuracy of the primary affective states has justified the performance of the proposed method.