A hybrid transposed attention based deep learning model for wearable and explainable stress recognition

Abstract

Stress is a prevalent issue in modern society which affects a large percentage of the population. Stress has negative effects on human daily lives such as reduced memory and concentration levels, forgetfulness, restlessness, and weakness. Although it is challenging to extinguish stress from human lives, monitoring and controlling its consequences can be implemented. Current methods for stress monitoring often lack interpretability of results and making it difficult to provide an interpretable system for immediate mental health counseling and intervention. Wearable devices are of growing interest due to the impact of the COVID-19 pandemic on mental health, as wearables provide monitoring of physiological parameters which helps in measuring stress levels. Here, an investigation on the feasibility of using wrist and chest-based multimodal physiological signals collected from wearables is executed for stress classification and identification of underlying modalities that impact the stress. Multimodalities from wrist and chest sensors are used to examine the three-class classification, baseline, stress and amusement. For improved and visualized effects, the multimodal signals from wearables were transformed into images. Then, a hybrid model was introduced, which comprised of attentive convolution neural network (CNN), transposed attentive CNN connections with long-short term memory (LSTM) and followed by attention layer for efficient feature extraction and improved classification accuracy. Results found that the proposed model with images provided 97 % accuracy and 96 % F1-score for chest wearable and 94 % accuracy and 93 % F1-score for wrist wearable data. Additionally, the post-hoc explainability approach (Local interpretable model agnostic applications, LIME) provided visual representations of contributing features from each signal for each class. LIME shows electrodermal activity (EDA), temperature (TEMP) and respiration (RESP) to be the significant factor in stress recognition from chest wearable and blood volume pulse (BVP) and EDA from wrist wearable. Additionally, increasing the number of features of the explainable model influences the modalities influence on the model explainability. The results establish a benchmark for explainable stress recognition using different sensor data.