Ensemble of Parametrized Quantum LSTM Neural Networks for Multimodal Stress Monitoring

Abstract

Stress is an unavoidable part of everyone’s life and its effective monitoring can prevent many health problems associated with it. Deep Learning provides effective solutions to continuous stress monitoring by analyzing data from multiple modalities. The aim of the paper is to develop a framework that utilizes the strengths of deep learning, quantum information processing, and multimodal data analysis providing an effective solution to continuous stress monitoring of knowledge workers. An ensemble framework for the quantum augmented deep LSTM model is proposed for the same. Parametrized quantum LSTM (PQLSTM) models are created by embedding quantum circuits into classical LSTM models. In our framework, each PQLSTM model is trained with different sensor data collected from the time series dataset. The SWELL-KW (Koldijk S, Sappelli M, Verberne S, et al. (2014) The swell knowledge work dataset for stress and User Modeling Research. Proceedings of the 16th International Conference on Multimodal Interaction. https://doi.org/10.1145/2663204.2663257 ) contains data from knowledge workers’ computer interactions, bodily postures, facial expressions, skin conductance, and heart rates (variability), recorded over time in a variety of working environments. A weighted averaging method based on the F1 score has been adapted to integrate the predictions by individual models. Experimental results show that the proposed method obtains a 90.6 F1 score. Ensembling the results of individual PQLSTMs on different modality data improved the overall prediction performance.