Automated Multimodal Stress Detection in Computer Office Workspace

Abstract

Nowadays, changes in the conditions and nature of the workplace make it imperative to create unobtrusive systems for the automatic detection of occupational stress, which can be feasibly addressed through the adoption of Internet of Things (IoT) technologies and advances in data analysis. This paper presents the development of a multimodal automated stress detection system in an office environment that utilizes measurements derived from individuals’ interactions with the computer and its peripheral units. In our analysis, behavioral parameters of computer keyboard and mouse dynamics are combined with physiological parameters recorded by sensors embedded in a custom-made smart computer mouse device. To validate the system, we designed and implemented an experimental protocol simulating an office environment and included the most known work stressors. We applied known classifiers and different data labeling methods to the physiological and behavioral parameters extracted from the collected data, resulting in high-performance metrics. The feature-level fusion analysis of physiological and behavioral parameters successfully detected stress with an accuracy of 90.06% and F1 score of 0.90. The decision-level fusion analysis, combining the features extracted from both the computer mouse and keyboard, showed an average accuracy of 66% and an average F1 score of 0.56.