Abstract
Humans always engage multiple modalities when performing tasks, such as eye activity, speech and head movement, which contain rich information indicative of task load that can help understand and predict human psychological state and behavior. In recent research into multimodal signal processing, the ideas of sequence- and coordination-based event features have been proposed, which explicitly utilize the interaction information among different modalities. In this paper, we propose event intensity and event duration-based features, which capture the extent and duration of onset events that denote major changes in behavior signal. These features are combined with sequence- and coordination-based event features to achieve state-of-the-art performance in assessing task load levels and load types. In experimental work, we collected eye activity, speech and head movement data from 24 participants during cognitive, perceptual, physical and communication tasks. Results suggest that by fusing these four compact, interpretable event-based features, strong accuracy can be achieved: 84% for two load level classification, 89% for four load type classification and 76% for 8-class classification, outperforming conventional statistical features and deep neural network self-learned features by up to 9% and 25% respectively. These features do not need to be selected during training and can generalize well for different participants and different task types.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.