Temporal-channel convolution with self-attention network for human activity recognition using wearable sensors

Abstract

Human activity recognition (HAR) is an essential task in many applications such as health monitoring, rehabilitation, and sports training. Sensor-based HAR has received increasing attention due to the widespread availability of sensors in daily life. In this paper, we propose two novel architectures, the convolution with self-attention network (CSNet) and the temporal-channel convolution with self-attention network (TCCSNet), for classifying sequences of human activity data from different sensors. CSNet leverages both convolution and self-attention to capture both local and global dependencies in the input data, while TCCSNet exploits both temporal and inter-channel dependencies through two branches of convolutions and self-attentions for extracting time-wise and channel-wise information. The proposed methods are evaluated on seven different sensor-based HAR datasets, namely: MHEALTH, PAMAP2, UTD1, UTD2, WHARF, USC-HAD, and WISDM, using the leave-one-subject-out cross-validation protocol. Our experiments show that the proposed models outperform other modern approaches, such as Transformers and long short-term memory (LSTM) based models.