Transformer-based deep reverse attention network for multi-sensory human activity recognition

Abstract

In today’s era, one of the important applications of Artificial Intelligence (AI) is Human Activity Recognition (HAR). It has a wide range of applicability in health monitoring for patients with chronic diseases, gaming consoles for gesture recognition, etc. Sensor-based HAR systems use signals collected over a period of time to label an activity. When we design an efficient sensor-based HAR system, a model requires learning an optimal association of spatial and temporal features. In this article, we propose a sensor-based HAR technique using the deep learning approach. We present a deep reverse transformer-based attention mechanism to guide the side residual features Unlike the conventional bottom-up approaches for feature fusion, we exploit a top-down feature fusion approach. The reverse attention is self-calibrated throughout the course of learning, which regularizes the attention modules and dynamically adjusts the learning rate. The overall framework outperforms several state-of-the-art methods and is shown to be statistically significant against these methods on five publicly available sensor-based HAR datasets, namely, MHEALTH, USC-HAD, WHARF, UTD-MHAD1, and UTD-MHAD2. Further, we conduct an ablation study to showcase the importance of each of the components of the proposed framework. Source code of this work is available at https://github.com/rishavpramanik/RevTransformerAttentionHAR.