A Multitask Deep Learning Approach for Sensor-Based Human Activity Recognition and Segmentation

Abstract

Deep learning for sensor-based human activity recognition (HAR) has been a focus of research in recent years. Sensor data stream segmentation is a core element in HAR, which has currently been treated as an independent preprocessing task, usually with a fixed-size window. This has led to two critical problems, namely the multi-class window problem caused by possible multiple activities within a fixed-size window and the fluctuation of prediction results due to noisy data and over-segmentation. To address these research challenges, in this paper, we conceive a novel Multi-Task deep learning approach to segmenting and recognizing human activity simultaneously. Specifically, we propose a multi-scale window method based on feature sequence generation to overcome the multi-class window problem. We develop a novel boundary offset prediction algorithm to adjust a windows boundary to tackle the over-segmentation issue. In addition, We design a multi-task framework to streamline and optimize the activity recognition and segmentation tasks simultaneously. We conduct extensive experiments on eight benchmark datasets to evaluate the proposed framework and associated methods. Initial results show that our approach outperforms the performance of current state-of-the-art HAR methods.