MultiCNN-FilterLSTM: Resource-efficient sensor-based human activity recognition in IoT applications

Abstract

With the recent advances in the Internet of Things (IoT) technologies, various human-centered applications have proliferated and improved the quality of users’ life. In the meantime, human activity recognition (HAR) has been considered as an essential component of human-centered applications in IoT due to its capability of providing substantial information about the user states. Whereas deep learning-based HAR methods have been recently proposed, there are still rooms to improve the HAR models, particularly from the perspective of IoT, as the models need to be precise but resource efficient. In this paper, to support IoT systems that require a resource-efficient model, we thereby propose a deep learning-based HAR model, called MultiCNN-FilterLSTM that combines a multihead convolutional neural network (CNN) with a long–short-term memory (LSTM) through a residual connection in which feature vectors are efficiently processed in hierarchical order. Accordingly, a novel approach to using LSTM cells, which we call filterwise LSTM (FilterLSTM), is proposed whereby the HAR model can learn the dependencies among the features at different hierarchical levels. The proposed HAR model has been exhaustively evaluated on two publicly available datasets. The proposed HAR model enhances the classification accuracy by 2.3%–4.4%, while it requires 21%–70% fewer operations than the state-of-the-art models. In addition, the proposed model is deployed to a Raspberry Pi 4 for further analysis in terms of deployment. The experimental results are presented to verify the merits of the proposed HAR model compared to the state-of-art models by exploiting hierarchical relationships and to deliver significant insights on the effectiveness of the proposed FilterLSTM from the perspective of the IoT system.