A Deep Learning Approach for Human Activities Recognition From Multimodal Sensing Devices

Isibor Kennedy Ihianle,Richard I Otuka,Solomon Henry Ebenuwa,Kayode Owa,Mobolaji O Orisatoki,Augustine O Nwajana

doi:10.1109/access.2020.3027979

Abstract

Research in the recognition of human activities of daily living has significantly improved using deep learning techniques. Traditional human activity recognition techniques often use handcrafted features from heuristic processes from single sensing modality. The development of deep learning techniques has addressed most of these problems by the automatic feature extraction from multimodal sensing devices to recognise activities accurately. In this paper, we propose a deep learning multi-channel architecture using a combination of convolutional neural network (CNN) and Bidirectional long short-term memory (BLSTM). The advantage of this model is that the CNN layers perform direct mapping and abstract representation of raw sensor inputs for feature extraction at different resolutions. The BLSTM layer takes full advantage of the forward and backward sequences to improve the extracted features for activity recognition significantly. We evaluate the proposed model on two publicly available datasets. The experimental results show that the proposed model performed considerably better than our baseline models and other models using the same datasets. It also demonstrates the suitability of the proposed model on multimodal sensing devices for enhanced human activity recognition.

Highlights

Human activity recognition (HAR) is a process aimed at recognising what an individual is doing, for example, sleeping, showering, and cooking and the context in which they occur
EXPERIMENTS We considered two datasets on HAR to evaluate the performance of the proposed multi-channel CNN Bidirectional LSTM (MCBLSTM) model
We compared the performance of the proposed model with convolutional neural network (CNN) and Bidirectional Long ShortTerm Memory (LSTM) models, which provided baseline references and with the results reported by other authors on these datasets

Summary

INTRODUCTION

Human activity recognition (HAR) is a process aimed at recognising what an individual is doing, for example, sleeping, showering, and cooking and the context in which they occur. The main advantage of the proposed model is that the CNN layers can achieve activity recognition of multiple and complex activities by the direct mapping and abstract representation of raw sensor inputs for feature extraction at different resolutions of the channels. We propose a new deep learning architecture for multiple human activity recognition using combined CNN and recurrent Bidirectional LSTM networks capable of feature extraction and the utilization of temporal dependencies. VOLUME 8, 2020 multiple human activities irrespective of the variability of the body movements. We evaluate the proposed model using publicly available datasets and show that it outperforms other deep learning models based on the published results and our baseline deep learning models

RELATED WORK

EXPERIMENTS

PERFORMANCE EVALUATION

Findings

CONCLUSION