Sensor-based Human Activity Recognition Systems Research Articles

Resource-efficient, sensor-based human activity recognition with lightweight deep models boosted with attention

With their automatic feature extraction capabilities, deep learning models have become more widespread in sensor-based human activity recognition, particularly on larger datasets. However, their direct use on mobile and wearable devices is challenging due to the extensive resource requirements. Concurrently, attention-based models are emerging to improve recognition performance by dynamically emphasizing relevant parts of features and disregarding the irrelevant ones, particularly in the computer vision domain. This study introduces a novel application of attention mechanisms to smaller deep architectures, investigating whether smaller models can achieve comparable recognition performance to larger models in sensor-based human activity recognition systems while keeping resource usage at lower levels. For this purpose, we integrate the convolutional block attention module into a hybrid model, deep convolutional and long short-term memory network. Experiments are conducted using five public datasets in three model sizes: lightweight, moderate and original. The results show that applying attention to the lightweight model enables achieving similar recognition performances to the moderate-size model, and the lightweight model requires approximately 2–13 times fewer parameters and 3.5 times fewer flops. We also conduct experiments with sensor data at lower sampling rates and from fewer sensors attached to different body parts. The results show that attention improves recognition performance under lower sampling rates, as well as under higher sampling rates when model sizes are smaller, and mitigates the impact of missing data from one or more body parts, making the model more suitable for real-world sensor-based applications.

In-depth analysis of design & development for sensor-based human activity recognition system

In-depth analysis of design & development for sensor-based human activity recognition system

Wearable Sensor-Based Human Activity Recognition System Employing Bi-LSTM Algorithm

AbstractHuman activity recognition (HAR) systems employing wearable sensors are a promising area of research for tracking human activity. Recently, wearable devices such as smartwatches and sensors have been developed for activity recognition and monitoring. These systems aim to obtain the subject’s state within his or her environment by exploiting heterogeneous sensors attached to the body. With the development of deep learning, new strategies have emerged to facilitate and solve the HAR problems. In this work, a deep multilayer bidirectional long-short memory (Bi-LSTM) architecture has been implemented to detect human activities. Instead of training a single model as in traditional LSTM methods, two models are presented in the Bi-LSTM scheme, one for learning the input data sequence and the other for learning the reverse sequence. Finally, a new novel postprocessing approach has been proposed based on windowing and voting in the last step to improve the average F1 score. Comprehensive investigations on the three publicly available datasets consisting of a different set of activities were used to evaluate the performance of the proposed framework. The empirical results of this paper on AReM, Mhealth and PAMAP2 datasets attained 95.46, 95.79 and 93.41% average F1 score, respectively. The results also revealed that selecting the window size and implementing the appropriate voting method has a significant effect on improving the average percentage of the F1 score.

HIT HAR: Human Image Threshing Machine for Human Activity Recognition Using Deep Learning Models.

In recent days, research in human activity recognition (HAR) has played a significant role in healthcare systems. The accurate activity classification results from the HAR enhance the performance of the healthcare system with broad applications. HAR results are useful in monitoring a person's health, and the system predicts abnormal activities based on user movements. The HAR system's abnormal activity predictions provide better healthcare monitoring and reduce users' health issues. The conventional HAR systems use wearable sensors, such as inertial measurement unit (IMU) and stretch sensors for activity recognition. These approaches show remarkable performances to the user's basic activities such as sitting, standing, and walking. However, when the user performs complex activities, such as running, jumping, and lying, the sensor-based HAR systems have a higher degree of misclassification results due to the reading errors from sensors. These sensor errors reduce the overall performance of the HAR system with the worst classification results. Similarly, radiofrequency or vision-based HAR systems are not free from classification errors when used in real time. In this paper, we address some of the existing challenges of HAR systems by proposing a human image threshing (HIT) machine-based HAR system that uses an image dataset from a smartphone camera for activity recognition. The HIT machine effectively uses a mask region-based convolutional neural network (R-CNN) for human body detection, a facial image threshing machine (FIT) for image cropping and resizing, and a deep learning model for activity classification. We demonstrated the effectiveness of our proposed HIT machine-based HAR system through extensive experiments and results. The proposed HIT machine achieved 98.53% accuracy when the ResNet architecture was used as its deep learning model.

CNN-LSTM-Based Late Sensor Fusion for Human Activity Recognition in Big Data Networks

The technological advancement in sensor technology and pervasive computing has brought smart devices into our daily life. Due to the continuous connectivity of the internet with our everyday devices, researchers can deploy IoT sensors to health care and other applications, such as human activity recognition. Most of the state-of-the-art sensor-based human activity recognition systems can detect basic activities (such as standing, sitting, and walking), but they cannot accurately distinguish similar activities (ascending stairs or descending stairs). Such systems are not efficient for critical healthcare applications having complex activity sets. This paper proposes two sensor fusion approaches, i.e., position-based early and late sensor fusion using convolutional neural network (CNN) and convolutional long-short-term memory (CNN-LSTM). The performance of our proposed models is evaluated on two publicly available datasets. We also evaluated the effect of different normalization techniques on recognition accuracy. Our results show that the CNN-LSTM-based late sensor fusion model also improves the recognition accuracy of similar activities.

A new approach for physical human activity recognition from sensor signals based on motif patterns and long-short term memory

Numerous studies have been carried out in recent years on the recognition, tracking, and discrimination of human activities. Automatic recognition of physical activities is often referred to as human activity recognition (HAR). There are generally vision-based and sensor-based approaches for activity recognition. The computer vision-based approach generally works well in laboratory conditions, but it can fail in real-world problems due to clutter, variable light intensity, and contrast. Sensor-based HAR systems are realized by continuously monitoring and analyzing physiological signals measured from heterogeneous sensors connected to the person's body. In this study, the Motif Patterns (MP) approach, which extracts features from sensor signals, is proposed for HAR. The success of the HAR systems depends on the effectiveness of the features extracted from the signals. The LSTM network is a special kind of recurrent neural network that has been used to make very successful predictions on time series data where long-term dependencies are. The LSTM network type offers a successful solution approach to solving long-term dependencies problems such as human activity recognition. The classification process was carried out with Long-Short Term Memory (LSTM) using MP features extracted from accelerometer, gyroscope, and magnetometer sensor signals. A large dataset of 9120 signals was used to test the proposed approach. A high success rate of 98.42 % was achieved with the proposed MP + LSTM method. As a result, it has been seen that the proposed approach has been obtained with a high success rate for HAR using sensor signals.

A Survey of Machine Learning and Meta-heuristics Approaches for Sensor-based Human Activity Recognition Systems

A Survey of Machine Learning and Meta-heuristics Approaches for Sensor-based Human Activity Recognition Systems

Sensor-based human activity recognition system with a multilayered model using time series shapelets

Human activity recognition can be exploited to benefit ubiquitous applications using sensors. Current research on sensor-based activity recognition is mainly using data-driven or knowledge-driven approaches. In terms of complex activity recognition, most data-driven approaches suffer from portability, extensibility and interpretability problems, whilst knowledge-driven approaches are often weak in handling intricate temporal data. To address these issues, we exploit time series shapelets for complex human activity recognition. In this paper, we first describe the association between activity and time series transformed from sensor data. Then, we present a recursively defined multilayered activity model to represent four types of activities and employ a shapelet-based framework to recognize various activities represented in the model. A prototype system was implemented to evaluate our approach on two public datasets. We also conducted two real-world case studies for system evaluation: daily living activity recognition and basketball play activity recognition. The experimental results show that our approach is capable of handling complex activity effectively. The results are interpretable and accurate, and our approach is fast and energy-efficient in real-time.