Sensor-based Human Activity Recognition Research Articles

Belief-Rule-Based System with Self-organizing and Multi-temporal Modeling for Sensor-based Human Activity Recognition.

Smart environment is an efficient and cost- effective way to afford intelligent supports for the elderly people. Human activity recognition (HAR) is a crucial aspect of the research field of smart environments, and it has attracted widespread attention lately. The goal of this study is to develop an effective sensor-based HAR model based on the belief-rule-based system (BRBS), which is one of representative rule-based expert systems. Specially, a new belief rule base (BRB) modeling approach is proposed by taking into account the self- organizing rule generation method and the multi-temporal rule representation scheme, in order to address the problem of combination explosion that existed in the traditional BRB modelling procedure and the time correlation found in continuous sensor data in chronological order. The new BRB modeling approach is so called self-organizing and multi-temporal BRB (SOMT-BRB) modeling procedure. A case study is further deducted to validate the effectiveness of the SOMT-BRB modeling procedure. By comparing with some conventional BRBSs and classical activity recognition models, the results show a significant improvement of the BRBS in terms of the number of belief rules, modelling efficiency, and activity recognition accuracy.

Deep Learning and Recurrent Signature Based Classification for Sensor-Based HAR: Addressing Explainability and Complexity in 5G Networks

When it comes to clinical applications, sensor-based human activity recognition (HAR) is invaluable, and numerous machine learning algorithms have effectively used to obtain excellent presentation. Using a variety of on-body sensors, these systems attempt to ascertain the subject's status relative to their immediate surroundings. There was a time when feature extraction was done by hand, but now more and more people are using Artificial Neural Networks (ANNs). A number of innovative approaches to HAR have surfaced since the advent of deep learning. Problems arise, however, for sensor-based HAR classification algorithms in today's communication networks. Among these, you can find solutions to problems like deal with complicated and large-scale data signals, extract characteristics from complicated datasets, and meet explainability standards. For complicated 5G networks, these difficulties become even more apparent. In particular, explainability is now critical for the broad use of sensor-based HAR in 5G networks and beyond. The research suggests a classification approach based on path signatures, recurrent signature (ReS), to address these issues. This cutting-edge model employs deep-learning (DL) approaches to circumvent the tedious feature selection challenge. Furthermore, the study investigates how to improve the ReS model's classification accuracy by using graph-based optimisation methods. To test how well the suggested framework worked, to dug deep into the publicly available dataset, which included a separate set of tasks. The paper's empirical results on AReM datasets achieved an average accuracy of 96%.

A Novel Active Learning Framework for Cross-Subject Human Activity Recognition from Surface Electromyography.

Wearable sensor-based human activity recognition (HAR) methods hold considerable promise for upper-level control in exoskeleton systems. However, such methods tend to overlook the critical role of data quality and still encounter challenges in cross-subject adaptation. To address this, we propose an active learning framework that integrates the relation network architecture with data sampling techniques. Initially, target data are used to fine tune two auxiliary classifiers of the pre-trained model, thereby establishing subject-specific classification boundaries. Subsequently, we assess the significance of the target data based on classifier discrepancy and partition the data into sample and template sets. Finally, the sampled data and a category clustering algorithm are employed to tune model parameters and optimize template data distribution, respectively. This approach facilitates the adaptation of the model to the target subject, enhancing both accuracy and generalizability. To evaluate the effectiveness of the proposed adaptation framework, we conducted evaluation experiments on a public dataset and a self-constructed electromyography (EMG) dataset. Experimental results demonstrate that our method outperforms the compared methods across all three statistical metrics. Furthermore, ablation experiments highlight the necessity of data screening. Our work underscores the practical feasibility of implementing user-independent HAR methods in exoskeleton control systems.

Hand-Crafted Features With a Simple Deep Learning Architecture for Sensor-Based Human Activity Recognition

Hand-Crafted Features With a Simple Deep Learning Architecture for Sensor-Based Human Activity Recognition

Multihead-Res-SE Residual Network with Attention for Human Activity Recognition

Human activity recognition (HAR) typically uses wearable sensors to identify and analyze the time-series data they collect, enabling recognition of specific actions. As such, HAR is increasingly applied in human–computer interaction, healthcare, and other fields, making accurate and efficient recognition of various human activities. In recent years, deep learning methods have been extensively applied in sensor-based HAR, yielding remarkable results. However, complex HAR research, which involves specific human behaviors in varied contexts, still faces several challenges. To solve these problems, we propose a multi-head neural network based on the attention mechanism. This framework contains three convolutional heads, with each head designed using one-dimensional CNN to extract features from sensory data. The model uses a channel attention module (squeeze–excitation module) to enhance the representational capabilities of convolutional neural networks. We conducted experiments on two publicly available benchmark datasets, UCI-HAR and WISDM, to evaluate our model. The results were satisfactory, with overall recognition accuracies of 96.72% and 97.73% on their respective datasets. The experimental results demonstrate the effectiveness of the network structure for the HAR, which ensures a higher level of accuracy.

Large Language Models for Wearable Sensor-Based Human Activity Recognition, Health Monitoring, and Behavioral Modeling: A Survey of Early Trends, Datasets, and Challenges.

The proliferation of wearable technology enables the generation of vast amounts of sensor data, offering significant opportunities for advancements in health monitoring, activity recognition, and personalized medicine. However, the complexity and volume of these data present substantial challenges in data modeling and analysis, which have been addressed with approaches spanning time series modeling to deep learning techniques. The latest frontier in this domain is the adoption of large language models (LLMs), such as GPT-4 and Llama, for data analysis, modeling, understanding, and human behavior monitoring through the lens of wearable sensor data. This survey explores the current trends and challenges in applying LLMs for sensor-based human activity recognition and behavior modeling. We discuss the nature of wearable sensor data, the capabilities and limitations of LLMs in modeling them, and their integration with traditional machine learning techniques. We also identify key challenges, including data quality, computational requirements, interpretability, and privacy concerns. By examining case studies and successful applications, we highlight the potential of LLMs in enhancing the analysis and interpretation of wearable sensor data. Finally, we propose future directions for research, emphasizing the need for improved preprocessing techniques, more efficient and scalable models, and interdisciplinary collaboration. This survey aims to provide a comprehensive overview of the intersection between wearable sensor data and LLMs, offering insights into the current state and future prospects of this emerging field.

Beyond Grids: Scaling Up Continuous Kernels via Adaptive Point Representations for Sensor-Based Human Activity Recognition

Beyond Grids: Scaling Up Continuous Kernels via Adaptive Point Representations for Sensor-Based Human Activity Recognition

Deep generative domain adaptation with temporal relation attention mechanism for cross-user activity recognition

In sensor-based Human Activity Recognition (HAR), a predominant assumption is that the data utilized for training and evaluation purposes are drawn from the same distribution. It is also assumed that all data samples are independent and identically distributed (i.i.d.). Contrarily, practical implementations often challenge this notion, manifesting data distribution discrepancies, especially in scenarios such as cross-user HAR. Domain adaptation is the promising approach to address these challenges inherent in cross-user HAR tasks. However, a clear gap in domain adaptation techniques is the neglect of the temporal dependency relation embedded within time series data during the phase of aligning data distributions. Addressing this oversight, our research presents the Deep Generative Domain Adaptation with Temporal Attention (DGDATA) method. This novel method uniquely recognizes and integrates temporal dependency relations during the domain adaptation process. By synergizing the capabilities of generative models with the Temporal Relation Attention mechanism, our method improves the classification performance in cross-user HAR. The evaluation has been conducted on three public sensor-based HAR datasets targeting daily living activity and sports fitness activity scenarios to demonstrate the efficacy of the proposed DGDATA method.

Using Graphs to Perform Effective Sensor-Based Human Activity Recognition in Smart Homes.

There has been a resurgence of applications focused on human activity recognition (HAR) in smart homes, especially in the field of ambient intelligence and assisted-living technologies. However, such applications present numerous significant challenges to any automated analysis system operating in the real world, such as variability, sparsity, and noise in sensor measurements. Although state-of-the-art HAR systems have made considerable strides in addressing some of these challenges, they suffer from a practical limitation: they require successful pre-segmentation of continuous sensor data streams prior to automated recognition, i.e., they assume that an oracle is present during deployment, and that it is capable of identifying time windows of interest across discrete sensor events. To overcome this limitation, we propose a novel graph-guided neural network approach that performs activity recognition by learning explicit co-firing relationships between sensors. We accomplish this by learning a more expressive graph structure representing the sensor network in a smart home in a data-driven manner. Our approach maps discrete input sensor measurements to a feature space through the application of attention mechanisms and hierarchical pooling of node embeddings. We demonstrate the effectiveness of our proposed approach by conducting several experiments on CASAS datasets, showing that the resulting graph-guided neural network outperforms the state-of-the-art method for HAR in smart homes across multiple datasets and by large margins. These results are promising because they push HAR for smart homes closer to real-world applications.

SenseMLP: a parallel MLP architecture for sensor-based human activity recognition

SenseMLP: a parallel MLP architecture for sensor-based human activity recognition

Real-Time Sensor-Based Human Activity Recognition for eFitness and eHealth Platforms

Human Activity Recognition (HAR) plays an important role in the automation of various tasks related to activity tracking in such areas as healthcare and eldercare (telerehabilitation, telemonitoring), security, ergonomics, entertainment (fitness, sports promotion, human–computer interaction, video games), and intelligent environments. This paper tackles the problem of real-time recognition and repetition counting of 12 types of exercises performed during athletic workouts. Our approach is based on the deep neural network model fed by the signal from a 9-axis motion sensor (IMU) placed on the chest. The model can be run on mobile platforms (iOS, Android). We discuss design requirements for the system and their impact on data collection protocols. We present architecture based on an encoder pretrained with contrastive learning. Compared to end-to-end training, the presented approach significantly improves the developed model’s quality in terms of accuracy (F1 score, MAPE) and robustness (false-positive rate) during background activity. We make the AIDLAB-HAR dataset publicly available to encourage further research.

TCN-Inception: Temporal Convolutional Network and Inception modules for sensor-based Human Activity Recognition

The field of Human Activity Recognition (HAR) has experienced a significant surge in interest due to its essential role across numerous areas, including human–computer interaction (HCI), healthcare, smart homes, and various Internet of Things (IoT) applications. The power of deep learning methods in performing various classification tasks, including HAR, has been well-demonstrated. In light of this, our paper presents an efficient HAR system developed using a unique deep-learning architecture called TCN-Inception, which is designed for multivariate time series tasks like HAR data, by combining Temporal Convolutional Network (TCN) and Inception modules. The network starts with an Inception module that uses parallel convolution layers with various kernel sizes for feature extraction. It then includes a TCN module with dilated convolutions to grasp extended temporal patterns. Features are merged from different channels, and the use of residual connections and batch normalization improves training and deepens the architecture. We use four public datasets, UCI-HAR, MobiAct, Daphnet, and DSADS to assess the performance of the TCN-Inception model, and it obtains an average accuracy of 96.15%, 98.86%, 92.63%, and 99.50% for each dataset, respectively. Moreover, we compare the TCN-Inception to several deep learning frameworks to verify its performance. Finally, we implement an ablation study using several architectural configurations of the TCN-Inception model.

Position-Aware Indoor Human Activity Recognition Using Multisensors Embedded in Smartphones.

Composite indoor human activity recognition is very important in elderly health monitoring and is more difficult than identifying individual human movements. This article proposes a sensor-based human indoor activity recognition method that integrates indoor positioning. Convolutional neural networks are used to extract spatial information contained in geomagnetic sensors and ambient light sensors, while transform encoders are used to extract temporal motion features collected by gyroscopes and accelerometers. We established an indoor activity recognition model with a multimodal feature fusion structure. In order to explore the possibility of using only smartphones to complete the above tasks, we collected and established a multisensor indoor activity dataset. Extensive experiments verified the effectiveness of the proposed method. Compared with algorithms that do not consider the location information, our method has a 13.65% improvement in recognition accuracy.

CrossHAR: Generalizing Cross-dataset Human Activity Recognition via Hierarchical Self-Supervised Pretraining

The increasing availability of low-cost wearable devices and smartphones has significantly advanced the field of sensor-based human activity recognition (HAR), attracting considerable research interest. One of the major challenges in HAR is the domain shift problem in cross-dataset activity recognition, which occurs due to variations in users, device types, and sensor placements between the source dataset and the target dataset. Although domain adaptation methods have shown promise, they typically require access to the target dataset during the training process, which might not be practical in some scenarios. To address these issues, we introduce CrossHAR, a new HAR model designed to improve model performance on unseen target datasets. CrossHAR involves three main steps: (i) CrossHAR explores the sensor data generation principle to diversify the data distribution and augment the raw sensor data. (ii) CrossHAR then employs a hierarchical self-supervised pretraining approach with the augmented data to develop a generalizable representation. (iii) Finally, CrossHAR fine-tunes the pretrained model with a small set of labeled data in the source dataset, enhancing its performance in cross-dataset HAR. Our extensive experiments across multiple real-world HAR datasets demonstrate that CrossHAR outperforms current state-of-the-art methods by 10.83% in accuracy, demonstrating its effectiveness in generalizing to unseen target datasets.

AutoAugHAR: Automated Data Augmentation for Sensor-based Human Activity Recognition

Sensor-based HAR models face challenges in cross-subject generalization due to the complexities of data collection and annotation, impacting the size and representativeness of datasets. While data augmentation has been successfully employed in domains like natural language and image processing, its application in HAR remains underexplored. This study presents AutoAugHAR, an innovative two-stage gradient-based data augmentation optimization framework. AutoAugHAR is designed to take into account the unique attributes of candidate augmentation operations and the unique nature and challenges of HAR tasks. Notably, it optimizes the augmentation pipeline during HAR model training without substantially extending the training duration. In evaluations on eight inertial-measurement-units-based benchmark datasets using five HAR models, AutoAugHAR has demonstrated superior robustness and effectiveness compared to other leading data augmentation frameworks. A salient feature of AutoAugHAR is its model-agnostic design, allowing for its seamless integration with any HAR model without the need for structural modifications. Furthermore, we also demonstrate the generalizability and flexible extensibility of AutoAugHAR on four datasets from other adjacent domains. We strongly recommend its integration as a standard protocol in HAR model training and will release it as an open-source tool1.

Resource-efficient, sensor-based human activity recognition with lightweight deep models boosted with attention

With their automatic feature extraction capabilities, deep learning models have become more widespread in sensor-based human activity recognition, particularly on larger datasets. However, their direct use on mobile and wearable devices is challenging due to the extensive resource requirements. Concurrently, attention-based models are emerging to improve recognition performance by dynamically emphasizing relevant parts of features and disregarding the irrelevant ones, particularly in the computer vision domain. This study introduces a novel application of attention mechanisms to smaller deep architectures, investigating whether smaller models can achieve comparable recognition performance to larger models in sensor-based human activity recognition systems while keeping resource usage at lower levels. For this purpose, we integrate the convolutional block attention module into a hybrid model, deep convolutional and long short-term memory network. Experiments are conducted using five public datasets in three model sizes: lightweight, moderate and original. The results show that applying attention to the lightweight model enables achieving similar recognition performances to the moderate-size model, and the lightweight model requires approximately 2–13 times fewer parameters and 3.5 times fewer flops. We also conduct experiments with sensor data at lower sampling rates and from fewer sensors attached to different body parts. The results show that attention improves recognition performance under lower sampling rates, as well as under higher sampling rates when model sizes are smaller, and mitigates the impact of missing data from one or more body parts, making the model more suitable for real-world sensor-based applications.

Multi-head CNN-based activity recognition and its application on chest-mounted sensor-belt

In recent years, a great deal of research has been done on the identification, monitoring, and classification of human activities. Human activity recognition (HAR) is a term commonly used to describe the automatic identification of physical activities. For activity recognition, there are primarily vision-based and sensor-based methods available. The computer vision-based method is generally effective in lab settings, but because of clutter, fluctuating light levels, and contrast, it may not perform well in real-world scenarios. Continuous monitoring and analysis of physiological signals obtained from heterogeneous sensors attached to an individual’s body is required to realise sensor-based HAR systems. Most of the previous research in human activity recognition (HAR) is biased along with feature engineering and pre-processing which requires a good amount of domain knowledge. Application-specific modelling and time-taking methods are involved in these approaches. In this work, the multi-head convolutional neural network-based human activity recognition framework is proposed where automatic feature extraction and classification are involved in the form of an end-to-end classification approach. Experiments of this approach are performed by taking raw wearable sensor data with few pre-processing steps and without the involvement of a handcrafted feature extraction technique. 99.23% and 93.55% accuracy are obtained on the WISDM and UCI-HAR datasets which denoted the much improvement in the assessment of HAR over other similar approaches. The model is also tested on locally collected data from a chest mounted belt with fabric sensors and an accuracy of 87.14% has been achieved on that data.

Deep Residual Network with a CBAM Mechanism for the Recognition of Symmetric and Asymmetric Human Activity Using Wearable Sensors

Wearable devices are paramount in health monitoring applications since they provide contextual information to identify and recognize human activities. Although sensor-based human activity recognition (HAR) has been thoroughly examined, prior studies have yet to definitively differentiate between symmetric and asymmetric motions. Determining these movement patterns might provide a more profound understanding of assessing physical activity. The main objective of this research is to investigate the use of wearable motion sensors and deep convolutional neural networks in the analysis of symmetric and asymmetric activities. This study provides a new approach for classifying symmetric and asymmetric motions using a deep residual network incorporating channel and spatial convolutional block attention modules (CBAMs). Two publicly accessible benchmark HAR datasets, which consist of inertial measurements obtained from wrist-worn sensors, are used to assess the model’s efficacy. The model we have presented is subjected to thorough examination and demonstrates exceptional accuracy on both datasets. The ablation experiment examination also demonstrates noteworthy contributions from the residual mappings and CBAMs. The significance of recognizing basic movement symmetries in increasing sensor-based activity identification utilizing wearable devices is shown by the enhanced accuracy and F1-score, especially in asymmetric activities. The technique under consideration can provide activity monitoring with enhanced accuracy and detail, offering prospective advantages in diverse domains like customized healthcare, fitness tracking, and rehabilitation progress evaluation.

Cross-Attention Enhanced Pyramid Multi-Scale Networks for Sensor-Based Human Activity Recognition.

Human Activity Recognition (HAR) has recently attracted widespread attention, with the effective application of this technology helping people in areas such as healthcare, smart homes, and gait analysis. Deep learning methods have shown remarkable performance in HAR. A pivotal challenge is the trade-off between recognition accuracy and computational efficiency, especially in resource-constrained mobile devices. This challenge necessitates the development of models that enhance feature representation capabilities without imposing additional computational burdens. Addressing this, we introduce a novel HAR model leveraging deep learning, ingeniously designed to navigate the accuracy-efficiency trade-off. The model comprises two innovative modules: 1) Pyramid Multi-scale Convolutional Network (PMCN), which is designed with a symmetric structure and is capable of obtaining a rich receptive field at a finer level through its multiscale representation capability; 2) Cross-Attention Mechanism, which establishes interrelationships among sensor dimensions, temporal dimensions, and channel dimensions, and effectively enhances useful information while suppressing irrelevant data. The proposed model is rigorously evaluated across four diverse datasets: UCI, WISDM, PAMAP2, and OPPORTUNITY. Additional ablation and comparative studies are conducted to comprehensively assess the performance of the model. Experimental results demonstrate that the proposed model achieves superior activity recognition accuracy while maintaining low computational overhead.

Trainable Gaussian-based activation functions for sensor-based human activity recognition

Trainable Gaussian-based activation functions for sensor-based human activity recognition