Sensor-based Human Activity Recognition Research Articles

Eff-WHAR: A Lightweight Design for Efficient Wearable Sensor-Based Human Activity Recognition

Eff-WHAR: A Lightweight Design for Efficient Wearable Sensor-Based Human Activity Recognition

Machine Learning Techniques for Sensor-Based Human Activity Recognition with Data Heterogeneity-A Review.

Sensor-based Human Activity Recognition (HAR) is crucial in ubiquitous computing, analyzing behaviors through multi-dimensional observations. Despite research progress, HAR confronts challenges, particularly in data distribution assumptions. Most studies assume uniform data distributions across datasets, contrasting with the varied nature of practical sensor data in human activities. Addressing data heterogeneity issues can improve performance, reduce computational costs, and aid in developing personalized, adaptive models with fewer annotated data. This review investigates how machine learning addresses data heterogeneity in HAR by categorizing data heterogeneity types, applying corresponding suitable machine learning methods, summarizing available datasets, and discussing future challenges.

Cross-Domain HAR: Few Shot Transfer Learning for Human Activity Recognition

The ubiquitous availability of smartphones and smartwatches with integrated inertial measurement units (IMUs) enables straightforward capturing of human activities through collecting movement data. For specific applications of sensor based human activity recognition (HAR), however, logistical challenges and burgeoning costs render especially the ground truth annotation of such data a difficult endeavor, resulting in limited scale and diversity of datasets available for deriving effective HAR systems and less than ideal recognition capabilities. Transfer learning, i.e., leveraging publicly available labeled datasets to first learn useful representations that can then be fine-tuned using limited amounts of labeled data from a target domain, can alleviate some of the performance issues of contemporary HAR systems. Yet they can fail when the differences between source and target conditions are too large and / or only few samples from a target application domain are available – each of which are typical challenges in real-world human activity recognition scenarios. In this paper, we present an approach for economic use of publicly available labeled HAR datasets for effective transfer learning. We introduce a novel transfer learning framework–Cross-Domain HAR–which follows the teacher-student self-training paradigm to more effectively recognize activities with very limited label information. It bridges conceptual gaps between source and target domains, including sensor locations and type of activities. Cross-Domain HAR enables substantial performance improvements over the state-of-the-art in sensor-based HAR scenarios. Through our extensive experimental evaluation on a range of benchmark datasets we specifically demonstrate the effectiveness of our approach for practically relevant few shot activity recognition scenarios. We also present a detailed analysis into how the individual components of our framework affect downstream performance and provide practical suggestions for using the framework in real-world applications.

LHAR: Lightweight Human Activity Recognition on Knowledge Distillation.

Sensor-based Human Activity Recognition (HAR) is widely used in daily life and is the basic-level bridge to virtual healthcare in the metaverse. The current challenge is the low recognition accuracy for personalized users on smart wearable devices. The limited resource cannot support large deep learning models updated locally. Besides, integrating and transmitting sensor data to the cloud would reduce the efficiency. Considering the tradeoff between performance and complexity, we propose a Lightweight Human Activity Recognition (LHAR) framework. In LHAR, we combine the cross-people HAR task with the lightweight model task. LHAR framework is designed on the teacher-student architecture and the student network consists of multiple depthwise separable convolution layers to achieve fewer parameters. The dark knowledge distilled from the complex teacher model enhances the generalization ability of LHAR. To achieve effective knowledge distillation, we propose two optimization methods. Firstly, we train the teacher model by ensemble learning to promote teacher performance. Secondly, a multi-channel data augmentation method is proposed for the diversity of the dataset, which is a plug-in operation for the ensemble teacher model. In the experiments, we compare LHAR with state-of-art models in comparison evaluation, ablation study and the hyperparameter analysis, which proves the better performance of LHAR in efficiency and effectiveness.

EGTCN: An Efficient Graph and Temporal Convolution Network for Sensor-Based Human Activity Recognition in Federated Learning

EGTCN: An Efficient Graph and Temporal Convolution Network for Sensor-Based Human Activity Recognition in Federated Learning

Belief-Rule-Based System with Self-organizing and Multi-temporal Modeling for Sensor-based Human Activity Recognition.

Smart environment is an efficient and cost- effective way to afford intelligent supports for the elderly people. Human activity recognition (HAR) is a crucial aspect of the research field of smart environments, and it has attracted widespread attention lately. The goal of this study is to develop an effective sensor-based HAR model based on the belief-rule-based system (BRBS), which is one of representative rule-based expert systems. Specially, a new belief rule base (BRB) modeling approach is proposed by taking into account the self- organizing rule generation method and the multi-temporal rule representation scheme, in order to address the problem of combination explosion that existed in the traditional BRB modelling procedure and the time correlation found in continuous sensor data in chronological order. The new BRB modeling approach is so called self-organizing and multi-temporal BRB (SOMT-BRB) modeling procedure. A case study is further deducted to validate the effectiveness of the SOMT-BRB modeling procedure. By comparing with some conventional BRBSs and classical activity recognition models, the results show a significant improvement of the BRBS in terms of the number of belief rules, modelling efficiency, and activity recognition accuracy.

Deep Learning and Recurrent Signature Based Classification for Sensor-Based HAR: Addressing Explainability and Complexity in 5G Networks

When it comes to clinical applications, sensor-based human activity recognition (HAR) is invaluable, and numerous machine learning algorithms have effectively used to obtain excellent presentation. Using a variety of on-body sensors, these systems attempt to ascertain the subject's status relative to their immediate surroundings. There was a time when feature extraction was done by hand, but now more and more people are using Artificial Neural Networks (ANNs). A number of innovative approaches to HAR have surfaced since the advent of deep learning. Problems arise, however, for sensor-based HAR classification algorithms in today's communication networks. Among these, you can find solutions to problems like deal with complicated and large-scale data signals, extract characteristics from complicated datasets, and meet explainability standards. For complicated 5G networks, these difficulties become even more apparent. In particular, explainability is now critical for the broad use of sensor-based HAR in 5G networks and beyond. The research suggests a classification approach based on path signatures, recurrent signature (ReS), to address these issues. This cutting-edge model employs deep-learning (DL) approaches to circumvent the tedious feature selection challenge. Furthermore, the study investigates how to improve the ReS model's classification accuracy by using graph-based optimisation methods. To test how well the suggested framework worked, to dug deep into the publicly available dataset, which included a separate set of tasks. The paper's empirical results on AReM datasets achieved an average accuracy of 96%.

A Novel Active Learning Framework for Cross-Subject Human Activity Recognition from Surface Electromyography.

Wearable sensor-based human activity recognition (HAR) methods hold considerable promise for upper-level control in exoskeleton systems. However, such methods tend to overlook the critical role of data quality and still encounter challenges in cross-subject adaptation. To address this, we propose an active learning framework that integrates the relation network architecture with data sampling techniques. Initially, target data are used to fine tune two auxiliary classifiers of the pre-trained model, thereby establishing subject-specific classification boundaries. Subsequently, we assess the significance of the target data based on classifier discrepancy and partition the data into sample and template sets. Finally, the sampled data and a category clustering algorithm are employed to tune model parameters and optimize template data distribution, respectively. This approach facilitates the adaptation of the model to the target subject, enhancing both accuracy and generalizability. To evaluate the effectiveness of the proposed adaptation framework, we conducted evaluation experiments on a public dataset and a self-constructed electromyography (EMG) dataset. Experimental results demonstrate that our method outperforms the compared methods across all three statistical metrics. Furthermore, ablation experiments highlight the necessity of data screening. Our work underscores the practical feasibility of implementing user-independent HAR methods in exoskeleton control systems.

Hand-Crafted Features With a Simple Deep Learning Architecture for Sensor-Based Human Activity Recognition

Hand-Crafted Features With a Simple Deep Learning Architecture for Sensor-Based Human Activity Recognition

Multihead-Res-SE Residual Network with Attention for Human Activity Recognition

Human activity recognition (HAR) typically uses wearable sensors to identify and analyze the time-series data they collect, enabling recognition of specific actions. As such, HAR is increasingly applied in human–computer interaction, healthcare, and other fields, making accurate and efficient recognition of various human activities. In recent years, deep learning methods have been extensively applied in sensor-based HAR, yielding remarkable results. However, complex HAR research, which involves specific human behaviors in varied contexts, still faces several challenges. To solve these problems, we propose a multi-head neural network based on the attention mechanism. This framework contains three convolutional heads, with each head designed using one-dimensional CNN to extract features from sensory data. The model uses a channel attention module (squeeze–excitation module) to enhance the representational capabilities of convolutional neural networks. We conducted experiments on two publicly available benchmark datasets, UCI-HAR and WISDM, to evaluate our model. The results were satisfactory, with overall recognition accuracies of 96.72% and 97.73% on their respective datasets. The experimental results demonstrate the effectiveness of the network structure for the HAR, which ensures a higher level of accuracy.

BSTCA-HAR: Human Activity Recognition Model Based on Wearable Mobile Sensors

Sensor-based human activity recognition has been widely used in various fields; however, there are still challenges involving recognition of daily complex human activities using sensors. In order to solve the problem of timeliness and homogeneity of recognition functions in human activity recognition models, we propose a human activity recognition model called ’BSTCA-HAR’ based on a long short-term memory (LSTM) network. The approach proposed in this paper combines an attention mechanism and a temporal convolutional network (TCN). The learning and prediction units in the model can efficiently learn important action data while capturing long time-dependent information as well as features at different time scales. Our series of experiments on three public datasets (WISDM, UCI-HAR, and ISLD) with different data features confirm the feasibility of the proposed method. This method excels in dynamically capturing action features while maintaining a low number of parameters and achieving a remarkable average accuracy of 93%, proving that the model has good recognition performance.

Large Language Models for Wearable Sensor-Based Human Activity Recognition, Health Monitoring, and Behavioral Modeling: A Survey of Early Trends, Datasets, and Challenges.

The proliferation of wearable technology enables the generation of vast amounts of sensor data, offering significant opportunities for advancements in health monitoring, activity recognition, and personalized medicine. However, the complexity and volume of these data present substantial challenges in data modeling and analysis, which have been addressed with approaches spanning time series modeling to deep learning techniques. The latest frontier in this domain is the adoption of large language models (LLMs), such as GPT-4 and Llama, for data analysis, modeling, understanding, and human behavior monitoring through the lens of wearable sensor data. This survey explores the current trends and challenges in applying LLMs for sensor-based human activity recognition and behavior modeling. We discuss the nature of wearable sensor data, the capabilities and limitations of LLMs in modeling them, and their integration with traditional machine learning techniques. We also identify key challenges, including data quality, computational requirements, interpretability, and privacy concerns. By examining case studies and successful applications, we highlight the potential of LLMs in enhancing the analysis and interpretation of wearable sensor data. Finally, we propose future directions for research, emphasizing the need for improved preprocessing techniques, more efficient and scalable models, and interdisciplinary collaboration. This survey aims to provide a comprehensive overview of the intersection between wearable sensor data and LLMs, offering insights into the current state and future prospects of this emerging field.

Beyond Grids: Scaling Up Continuous Kernels via Adaptive Point Representations for Sensor-Based Human Activity Recognition

Beyond Grids: Scaling Up Continuous Kernels via Adaptive Point Representations for Sensor-Based Human Activity Recognition

Deep generative domain adaptation with temporal relation attention mechanism for cross-user activity recognition

In sensor-based Human Activity Recognition (HAR), a predominant assumption is that the data utilized for training and evaluation purposes are drawn from the same distribution. It is also assumed that all data samples are independent and identically distributed (i.i.d.). Contrarily, practical implementations often challenge this notion, manifesting data distribution discrepancies, especially in scenarios such as cross-user HAR. Domain adaptation is the promising approach to address these challenges inherent in cross-user HAR tasks. However, a clear gap in domain adaptation techniques is the neglect of the temporal dependency relation embedded within time series data during the phase of aligning data distributions. Addressing this oversight, our research presents the Deep Generative Domain Adaptation with Temporal Attention (DGDATA) method. This novel method uniquely recognizes and integrates temporal dependency relations during the domain adaptation process. By synergizing the capabilities of generative models with the Temporal Relation Attention mechanism, our method improves the classification performance in cross-user HAR. The evaluation has been conducted on three public sensor-based HAR datasets targeting daily living activity and sports fitness activity scenarios to demonstrate the efficacy of the proposed DGDATA method.

Using Graphs to Perform Effective Sensor-Based Human Activity Recognition in Smart Homes.

There has been a resurgence of applications focused on human activity recognition (HAR) in smart homes, especially in the field of ambient intelligence and assisted-living technologies. However, such applications present numerous significant challenges to any automated analysis system operating in the real world, such as variability, sparsity, and noise in sensor measurements. Although state-of-the-art HAR systems have made considerable strides in addressing some of these challenges, they suffer from a practical limitation: they require successful pre-segmentation of continuous sensor data streams prior to automated recognition, i.e., they assume that an oracle is present during deployment, and that it is capable of identifying time windows of interest across discrete sensor events. To overcome this limitation, we propose a novel graph-guided neural network approach that performs activity recognition by learning explicit co-firing relationships between sensors. We accomplish this by learning a more expressive graph structure representing the sensor network in a smart home in a data-driven manner. Our approach maps discrete input sensor measurements to a feature space through the application of attention mechanisms and hierarchical pooling of node embeddings. We demonstrate the effectiveness of our proposed approach by conducting several experiments on CASAS datasets, showing that the resulting graph-guided neural network outperforms the state-of-the-art method for HAR in smart homes across multiple datasets and by large margins. These results are promising because they push HAR for smart homes closer to real-world applications.

SenseMLP: a parallel MLP architecture for sensor-based human activity recognition

SenseMLP: a parallel MLP architecture for sensor-based human activity recognition

Real-Time Sensor-Based Human Activity Recognition for eFitness and eHealth Platforms

Human Activity Recognition (HAR) plays an important role in the automation of various tasks related to activity tracking in such areas as healthcare and eldercare (telerehabilitation, telemonitoring), security, ergonomics, entertainment (fitness, sports promotion, human–computer interaction, video games), and intelligent environments. This paper tackles the problem of real-time recognition and repetition counting of 12 types of exercises performed during athletic workouts. Our approach is based on the deep neural network model fed by the signal from a 9-axis motion sensor (IMU) placed on the chest. The model can be run on mobile platforms (iOS, Android). We discuss design requirements for the system and their impact on data collection protocols. We present architecture based on an encoder pretrained with contrastive learning. Compared to end-to-end training, the presented approach significantly improves the developed model’s quality in terms of accuracy (F1 score, MAPE) and robustness (false-positive rate) during background activity. We make the AIDLAB-HAR dataset publicly available to encourage further research.

TCN-Inception: Temporal Convolutional Network and Inception modules for sensor-based Human Activity Recognition

The field of Human Activity Recognition (HAR) has experienced a significant surge in interest due to its essential role across numerous areas, including human–computer interaction (HCI), healthcare, smart homes, and various Internet of Things (IoT) applications. The power of deep learning methods in performing various classification tasks, including HAR, has been well-demonstrated. In light of this, our paper presents an efficient HAR system developed using a unique deep-learning architecture called TCN-Inception, which is designed for multivariate time series tasks like HAR data, by combining Temporal Convolutional Network (TCN) and Inception modules. The network starts with an Inception module that uses parallel convolution layers with various kernel sizes for feature extraction. It then includes a TCN module with dilated convolutions to grasp extended temporal patterns. Features are merged from different channels, and the use of residual connections and batch normalization improves training and deepens the architecture. We use four public datasets, UCI-HAR, MobiAct, Daphnet, and DSADS to assess the performance of the TCN-Inception model, and it obtains an average accuracy of 96.15%, 98.86%, 92.63%, and 99.50% for each dataset, respectively. Moreover, we compare the TCN-Inception to several deep learning frameworks to verify its performance. Finally, we implement an ablation study using several architectural configurations of the TCN-Inception model.

Position-Aware Indoor Human Activity Recognition Using Multisensors Embedded in Smartphones.

Composite indoor human activity recognition is very important in elderly health monitoring and is more difficult than identifying individual human movements. This article proposes a sensor-based human indoor activity recognition method that integrates indoor positioning. Convolutional neural networks are used to extract spatial information contained in geomagnetic sensors and ambient light sensors, while transform encoders are used to extract temporal motion features collected by gyroscopes and accelerometers. We established an indoor activity recognition model with a multimodal feature fusion structure. In order to explore the possibility of using only smartphones to complete the above tasks, we collected and established a multisensor indoor activity dataset. Extensive experiments verified the effectiveness of the proposed method. Compared with algorithms that do not consider the location information, our method has a 13.65% improvement in recognition accuracy.

CrossHAR: Generalizing Cross-dataset Human Activity Recognition via Hierarchical Self-Supervised Pretraining

The increasing availability of low-cost wearable devices and smartphones has significantly advanced the field of sensor-based human activity recognition (HAR), attracting considerable research interest. One of the major challenges in HAR is the domain shift problem in cross-dataset activity recognition, which occurs due to variations in users, device types, and sensor placements between the source dataset and the target dataset. Although domain adaptation methods have shown promise, they typically require access to the target dataset during the training process, which might not be practical in some scenarios. To address these issues, we introduce CrossHAR, a new HAR model designed to improve model performance on unseen target datasets. CrossHAR involves three main steps: (i) CrossHAR explores the sensor data generation principle to diversify the data distribution and augment the raw sensor data. (ii) CrossHAR then employs a hierarchical self-supervised pretraining approach with the augmented data to develop a generalizable representation. (iii) Finally, CrossHAR fine-tunes the pretrained model with a small set of labeled data in the source dataset, enhancing its performance in cross-dataset HAR. Our extensive experiments across multiple real-world HAR datasets demonstrate that CrossHAR outperforms current state-of-the-art methods by 10.83% in accuracy, demonstrating its effectiveness in generalizing to unseen target datasets.