Human Activity Recognition Research Articles

Effective Human Activity Recognition through Accelerometer Data

Effective Human Activity Recognition through Accelerometer Data

WiCAM2.0: Imperceptible and Targeted Attack on Deep Learning based WiFi Sensing

With the widespread adoption of deep learning models in wireless sensing, substantial efforts have been made to develop sophisticated models that improve the accuracy and performance of sensing applications. However, the exploration of potential vulnerabilities in deep models has been limited, with existing studies primarily focusing on evaluating wireless adversarial performance in communication or sensing alone. Moreover, there is a lack of a comprehensive definition for attack imperceptibility. In this paper, we come up with a definition of the wireless attack imperceptibility for both communication and sensing. Our objective is to create an adversarial perturbation capable of degrading WiFi sensing performance while preserving WiFi communication integrity. To achieve this, we propose WiCAM2.0 to reveal the temporal and spatial attention of a DNN, capturing the crucial portions of its input. Then we design a mask to confine adversarial perturbations in the attended parts only, minimizing the impact on WiFi communication. WiCAM2.0 is a general adversarial framework that integrates adversarial methods like FGSM and PGD to generate perturbations, capable of initiating both non-targeted and targeted attacks. We carry out experiments on three popular WiFi sensing applications, including human activity recognition, gesture recognition, and user identification. Extensive experiments are conducted on both public datasets and self-collected datasets.

Deep Learning and Recurrent Signature Based Classification for Sensor-Based HAR: Addressing Explainability and Complexity in 5G Networks

When it comes to clinical applications, sensor-based human activity recognition (HAR) is invaluable, and numerous machine learning algorithms have effectively used to obtain excellent presentation. Using a variety of on-body sensors, these systems attempt to ascertain the subject's status relative to their immediate surroundings. There was a time when feature extraction was done by hand, but now more and more people are using Artificial Neural Networks (ANNs). A number of innovative approaches to HAR have surfaced since the advent of deep learning. Problems arise, however, for sensor-based HAR classification algorithms in today's communication networks. Among these, you can find solutions to problems like deal with complicated and large-scale data signals, extract characteristics from complicated datasets, and meet explainability standards. For complicated 5G networks, these difficulties become even more apparent. In particular, explainability is now critical for the broad use of sensor-based HAR in 5G networks and beyond. The research suggests a classification approach based on path signatures, recurrent signature (ReS), to address these issues. This cutting-edge model employs deep-learning (DL) approaches to circumvent the tedious feature selection challenge. Furthermore, the study investigates how to improve the ReS model's classification accuracy by using graph-based optimisation methods. To test how well the suggested framework worked, to dug deep into the publicly available dataset, which included a separate set of tasks. The paper's empirical results on AReM datasets achieved an average accuracy of 96%.

An IoT solution that detects human physical activities to support health assessment based on sensors and supervised learning algorithms

At the moment, human physical activities can be detected by activity recognition systems to help with health status evaluation. One of the key issues in human activity recognition (HAR) systems is cost, speed, and accuracy. With the advancement of mobile devices and increasingly sophisticated microprocessors, along with the integration of various sensors including accelerometers, human activity recognition has become easier and faster. However, the accuracy of activity classification based on accelerometer data depends on many different factors, which can lead to potential deviations in results. Through various experiments, we have found that selecting optimal features is crucial for the performance of classification. In this paper, we propose a simple feature set that yields very promising results with an accuracy of up to 99%.

Human activity recognition using binary sensors: A systematic review

Human activity recognition (HAR) is an emerging area of study and research field that explores the development of automated systems to identify and categorize human activities using data collected from various sensors. In the field of Human Activity Recognition (HAR), binary sensors offer a distinct approach by providing simpler on/off readings to indicate the presence of events such as door openings or light switch activations. Compared to other sensors used for HAR, binary sensors have several advantages, including lower cost, low power consumption, ease of installation, and privacy preservation. For instance, they can be effectively used in smart homes to detect when someone enters or leaves a room without user input. This study presents a systematic review of the state-of-the-art methods and techniques for HAR using binary sensors. We comprehensively consider five crucial aspects: data collection methods, preprocessing techniques, feature extraction and fusion strategies, classification algorithms, and evaluation metrics. Furthermore, we identify the gaps and limitations of the existing studies and provide directions for future research. This comprehensive and up-to-date review can serve as a valuable reference for researchers and practitioners in the field of HAR using binary sensors.

Hybrid random projection: Integrating dense and sparse techniques for enhanced representation in high-dimensional data

This paper introduces a novel hybrid random projection method (HRP) that effectively combines the strengths of normal random projection (NRP) and plus-minus one random projection (PMRP). By incorporating a blending parameter, HRP optimises the contributions of the NRP and PMRP, reducing the dimensions of the data while ensuring that the structure of the data is preserved. Conventional methods, such as NRP and PMRP, are recognised for their benefits; however, they also have limitations. NRP is generally accurate but can encounter difficulties with certain data types or noise. The PMRP is simple, but occasionally fails to capture complex relationships within the data. The performance of HRP is investigated through comprehensive evaluations, including simulations and real-world data analyses from key machine learning datasets, such as period change, toxicity, MNIST, and human activity recognition (HAR). We examine various factors such as sample size, dimensions of the original and reduced data, and sparsity. Distance distortion is the primary metric utilised to measure performance, and indicates how well the dataset structure is maintained after dimension reduction. In every test, HRP consistently demonstrates the least distance distortion, performing superiorly to the NRP and PMRP. This is true for a range of scenarios and datasets. HRP represents a significant advancement in dimension reduction techniques. Its ability to minimise information loss during the reduction process demonstrates its potential as a powerful tool for managing complex high-dimensional data in practical applications. This renders HRP an important development in the fields of machine learning, intelligent systems, and artificial intelligence, offering improved efficiency and precision in data analyses.

Graph-Structure-Based Multigranular Belief Fusion for Human Activity Recognition.

The belief functions (BFs) introduced by Shafer in the mid of 1970s are widely applied in information fusion to model epistemic uncertainty and to reason about uncertainty. Their success in applications is however limited because of their high-computational complexity in the fusion process, especially when the number of focal elements is large. To reduce the complexity of reasoning with BFs, we can envisage as a first method to reduce the number of focal elements involved in the fusion process to convert the original basic belief assignments (BBAs) into simpler ones, or as a second method to use a simple rule of combination with potentially a loss of the specificity and pertinence of the fusion result, or to apply both methods jointly. In this article, we focus on the first method and propose a new BBA granulation method inspired by the community clustering of nodes in graph networks. This article studies a novel efficient multigranular belief fusion (MGBF) method. Specifically, focal elements are regarded as nodes in the graph structure, and the distance between nodes will be used to discover the local community relationship of focal elements. Afterward, the nodes belonging to the decision-making community are specially selected, and then the derived multigranular sources of evidence can be efficiently combined. To evaluate the effectiveness of the proposed graph-based MGBF, we further apply this new approach to combine the outputs of convolutional neural networks + attention (CNN + Attention) in the human activity recognition (HAR) problem. The experimental results obtained with real datasets prove the potential interest and feasibility of our proposed strategy with respect to classical BF fusion methods.

GRASNet: A novel graph neural network for improving human action recognition and well-being assessment in smart manufacturing

In the manufacturing environment, prioritizing the well-being of workers is both essential and challenging. Recognizing and monitoring workers’ actions in real time is crucial in ensuring safety and health. This study introduces innovative approaches leveraging smart manufacturing technologies and Graph Neural Networks (GNNs) to enhance human action recognition using accessible skeletal data. Traditional Neural Networks encounter limitations in processing readily accessible human-related data, such as image and skeletal data, prompting the development of the Graph Residual Attention SAGE Network (GRASNet). GRASNet exhibits superior capabilities in capturing intricate local connectivity within graph data, ensuring a delicate balance of complexity and efficiency. Furthermore, experimental validations on two manufacturing-related datasets, derived from the NTU RGB + D 120 benchmark dataset, underscore GRASNet’s ability for high-accuracy monitoring of human abnormal behaviors and health issues, positioning it ahead of other existing deep learning models.

An Accurate Classification for Human Sport Activity Recognition from Sensor Data Using Hybrid Machine Learning Models

The significance of human sports activities recognition (HSAR) in many computing applications, including sports analytics, smart surroundings, and healthcare monitoring is huge. The current use of vision sensors for HSAR is a difficult task due to the intricate movements involved in sports and fitness workouts, as well as fluctuations in illumination conditions. Therefore, a comprehensive review of machine learning algorithms (such as XGBoost, Random Forest (RF), Decision Trees (DT), and K-Nearest Neighbors (KNN)) and their usage for human sports activities classification based on sensor data called ML-HSAR is presented in this article. A dataset containing 19 different human activities gathered from 8 different people through different types of sensors was used for this work; the dataset contains activities like sitting and standing, as well as other activities that require serious movements, such as running, cycling, and playing basketball. This study primarily aims to determine the performance of the considered ML models in accurately classifying these activities found in the dataset. The training of the algorithms and the subsequent evaluation were done on the segmented sensor data, effectively leveraging the dataset’s temporal aspect. In addition to the presented results for each of these algorithms, the limitations of each algorithm and their strengths in handling the complexities involved in HSAR were also presented. The contribution of this analysis is towards the understanding of the most appropriate ML models for HSAR tasks; it also offers valuable insights for future research and practical applications.

Hybrid Deep Learning Approaches for Human Activity Recognition and Postural Transitions Using Mobile Device Sensors

Hybrid Deep Learning Approaches for Human Activity Recognition and Postural Transitions Using Mobile Device Sensors

Ensem-DeepHAR: Identification of human activity in smart environments using ensemble of deep learning methods and motion sensor data

Recognizing human activity plays a crucial role in many applications such as medical care services in smart healthcare environments. Inertial or motion sensors can measure physiognomies such as acceleration and angular velocity of body movement while performing the activities and we can use them to learn the models capable of activity recognition. Over the past decades, many state-of-the-art activity recognition systems have been developed but there is still room to improve. In this paper, we have proposed a novel approach to identify human activity from motion sensor data by employing an enormous analysis of sensor data. Based on data analysis, we yielded quality data by preprocessing using a preprocessing chain for human activity recognition (PC-HAR) which also includes the Synthetic Minority Over-sampling Technique to balance the data of the dataset. As a recognition model, we proposed an ensemble of three different deep learning algorithms, namely, modified DeepConvLSTM, modified InceptionTime, and modified ResNet which is named ‘Ensem-DeepHAR’. The outcome of the proposed model is carried out by stacking predictions from each of the mentioned models and then a Random Forest as a meta-model uses those predictions to recognize the final activity. We evaluated our method on both person-dependent and person-independent cases and achieved 99.31 %, 99.08 %, and 97.52 % accuracies for the former case and 97.95 %, 98.11 %, and 99.51 % accuracies for the latter case using three common benchmark datasets: WISDM_ar_v1.1, PAMAP2, and UCI-HAR respectively. The various performance metrics and measures of experimental results establish the supremacy of the proposed model over the state-of-the-arts.

A Lightweight Deep Human Activity Recognition Algorithm Using Multiknowledge Distillation

A Lightweight Deep Human Activity Recognition Algorithm Using Multiknowledge Distillation

Evidence combination with multi-granularity belief structure for pattern classification

Belief function (BF) theory provides a framework for effective modeling, quantifying uncertainty, and combining evidence, rendering it a potent tool for tackling uncertain decision-making problems. However, with the expansion of the frame of discernment, the increasing number of focal elements processed during the fusion procedure leads to a rapid increase in computational complexity, which limits the practical application of BF theory. To overcome this issue, a novel multi-granularity belief structure (MGBS) method was proposed in this study. The construction of MGBS reduced the number of focal elements and preserved crucial information in the basic belief assignment. This effectively reduced the computational complexity of fusion while ensuring the highest possible classification accuracy. We applied the proposed MGBS algorithm to a human activity recognition task and verified its effectiveness using the University of California, Irvine mHealth, PAMAP2, and Smartphone datasets.

Feature-Level Fusion Multi-Sensor Aggregation Temporal Network for Smartphone-Based Human Activity Recognition

Smartphone-based Human Activity Recognition (HAR) identifies human movements using inertial signals gathered from multiple smartphone sensors. Generally, these signals are stacked as one (data-level fusion) and fed into deep learning algorithms for feature extractions. This research studies feature-level fusion, individually processing inertial signals from each sensor, and proposes a lightweight deep temporal learning model, Feature-Level Fusion Multi-Sensor Aggregation Temporal Network (FLF-MSATN), that performs feature extraction on inertial signals from each sensor separately. The raw signals, segmented into equally sized time windows, are passed into individual Dilated-Pooled Convolutional Heads (DPC Heads) for temporal feature analysis. Each DPC Head has a spatiotemporal block containing dilated causal convolutions and average pooling, to extract underlying patterns. The DPC Heads’ outputs are concatenated and passed into a Global Average Pooling layer to generate a condensed confidence map before activity classification. FLF-MSATN is assessed using a subject-independent protocol on a publicly available HAR dataset, UCI HAR, and a self-collected HAR dataset, achieving 96.67% and 82.70% accuracies, respectively. A Data-Level Fusion MSATN is built to compare and verify the model performance attained by the proposed FLF-MSATN. The empirical results show that implementing FLF-MSATN enhances the accuracy by ~3.4% for UCI HAR and ~9.68% for self-collected datasets.

Recognizing human activities using light-weight and effective machine learning methodologies

Background Human activity recognition poses a complex challenge in predicting individuals’ movements from raw sensor data using machine learning models. This paper explores the application of six prominent machine learning techniques – decision tree, random forest, linear regression, Naïve Bayes, k-nearest neighbor, and neural networks – to enhance the accuracy of human activity detection for e-health systems. Despite previous research efforts employing data mining and machine learning, there remains room for improvement in performance. The study focuses on predicting activities such as walking, standing, laying, sitting, walking upstairs, and walking downstairs. Methods The research employs six machine learning algorithms to recognize human activities, including decision tree, random forest, linear regression, Naïve Bayes, k-nearest neighbor, and neural networks. Results Evaluation of the human activity recognition dataset reveals that the random forest classifier, CNN, GRN and neural network yield promising results, achieving high accuracy. However, Naïve Bayes falls short of satisfying outcomes. Conclusions The study successfully classifies activities like SITTING, STANDING, LAYING, WALKING, WALKING_DOWNSTAIRS, and WALKING_UPSTAIRS with a remarkable accuracy of 98%. The contribution lies in the thorough exploration of machine learning techniques, with neural networks emerging as the most effective in enhancing human activity recognition. The findings showcase the potential for advanced applications in e-health systems and beyond.

Detection of sensors used for adversarial examples against machine learning models

Machine learning (ML) systems that utilize sensor inputs have become integral in various applications; however, they remain vulnerable to sensor-based adversarial example (AE) attacks, where compromised sensors can be exploited to manipulate system outputs. This study addresses the critical issue of safeguarding these systems by identifying and mitigating compromised sensors, thereby enhancing their resilience. This study introduces a novel detection method using a feature-removable model (FRM), which allows the selective removal of features to identify inconsistencies in the model's outputs when different sensor features are altered. The methodology was validated on a human activity recognition (HAR) model utilizing sensors placed on the chest, wrist, and ankle, with a focus on identifying attacker-compromised sensors. The results demonstrated the efficacy of the method, achieving an average Recall of detected sensors of 0.92 and an average Precision of detected sensors of 0.72, highlighting the ability of the approach to accurately detect and identify compromised sensors. This study significantly contributes to advancing the security and robustness of ML systems against sensor-based AE attacks.

Improved RNN Model for Real-Time Human Activity Recognition

This study focuses on the automatic detection of human actions in video streams. The requirement to detect what human activities happen in videos is recognition of human action due to significant differences in people's visual and motion appearance and actions, camera perspective shifts, moving background, occlusions, noise, and a massive amount of video data. The human activity recognition challenge involves identifying physical activities carried out by individuals or groups based on traces of movements, including gestures, actions, interactions, and group activities. The detection of concepts usually requires additional annotations for the training dataset. In this paper, useful methods for categorizing human action recognition are discussed. The current models are an accurate deep learning method that is based on models that have been changed to be more useful. The large disparities that result from the backdrop and the size of the objects have prevented the identification of activities in videos from being fully and effectively addressed. The main objective is to achieve better accuracy for the Long Short-Term Memory (LSTM) method, which was used to improve the Recurrent Neural Networks (RNN) model. In this paper, LSTM is used to come up with models for different action recognition tasks. The model was made better by making the LSTM have four layers and putting 128 units, 64 units, 32 units, and 16 units in each layer, respectively. In addition, the performance evaluation of deep learning-based approaches has been compared to other related works. Therefore, an improved approach to RNN is proposed to recognize human actions. To classify the videos, a multilayer RNN with a specific type of LSTM is used to extract features from video sequences. The UCF-101 and UCF Sports human action recognition datasets are utilized in this study for both training and assessment. Test findings demonstrate that the suggested strategy achieved increased accuracy. Finally, the enhanced RNN model's total model accuracy in the UCF-101 dataset is 93.78% and 95.70% for the UCF Sport dataset.

An Optimal Feature Selection Method for Human Activity Recognition Using Multimodal Sensory Data

Recently, the research community has taken great interest in human activity recognition (HAR) due to its wide range of applications in different fields of life, including medicine, security, and gaming. The use of sensory data for HAR systems is most common because the sensory data are collected from a person’s wearable device sensors, thus overcoming the privacy issues being faced in data collection through video cameras. Numerous systems have been proposed to recognize some common activities of daily living (ADLs) using different machine learning, image processing, and deep learning techniques. However, the existing techniques are computationally expensive, limited to recognizing short-term activities, or require large datasets for training purposes. Since an ADL is made up of a sequence of smaller actions, recognizing them directly from raw sensory data is challenging. In this paper, we present a computationally efficient two-level hierarchical framework for recognizing long-term (composite) activities, which does not require a very large dataset for training purposes. First, the short-term (atomic) activities are recognized from raw sensory data, and the probabilistic atomic score of each atomic activity is calculated relative to the composite activities. In the second step, the optimal features are selected based on atomic scores for each composite activity and passed to the two classification algorithms: random forest (RF) and support vector machine (SVM) due to their well-documented effectiveness for human activity recognition. The proposed method was evaluated on the publicly available CogAge dataset that contains 890 instances of 7 composite and 9700 instances of 61 atomic activities. The data were collected from eight sensors of three wearable devices: a smartphone, a smartwatch, and smart glasses. The proposed method achieved the accuracy of 96.61% and 94.1% by random forest and SVM classifiers, respectively, which shows a remarkable increase in the classification accuracy of existing HAR systems for this dataset.

A Review of Recent Techniques for Human Activity Recognition: Multimodality, Reinforcement Learning, and Language Models

Human Activity Recognition (HAR) is a rapidly evolving field with the potential to revolutionise how we monitor and understand human behaviour. This survey paper provides a comprehensive overview of the state-of-the-art in HAR, specifically focusing on recent techniques such as multimodal techniques, Deep Reinforcement Learning and large language models. It explores the diverse range of human activities and the sensor technologies employed for data collection. It then reviews novel algorithms used for Human Activity Recognition with emphasis on multimodality, Deep Reinforcement Learning and large language models. It gives an overview of multimodal datasets with physiological data. It also delves into the applications of HAR in healthcare. Additionally, the survey discusses the challenges and future directions in this exciting field, highlighting the need for continued research and development to fully realise the potential of HAR in various real-world applications.

A novel rehabilitation action recognition approach using AF-LiteFormer from millimeter wave radar

Abstract Rehabilitation action recognition is a hot research topic in the medical field, which serves as the foundation for achieving remote healthcare, unsupervised exercise, intelligent home healthcare, and possesses extensive application value. Currently, vision-based action recognition methods are susceptible to limitations imposed by factors such as range of motion and environmental lighting during human motion capture. Due to its ability to effectively protect patient privacy and its immunity to lighting conditions, this paper proposed a millimeter-wave radar-based rehabilitation action recognition system, AF-LiteFormer. Firstly, EfficientFormerV2 is employed as the baseline, then a Lite-MSLA-FFN block-and-layer is designed to replace MHSA in EfficientFormerV2, which improves the diversity of attention and reduces computational complexity. Meanwhile, a Lite-Subsample block-and-layer is designed to replace the dual-path downsampling part that composed of attention downsampling and stride attention to realize global modeling and multi-scale learning.
Secondly, the iterative attention feature fusion (iAFF) mechanism is introduced to improve the recognition accuracy of rehabilitation actions. Finally, the effectiveness of the AF-LiteFormer model is validated on a self-collected rehabilitation action dataset and a publicly available micro-Doppler dataset. Experimental results show that the overall performance of the AF-LiteFormer model is better than the State-of-the-Art model (SOTA), the recognition accuracy of rehabilitation actions is as high as 99.7%, and it has strong generalization ability.