Human Activity Recognition System Research Articles

One-Dimensional Deep Residual Network with Aggregated Transformations for Internet of Things (IoT)-Enabled Human Activity Recognition in an Uncontrolled Environment

Human activity recognition (HAR) in real-world settings has gained significance due to the growth of Internet of Things (IoT) devices such as smartphones and smartwatches. Nonetheless, limitations such as fluctuating environmental conditions and intricate behavioral patterns have impacted the accuracy of the current procedures. This research introduces an innovative methodology employing a modified deep residual network, called 1D-ResNeXt, for IoT-enabled HAR in uncontrolled environments. We developed a comprehensive network that utilizes feature fusion and a multi-kernel block approach. The residual connections and the split–transform–merge technique mitigate the accuracy degradation and reduce the parameter number. We assessed our suggested model on three available datasets, mHealth, MotionSense, and Wild-SHARD, utilizing accuracy metrics, cross-entropy loss, and F1 score. The findings indicated substantial enhancements in proficiency in recognition, attaining 99.97% on mHealth, 98.77% on MotionSense, and 97.59% on Wild-SHARD, surpassing contemporary methodologies. Significantly, our model attained these outcomes with considerably fewer parameters (24,130–26,118) than other models, several of which exceeded 700,000 parameters. The 1D-ResNeXt model demonstrated outstanding effectiveness under various ambient circumstances, tackling a significant obstacle in practical HAR applications. The findings indicate that our modified deep residual network presents a viable approach for improving the dependability and usability of IoT-based HAR systems in dynamic, uncontrolled situations while preserving the computational effectiveness essential for IoT devices. The results significantly impact multiple sectors, including healthcare surveillance, intelligent residences, and customized assistive devices.

Cross-Domain HAR: Few Shot Transfer Learning for Human Activity Recognition

The ubiquitous availability of smartphones and smartwatches with integrated inertial measurement units (IMUs) enables straightforward capturing of human activities through collecting movement data. For specific applications of sensor based human activity recognition (HAR), however, logistical challenges and burgeoning costs render especially the ground truth annotation of such data a difficult endeavor, resulting in limited scale and diversity of datasets available for deriving effective HAR systems and less than ideal recognition capabilities. Transfer learning, i.e., leveraging publicly available labeled datasets to first learn useful representations that can then be fine-tuned using limited amounts of labeled data from a target domain, can alleviate some of the performance issues of contemporary HAR systems. Yet they can fail when the differences between source and target conditions are too large and / or only few samples from a target application domain are available – each of which are typical challenges in real-world human activity recognition scenarios. In this paper, we present an approach for economic use of publicly available labeled HAR datasets for effective transfer learning. We introduce a novel transfer learning framework–Cross-Domain HAR–which follows the teacher-student self-training paradigm to more effectively recognize activities with very limited label information. It bridges conceptual gaps between source and target domains, including sensor locations and type of activities. Cross-Domain HAR enables substantial performance improvements over the state-of-the-art in sensor-based HAR scenarios. Through our extensive experimental evaluation on a range of benchmark datasets we specifically demonstrate the effectiveness of our approach for practically relevant few shot activity recognition scenarios. We also present a detailed analysis into how the individual components of our framework affect downstream performance and provide practical suggestions for using the framework in real-world applications.

State-of-the-art in human activity recognition based on inertial measurement unit sensors: survey and applications

Human activity recognition systems using wearable sensors is an important issue in pervasive computing, which applies to various domains related to healthcare, context aware and pervasive computing, sports, surveillance and monitoring, and the military. Three approaches can be considered for activity recognition: video sensor-based, physical sensor-based, and environmental sensor-based. This paper investigates the related work regarding the physical sensor-based approaches to motion processing. In this paper, a wide range of artificial intelligence models, from single classifications to methods based on deep learning, have been reviewed. The human activity detection accuracy of different methods, under natural and experimental conditions poses several challenges. These challenges cause problems regarding the accuracy and applicability of the proposed methods. This paper analyzes the methods, challenges, approaches, and future work. The goal of this paper is to establish a clear distinction in the field of motion detection using inertial sensors.

Human Activity Recognition System Based on Continuous Learning with Human Skeleton Information

Human Activity Recognition System Based on Continuous Learning with Human Skeleton Information

AUTOMATED INDOOR ACTIVITY MONITORING FOR ELDERLY AND VISUALLY IMPAIRED PEOPLE USING QUANTUM SALP SWARM ALGORITHM WITH MACHINE LEARNING

Elderly and vision-impaired indoor activity monitoring uses sensor technology to monitor interaction and movement in the living area. This system can identify deviations from regular patterns, provide alerts, and ensure safety in case of emergencies or potential risks. These solutions enhance quality of life by promoting independent living while offering peace of mind to loved ones and caregivers. Visual impairments have become the chief part of these problems over the past few years. Usually, visually impaired people seek help from others to keep doing their daily tasks. An automated human activity recognition (HAR) technique could enhance a vision-impaired person’s transaction activity and safe movement. HAR using deep learning (DL) includes training neural networks to automatically detect and classify various activities carried out by humans based on sensor information. By leveraging DL approaches, the HAR system could accurately identify and classify activities, including sitting, standing, walking, or running, from the information captured by sensors or wearable devices. This system is integral to smart home automation, healthcare, and sports analytics. This study develops an Automated Fractal Quantum Salp Swarm Algorithm with Machine Learning based HAR (QSSA-MLHAR) for assisting elderly and visually impaired people. The QSSA-MLHAR technique uses sensory input data to identify human activities in diverse classes. In the QSSA-MLHAR technique, the min–max scalar is primarily used to scale the input data. Besides, the QSSA-MLHAR technique applies a mixed extreme learning machine (M-ELM) model to classify various human actions. The QSSA-based hyperparameter tuning process is used to improve the detection results of the M-ELM system. A wide range of simulations was performed to ensure the better performance of the QSSA-MLHAR method. The comprehensive outcomes assured that the QSSA-MLHAR technique performs well compared to other models.

Retraction Note: Rule-based multi-view human activity recognition system in real time using skeleton data from RGB-D sensor

Retraction Note: Rule-based multi-view human activity recognition system in real time using skeleton data from RGB-D sensor

An IoT solution that detects human physical activities to support health assessment based on sensors and supervised learning algorithms

At the moment, human physical activities can be detected by activity recognition systems to help with health status evaluation. One of the key issues in human activity recognition (HAR) systems is cost, speed, and accuracy. With the advancement of mobile devices and increasingly sophisticated microprocessors, along with the integration of various sensors including accelerometers, human activity recognition has become easier and faster. However, the accuracy of activity classification based on accelerometer data depends on many different factors, which can lead to potential deviations in results. Through various experiments, we have found that selecting optimal features is crucial for the performance of classification. In this paper, we propose a simple feature set that yields very promising results with an accuracy of up to 99%.

Exploring the Efficacy of a Dynamic Harness System on Gross Motor Development and Motivation for Infants With Down Syndrome: A Pilot Study.

To explore the benefits of a Partial Body Weight Support (PBWS) harness system within a play enriched environment on gross motor development and mastery motivation of infants with Down Syndrome (DS). A randomized crossover study with 17 pre-walking infants with DS in two conditions-play with or without the harness engaged-each for 3×/week over 3weeks with a 1-week washout. Assessments took place at baseline, crossover, and completion. Statistically and clinically significant changes were evident on the Gross Motor Function Measure-88; however, there were no significant changes in parent-reported mastery motivation. The combination of PBWS harness system support and high frequency-facilitated play within an enriched play environment positively affected gross motor development. The intervention did not impact mastery motivation skills, and the direct impact of the harness remains unclear.

Commentary on "Effects of a Dynamic Harness System on Motor Development of Infants with DS: Pilot Study".

Commentary on "Effects of a Dynamic Harness System on Motor Development of Infants with DS: Pilot Study".

An Optimal Feature Selection Method for Human Activity Recognition Using Multimodal Sensory Data

Recently, the research community has taken great interest in human activity recognition (HAR) due to its wide range of applications in different fields of life, including medicine, security, and gaming. The use of sensory data for HAR systems is most common because the sensory data are collected from a person’s wearable device sensors, thus overcoming the privacy issues being faced in data collection through video cameras. Numerous systems have been proposed to recognize some common activities of daily living (ADLs) using different machine learning, image processing, and deep learning techniques. However, the existing techniques are computationally expensive, limited to recognizing short-term activities, or require large datasets for training purposes. Since an ADL is made up of a sequence of smaller actions, recognizing them directly from raw sensory data is challenging. In this paper, we present a computationally efficient two-level hierarchical framework for recognizing long-term (composite) activities, which does not require a very large dataset for training purposes. First, the short-term (atomic) activities are recognized from raw sensory data, and the probabilistic atomic score of each atomic activity is calculated relative to the composite activities. In the second step, the optimal features are selected based on atomic scores for each composite activity and passed to the two classification algorithms: random forest (RF) and support vector machine (SVM) due to their well-documented effectiveness for human activity recognition. The proposed method was evaluated on the publicly available CogAge dataset that contains 890 instances of 7 composite and 9700 instances of 61 atomic activities. The data were collected from eight sensors of three wearable devices: a smartphone, a smartwatch, and smart glasses. The proposed method achieved the accuracy of 96.61% and 94.1% by random forest and SVM classifiers, respectively, which shows a remarkable increase in the classification accuracy of existing HAR systems for this dataset.

Transition-aware human activity recognition using an ensemble deep learning framework

Understanding human activities in daily life is of utmost importance, especially in the context of personalized and adaptive ubiquitous learning. Although existing HAR systems perform well-identifying activities based on their inter-spatial and temporal relationships, they lack in identifying the importance of accurately detecting postural transitions that not only enhance the activity recognition rate and reduced the error rate but also provides added motivation to explore and develop hybrid models. It's in this context we propose an ensemble approach of 1D-CNN and LSTM for the task of postural transition recognition, facilitated by wireless computing and wearable sensors. The proliferation of achieving ubiquitous learning will ultimately lead to the creation of adaptive devices enabled by various data analysis and relation learning techniques. Our approach is one of the methods that can be incorporated to enable seamless learning and acquire correlations with adaptive learning techniques. The experimental results on testing datasets including newly produced HAPT (Human Activities and Postural Transitions) show better classification accuracy than existing state-of-the-art HAR approaches (97.84% for transitional activities and 99.04% for dynamic human activities) indicating the capability of the model in ubiquitous learning scenarios and personalized and adaptive human learning environments.

Learning Heterogeneous Spatial-Temporal Context for Skeleton-Based Action Recognition.

Graph convolution networks (GCNs) have been widely used and achieved fruitful progress in the skeleton-based action recognition task. In GCNs, node interaction modeling dominates the context aggregation and, therefore, is crucial for a graph-based convolution kernel to extract representative features. In this article, we introduce a closer look at a powerful graph convolution formulation to capture rich movement patterns from these skeleton-based graphs. Specifically, we propose a novel heterogeneous graph convolution (HetGCN) that can be considered as the middle ground between the extremes of (2 + 1)-D and 3-D graph convolution. The core observation of HetGCN is that multiple information flows are jointly intertwined in a 3-D convolution kernel, including spatial, temporal, and spatial-temporal cues. Since spatial and temporal information flows characterize different cues for action recognition, HetGCN first dynamically analyzes pairwise interactions between each node and its cross-space-time neighbors and then encourages heterogeneous context aggregation among them. Considering the HetGCN as a generic convolution formulation, we further develop it into two specific instantiations (i.e., intra-scale and inter-scale HetGCN) that significantly facilitate cross-space-time and cross-scale learning on skeleton graphs. By integrating these modules, we propose a strong human action recognition system that outperforms state-of-the-art methods with the accuracy of 93.1% on NTU-60 cross-subject (X-Sub) benchmark, 88.9% on NTU-120 X-Sub benchmark, and 38.4% on kinetics skeleton.

WISNet: A deep neural network based human activity recognition system

Nowadays, Human Activity Recognition (HAR) is a key research area with many ubiquitous innovative solutions, where both accelerometer and gyroscope data provide information about an observed person’s physical activity. HAR offers a diverse variety of important applications, including healthcare, burglary detection, workplace monitoring, and emergency identification. Traditional recognition approaches rely on extracting handmade features from the obtained data to identify the typeof human action. Additionally, the efficacy of these works is dependent upon the specific customized features that are chosen. One potential approach to tackle this issue is to utilize Convolutional Neural Networks (CNN) to automatically learn the relevant features. In this paper, we propose a deep learning model, WISNet, a custom 1D-CNN approach to recognize six complex human activities: Jogging, Walking Downstairs, Sitting, Standing, Walking and Climbing Upstairs. The model includes a Convolved Normalized Pooled (CNPM) Block to generate significant features from the initial layers. An Identity and Basic (IDBN) Block is incorporated to extract residual progressive features for capturing complex sequential data dependencies. Channel and Spatial attention (CASb) Block is integrated with the network to prioritize or minimize essential features based on relative weights. The proposed WISNet model achieved an enhanced accuracy and F1-score of 96.41 % and 0.95 for the HAR dataset by surpassing the existing transfer learning architectures such as Gated Recurrent Units (GRU), Long Short-Term Memory (LSTM), and Recurrent Neural Network (SimpleRNN). By strategically integrating the CNPM, IDBN, and CASb blocks, this study aims to tackle distinct challenges encountered in the classification process by enhancing the discernment of features essential for the precise identification of multi-class human activity recognition. The seamless integration of these blocks within the model plays a pivotal role in elevating the overall performance of the WISNet architecture. The work also validates WISNet with similar open-source datasets (UCI-HAR and KU-HAR) and dissimilar open-source datasets (Sleep state detection, Fall detection, and ECG Heartbeat).

Experience: A Comparative Analysis of Multivariate Time-Series Generative Models: A Case Study on Human Activity Data

Human activity recognition (HAR) is an active research field that has seen great success in recent years due to advances in sensory data collection methods and activity recognition systems. Deep artificial intelligence (AI) models have contributed to the success of HAR systems lately although still suffering from limitations such as data scarcity, the high costs of labelling data instances and datasets’ imbalance and bias. The temporal nature of human activity data, represented as time series data, impose an additional challenge to using AI models in HAR because most state-of-the-art models do not account for the time component of the data instances. These limitations have inspired the time-series research community to design generative models for sequential data but very little work has been done to evaluate the quality of such models. In this work, we conduct a comparative quality analysis of three generative models for time-series data, using a case study in which we aim to generate sensory human activity data from a seed public dataset. Additionally, we adapt and clearly explain four evaluation methods of synthetic time-series data from the literature and apply them to assess the quality of the synthetic activity data we generate. We show experimentally that high quality human activity data can be generated using deep generative models, and the synthetic data can thus be used in HAR systems to augment real activity data. We also demonstrate that the chosen evaluation methods effectively ensure that the generated data meets the essential quality benchmarks of realism, diversity, coherence and utility. Our findings suggest that using deep generative models to produce synthetic human activity data can potentially address challenges related to data scarcity, biases, and expensive labeling. This holds promise for enhancing the efficiency and reliability of HAR systems.

Exposing Data Leakage in Wi-Fi CSI-Based Human Action Recognition: A Critical Analysis

Wi-Fi channel state information (CSI)-based human action recognition systems have garnered significant interest for their non-intrusive monitoring capabilities. However, the integrity of these systems can be compromised by data leakage, particularly when improper dataset partitioning strategies are employed. This paper investigates the presence and impact of data leakage in three published Wi-Fi CSI-based human action recognition methods that utilize deep learning techniques. The original studies achieve precision rates of 95% or higher, attributed to the lack of human-based dataset splitting. By re-evaluating these systems with proper subject-based partitioning, our analysis reveals a substantial decline in performance, underscoring the prevalence of data leakage. This study highlights the critical need for rigorous dataset management and evaluation protocols to ensure the development of robust and reliable human action recognition systems. Our findings advocate for standardized practices in dataset partitioning to mitigate data leakage and enhance the generalizability of Wi-Fi CSI-based models.

Edge human activity recognition using federated learning on constrained devices

Human Activity Recognition (HAR) using wearable Internet of Things (IoT) devices represents a well investigated researched field encompassing various application domains. Many current approaches rely on cloud-based methodologies for gathering data from diverse users, resulting in the creation of extensive training datasets. Although this strategy facilitates the application of powerful Machine Learning (ML) techniques, it raises significant privacy concerns, which can become particularly severe given the sensitivity of HAR data. Moreover, the labeling process can be extremely time-consuming and even more challenging for IoT wearable devices due to the absence of efficient input systems. In this paper, we address both aforementioned challenges by designing, implementing, and validating edge-based Human Activity Recognition (HAR) systems that operate on resource-constrained IoT devices, which relies on the utilization of Self-Organizing Maps (SOM) for activity detection. We incorporate a feature selection process before training to reduce data dimensionality and, consequently, the SOM size, aligning with the resource limitations of wearable IoT devices. Additionally, we explore the application of Federated Learning (FL) techniques for HAR tasks, enabling new users to leverage SOM models trained by others on their respective datasets. Our federated Extreme Edge (EE)-aware HAR system is implemented on a wearable IoT device and rigorously tested against state-of-the-art and experimental datasets. The results demonstrate that our C++-based SOM implementation achieves a consistent reduction in model size compared to state-of-the-art approaches. Furthermore, our findings highlight the effectiveness of the FL-based approach in overcoming personalized training challenges, particularly in onboarding scenarios.

Lightweight human activity recognition system for resource constrained environments

Lightweight human activity recognition system for resource constrained environments

Human activity recognition system using wearable accelerometers for classification of leg movements: a first, detailed approach

Abstract. A human activity recognition (HAR) system carried by masseurs for controlling a therapy table via different movements of legs or hip is studied. This work starts with a survey on HAR systems using the sensor position named “trouser pockets”. Afterwards, in the experiments, the impacts of different hardware systems, numbers of subjects, data generation processes (online streams/offline data snippets), sensor positions, sampling rates, sliding window sizes and shifts, feature sets, feature elimination processes, operating legs, tag orientations, classification processes (concerning method, parameters and an additional smoothing process), numbers of activities, training databases, and the use of a preceding teaching process on the classification accuracy are examined to get a thorough understanding of the variables influencing the classification quality. Besides the impacts of different adjustable parameters, this study also serves as an advisor for the implementation of classification tasks. The proposed system has three operating classes: do nothing, pump therapy table up or pump therapy table down. The first operating class consists of three activity classes (go, run, massage) such that the whole classification process exists with five classes. Finally, using online data streams, a classification accuracy of 98 % could be achieved for one skilled subject and about 90 % for one randomly chosen subject (mean of 1 skilled and 11 unskilled subjects). With the LOSO (leave-one-subject-out) technique for 12 subjects, up to 86 % can be attained. With our offline data approach, we get accuracies of 98 % for 12 subjects and up to 100 % for 1 skilled subject.

Enhancing Human Activity Recognition through Integrated Multimodal Analysis: A Focus on RGB Imaging, Skeletal Tracking, and Pose Estimation.

Human activity recognition (HAR) is pivotal in advancing applications ranging from healthcare monitoring to interactive gaming. Traditional HAR systems, primarily relying on single data sources, face limitations in capturing the full spectrum of human activities. This study introduces a comprehensive approach to HAR by integrating two critical modalities: RGB imaging and advanced pose estimation features. Our methodology leverages the strengths of each modality to overcome the drawbacks of unimodal systems, providing a richer and more accurate representation of activities. We propose a two-stream network that processes skeletal and RGB data in parallel, enhanced by pose estimation techniques for refined feature extraction. The integration of these modalities is facilitated through advanced fusion algorithms, significantly improving recognition accuracy. Extensive experiments conducted on the UTD multimodal human action dataset (UTD MHAD) demonstrate that the proposed approach exceeds the performance of existing state-of-the-art algorithms, yielding improved outcomes. This study not only sets a new benchmark for HAR systems but also highlights the importance of feature engineering in capturing the complexity of human movements and the integration of optimal features. Our findings pave the way for more sophisticated, reliable, and applicable HAR systems in real-world scenarios.

Humanactivity Recognition using Deep Learning

Human activity recognition (HAR) plays a crucial role in various fields, including healthcare, surveillance, and human-computer interaction. This study explores the application of deep learning techniques for accurate and efficient human activity recognition. Leveraging the capabilities of deep neural networks, specifically convolutional neural networks (CNNs) and recurrent neural networks (RNNs), the proposed approach aims to capture both spatial and temporal features from sensor data. The dataset utilized in this research comprises multi-model sensor inputs, such as accelerometer and gyroscope readings, collected from wearable devices. The deep learning model is designed to automatically learn hierarchical representations of the raw sensor data, enabling robust feature extraction and discrimination between different human activities. Transfer learning is employed to enhance model generalization across diverse activity categories and varying sensor setups. Experimental evaluations are conducted on real-world datasets, demonstrating the effectiveness of the proposed deep learning framework in accurately classifying activities, including walking, running, sitting, and standing. Comparative analyses against traditional machine learning methods underscore the superior performance and adaptability of deep learning in handling complex and dynamic activity patterns. The results showcase the potential of deploying deep learning models in real-time human activity recognition systems, highlighting their scalability and efficiency. The study contributes to the advancement of HAR methodologies, paving the way for the development of more reliable and robust systems in applications such as healthcare monitoring, assisted living, and smart environments.