Field Of Human Activity Recognition Research Articles

A novel hybrid deep learning approach with GWO–WOA optimization technique for human activity recognition

The effectiveness of Human Activity Recognition (HAR) models can be largely attributed to the components derived from domain expertise. The classification system swiftly and effectively categorizes human physical activity by utilizing a comprehensive collection of variables. To construct HAR models and categorize different activities, deep learning algorithms have recently seen increased application in academic research for autonomously extracting features from raw sensory data. The primary goal of this study is to develop a strong and accurate activity classification model by first extracting features from an input dataset and then using deep learning models. In this research, we look at how well the model does in the field of HAR. Analyzing time-series data from smartphones and wearable sensors, algorithms based on deep learning (DL) have been used to forecast various human actions. Applying DL-based methods to time-series data presents a variety of challenges, notwithstanding their usage for activity identification. If the suggested approach is put into practice, the concerns mentioned before may be alleviated. To construct HAR classification techniques using data collected from wearable sensors, this study presents two Hybrid Learning Algorithms (HLA). This research employs Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) methods to teach the model sequential and spatial patterns. The process of selecting characteristics has been enhanced through the use of optimization techniques. Together, the Whale Optimization Algorithm and the Grey Wolf Optimizer form a Hybrid Optimization approach, which is introduced in this study. By combining and capitalizing the benefits of both approaches, the hybrid technique enhances the optimization process as a whole. This research may also assist individuals who have undergone amputations or experienced strokes in selecting an appropriate model and developing a personalized gait reference trajectory.

IMUGPT 2.0: Language-Based Cross Modality Transfer for Sensor-Based Human Activity Recognition

One of the primary challenges in the field of human activity recognition (HAR) is the lack of large labeled datasets. This hinders the development of robust and generalizable models. Recently, cross modality transfer approaches have been explored that can alleviate the problem of data scarcity. These approaches convert existing datasets from a source modality, such as video, to a target modality, such as inertial measurement units (IMUs). With the emergence of generative AI models such as large language models (LLMs) and text-driven motion synthesis models, language has become a promising source data modality as well - as shown in proof of concepts such as IMUGPT. In this work, we conduct a large-scale evaluation of language-based cross modality transfer to determine their effectiveness for HAR. Based on this study, we introduce two new extensions for IMUGPT that enhance its use for practical HAR application scenarios: a motion filter capable of filtering out irrelevant motion sequences to ensure the relevance of the generated virtual IMU data, and a set of metrics that measure the diversity of the generated data facilitating the determination of when to stop generating virtual IMU data for both effective and efficient processing. We demonstrate that our diversity metrics can reduce the effort needed for the generation of virtual IMU data by at least 50%, which opens up IMUGPT for practical use cases beyond a mere proof of concept.

Deep Learning for Skeleton-Based Human Activity Segmentation: An Autoencoder Approach

Automatic segmentation is essential for enhancing human activity recognition, especially given the limitations of publicly available datasets that often lack diversity in daily activities. This study introduces a novel segmentation method that utilizes skeleton data for a more accurate and efficient analysis of human actions. By employing an autoencoder, this method extracts representative features and reconstructs the dataset, using the discrepancies between the original and reconstructed data to establish a segmentation threshold. This innovative approach allows for the automatic segmentation of activity datasets into distinct segments. Rigorous evaluations against ground truth across three publicly available datasets demonstrate the method’s effectiveness, achieving impressive average annotation error, precision, recall, and F1-score values of 3.6, 90%, 87%, and 88%, respectively. This illustrates the robustness of the proposed method in accurately identifying change points and segmenting continuous skeleton-based activities as compared to two other state-of-the-art techniques: one based on deep learning and another using the classical time-series segmentation algorithm. Additionally, the dynamic thresholding mechanism enhances the adaptability of the segmentation process to different activity dynamics improving overall segmentation accuracy. This performance highlights the potential of the proposed method to significantly advance the field of human activity recognition by improving the accuracy and efficiency of identifying and categorizing human movements.

TCN-Inception: Temporal Convolutional Network and Inception modules for sensor-based Human Activity Recognition

The field of Human Activity Recognition (HAR) has experienced a significant surge in interest due to its essential role across numerous areas, including human–computer interaction (HCI), healthcare, smart homes, and various Internet of Things (IoT) applications. The power of deep learning methods in performing various classification tasks, including HAR, has been well-demonstrated. In light of this, our paper presents an efficient HAR system developed using a unique deep-learning architecture called TCN-Inception, which is designed for multivariate time series tasks like HAR data, by combining Temporal Convolutional Network (TCN) and Inception modules. The network starts with an Inception module that uses parallel convolution layers with various kernel sizes for feature extraction. It then includes a TCN module with dilated convolutions to grasp extended temporal patterns. Features are merged from different channels, and the use of residual connections and batch normalization improves training and deepens the architecture. We use four public datasets, UCI-HAR, MobiAct, Daphnet, and DSADS to assess the performance of the TCN-Inception model, and it obtains an average accuracy of 96.15%, 98.86%, 92.63%, and 99.50% for each dataset, respectively. Moreover, we compare the TCN-Inception to several deep learning frameworks to verify its performance. Finally, we implement an ablation study using several architectural configurations of the TCN-Inception model.

User-agnostic adaptation of human locomotion intent: Leveraging Teacher-Student-Learning and ensemble modeling

Wearable robot control in complicated contexts requires an understanding of human locomotion intent and behaviors. However, signals from human–robot interfaces are typically reliant on the user, leading to poor performance of the model trained on training user when applied to new end-point users. This study aims to address this problem by developing a novel Teacher-Student-Learning (TSL) approach using Heterogeneous Ensemble Hypotheses (HEH) to achieve unsupervised user-agnostic adaptation. The motivation behind using the Teacher-Student-Learning architecture is to leverage the diverse features extracted by HEH while maintaining computational efficiency. HEH reduces the difference between labeled training users and unlabeled end-point users by incorporating multiple diverse feature generators to extract a wide range of features, thereby increasing classification accuracy and reducing variance. However, this approach sacrifices efficiency due to the ensemble nature of HEH. To address this trade-off, the knowledge from HEH is distilled into a single network using TSL, ensuring precision and efficiency. The proposed approach is evaluated on two publicly available human locomotion datasets and a 2D moon dataset. Experimental results demonstrate the effectiveness of the proposed method, outperforming all other methods. Tested with three datasets, the proposed method can classify end-point users’ data with an average accuracy of 98.9%, 96.7%, and 96.9% while yielding a low processing time (1 ms). Compared to a benchmark method, the suggested strategy improves the average accuracy by 1.5% and 7.2% for categorizing the target users’ locomotor modes and stabilizes the learning curves. The proposed method represents a significant contribution to the field of human activity recognition and human intent prediction for human–robot interaction systems, enhancing the efficiency and generalization capacity of these systems.

Human Activity Recognition Based on Deep Learning and Micro-Doppler Radar Data.

Activity recognition is one of the significant technologies accompanying the development of the Internet of Things (IoT). It can help in recording daily life activities or reporting emergencies, thus improving the user's quality of life and safety, and even easing the workload of caregivers. This study proposes a human activity recognition (HAR) system based on activity data obtained via the micro-Doppler effect, combining a two-stream one-dimensional convolutional neural network (1D-CNN) with a bidirectional gated recurrent unit (BiGRU). Initially, radar sensor data are used to generate information related to time and frequency responses using short-time Fourier transform (STFT). Subsequently, the magnitudes and phase values are calculated and fed into the 1D-CNN and Bi-GRU models to extract spatial and temporal features for subsequent model training and activity recognition. Additionally, we propose a simple cross-channel operation (CCO) to facilitate the exchange of magnitude and phase features between parallel convolutional layers. An open dataset collected through radar, named Rad-HAR, is employed for model training and performance evaluation. Experimental results demonstrate that the proposed 1D-CNN+CCO-BiGRU model demonstrated superior performance, achieving an impressive accuracy rate of 98.2%. This outperformance of existing systems with the radar sensor underscores the proposed model's potential applicability in real-world scenarios, marking a significant advancement in the field of HAR within the IoT framework.

An overview of Graph Convolutional Networks in skeleton-based action recognition

The field of research is currently focused on human activity recognition. Hence, numerous pertinent literature reviews have expounded upon the multifaceted nature of data, the process of selecting feature vectors, and the advantages and disadvantages of classification networks. Graph Convolutional Networks (GCNs) have demonstrated significant efficacy in the domain of human action recognition. In recent years, with the rapid development of 3D skeleton data collection, a plethora of studies in action recognition based on skeleton data have emerged. Skeleton data consists of three-dimensional coordinates of multiple spatiotemporal skeletal joints, making it an effective representation of kinematics. It can be easily acquired through low-cost depth sensors and also directly extracted from two-dimensional images using video-based pose estimation algorithms, attracting widespread attention. As relational networks continue to evolve, GCNs have been applied to various fields, including human action recognition. GCNs have demonstrated significant advantages in feature extraction from skeleton data. However, using GCNs alone may have various limitations. Therefore, in recent years, many enhancement measures for GCNs have emerged. This review aims to summarize the research achievements of Graph Convolutional Network improvements in the field of human action recognition in recent years. It intends to assist future researchers in quickly organizing their research ideas, facilitating the generation of new results.

Comparative performance of machine learning models for the classification of human gait

The efficacy of human activity recognition (HAR) models mostly relies on the characteristics derived from domain expertise. The input of the classification algorithm consists of many characteristics that are utilized to accurately and effectively classify human physical activities. In contemporary research, machine learning techniques have been increasingly employed to automatically extract characteristics from unprocessed sensory input to develop models for Human Activity Recognition (HAR) and classify various activities. The primary objective of this research is to compare and contrast several machine learning models and determine a reliable and precise classification model for classifying activities. This study does a comparison analysis in order to assess the efficacy of 10 distinct machine learning models using frequently used datasets in the field of HAR. In this work, three benchmark public human walking datasets are being used. The research is conducted based on eight evaluating parameters. Based on the study conducted, it was seen that the machine learning classification models Random Forest, Extra Tree, and Light Gradient Boosting Machine had superior performance in all the eight evaluating parameters compared to specific datasets. Consequently, it can be inferred that machine learning significantly enhances performance within the area of Human Activity Recognition (HAR). This study can be utilized to provide suitable model selection for HAR-based datasets. Furthermore, this research can be utilized to facilitate the identification of various walking patterns for bipedal robotic systems.

A Rapid Response System for Elderly Safety Monitoring Using Progressive Hierarchical Action Recognition.

The global trend of population aging presents an urgent challenge in ensuring the safety and well-being of elderly individuals, especially those living alone due to various circumstances. A promising approach to this challenge involves leveraging Human Action Recognition (HAR) by integrating data from multiple sensors. However, the field of HAR has struggled to strike a balance between accuracy and response time. While technological advancements have improved recognition accuracy, complex algorithms often come at the expense of response time. To address this issue, we introduce an innovative asynchronous detection method called Rapid Response Elderly Safety Monitoring (RESAM), which relies on progressive hierarchical action recognition and multi-sensor data fusion. Through initial analysis of inertial sensor data using Kernel Principal Component Analysis (KPCA) and multi-class classifiers, we efficiently reduce processing time and lower the false-negative rate (FNR). The inertial sensor identification serves as a pre-filter, enabling the identification of filtered abnormal signals. Decision-level data fusion is then executed, incorporating skeleton image analysis based on ResNet and the inertial sensor data from the initial step. This integration enables the accurate differentiation between normal and abnormal behaviors. The RESAM method achieves an impressive 97.4% accuracy on the UTD-MHAD database with a minimal delay of 1.22 seconds. On our internally collected database, the RESAM system attains an accuracy of 99%, ranking among the most accurate state-of-the-art methods available. These results underscore the practicality and effectiveness of our approach in meeting the critical demand for swift and precise responses in healthcare scenarios.

A review on monitoring activities through human activity recognition

field of human activity recognition (HAR) is advancing with the integration of cutting-edge technologies such as artificial intelligence and sensor networks. These developments enable accurate detection and interpretation of human behavior, driven by advances in AI and machine learning. A growing number of electronic devices and applications contribute to a comprehensive understanding of the three pillars of HAR – AI, various devices and methodologies and their applications. This review consolidates HAR research and highlights its processes, applications, and device comparisons. Key findings include the importance of non-invasive, real-time sensor-based approaches, particularly in healthcare for the detection of behavioral abnormalities in the elderly. The study highlights Adam as an outstanding optimizer in HAR applications, who predicts continued expansion and development in various industries. particularly in the healthcare sector.

A Combination Model of Shifting Joint Angle Changes With 3D-Deep Convolutional Neural Network to Recognize Human Activity.

Research in the field of human activity recognition is very interesting due to its potential for various applications such as in the field of medical rehabilitation. The need to advance its development has become increasingly necessary to enable efficient detection and response to a wide range of movements. Current recognition methods rely on calculating changes in joint distance to classify activity patterns. Therefore, a different approach is required to identify the direction of movement to distinguish activities exhibiting similar joint distance changes but differing motion directions, such as sitting and standing. The research conducted in this study focused on determining the direction of movement using an innovative joint angle shift approach. By analyzing the joint angle shift value between specific joints and reference points in the sequence of activity frames, the research enabled the detection of variations in activity direction. The joint angle shift method was combined with a Deep Convolutional Neural Network (DCNN) model to classify 3D datasets encompassing spatial-temporal information from RGB-D video image data. Model performance was evaluated using the confusion matrix. The results show that the model successfully classified nine activities in the Florence 3D Actions dataset, including sitting and standing, obtaining an accuracy of (96.72 ± 0.83)%. In addition, to evaluate its robustness, this model was tested on the UTKinect Action3D dataset, obtaining an accuracy of 97.44%, proving that state-of-the-art performance has been achieved.

An improved human activity recognition technique based on convolutional neural network

A convolutional neural network (CNN) is an important and widely utilized part of the artificial neural network (ANN) for computer vision, mostly used in the pattern recognition system. The most important applications of CNN are medical image analysis, image classification, object recognition from videos, recommender systems, financial time series analysis, natural language processing, and human–computer interfaces. However, after the technological advancement in the power of computing ability and the emergence of huge quantities of labeled data provided through enhanced algorithms, nowadays, CNN is widely used in almost every area of study. One of the main uses of wearable technology and CNN within medical surveillance is human activity recognition (HAR), which must require constant tracking of everyday activities. This paper provides a comprehensive study of the application of CNNs in the classification of HAR tasks. We describe their enhancement, from their antecedents up to the current state-of-the-art systems of deep learning (DL). We have provided a comprehensive working principle of CNN for HAR tasks, and a CNN-based model is presented to perform the classification of human activities. The proposed technique interprets data from sensor sequences of inputs by using a multi-layered CNN that gathers temporal and spatial data related to human activities. The publicly available WISDM dataset for HAR has been used to perform this study. This proposed study uses the two-dimensional CNN approach to make a model for the classification of different human activities. A recent version of Python software has been used to perform the study. The rate of accuracy for HAR through the proposed model in this experiment is 97.20%, which is better than the previously estimated state-of-the-art technique. The findings of the study imply that using DL methods for activity recognition might greatly increase accuracy and increase the range of applications where HAR can be used successfully. We have also described the future research trends in the field of HAR in this article.

A Survey on Different Deep Learning Model for Human Activity Recognition based on Application

The field of human activity recognition (HAR) seeks to identify and classify an individual's unique movements or activities. However, recognizing human activity from video is a challenging task that requires careful attention to individuals, their behaviors, and relevant body parts. Multimodal activity recognition systems are necessary for many applications, including video surveillance systems, human-computer interfaces, and robots that analyze human behavior. This study provides a comprehensive analysis of recent breakthroughs in human activity classification, including different approaches, methodologies, applications, and limitations. Additionally, the study identifies several challenges that require further investigation and improvements. The specifications for an ideal human activity recognition dataset are also discussed, along with a thorough examination of the publicly available human activity classification datasets.

Ambient assisted living for frail people through human activity recognition: state-of-the-art, challenges and future directions.

Ambient Assisted Living is a concept that focuses on using technology to support and enhance the quality of life and well-being of frail or elderly individuals in both indoor and outdoor environments. It aims at empowering individuals to maintain their independence and autonomy while ensuring their safety and providing assistance when needed. Human Activity Recognition is widely regarded as the most popular methodology within the field of Ambient Assisted Living. Human Activity Recognition involves automatically detecting and classifying the activities performed by individuals using sensor-based systems. Researchers have employed various methodologies, utilizing wearable and/or non-wearable sensors, and employing algorithms ranging from simple threshold-based techniques to more advanced deep learning approaches. In this review, literature from the past decade is critically examined, specifically exploring the technological aspects of Human Activity Recognition in Ambient Assisted Living. An exhaustive analysis of the methodologies adopted, highlighting their strengths and weaknesses is provided. Finally, challenges encountered in the field of Human Activity Recognition for Ambient Assisted Living are thoroughly discussed. These challenges encompass issues related to data collection, model training, real-time performance, generalizability, and user acceptance. Miniaturization, unobtrusiveness, energy harvesting and communication efficiency will be the crucial factors for new wearable solutions.

KINECAL: A Dataset for Falls-Risk Assessment and Balance Impairment Analysis

The field of human action recognition has made great strides in recent years, much helped by the availability of a wide variety of datasets that use Kinect to record human movement. Conversely, progress towards the use of Kinect in clinical practice has been hampered by the lack of appropriate data. In particular, datasets that contain clinically significant movements and appropriate metadata. This paper proposes a dataset to address this issue, namely KINECAL. It contains the recordings of 90 individuals carrying out 11 movements, commonly used in the clinical assessment of balance. The dataset contains relevant metadata, including clinical labelling, falls history labelling and postural sway metrics. KINECAL should be of interest to researchers interested in the clinical use of motion capture and motion analysis.

Deep learning models for real-life human activity recognition from smartphone sensor data

Nowadays, the field of human activity recognition (HAR) is a remarkably hot topic within the scientific community. Given the low cost, ease of use and high accuracy of the sensors from different wearable devices and smartphones, more and more researchers are opting to do their bit in this area. However, until very recently, all the work carried out in this field was done in laboratory conditions, with very few similarities with our daily lives. This paper will focus on this new trend of integrating all the knowledge acquired so far into a real-life environment. Thus, a dataset already published following this philosophy was used. In this way, this work aims to be able to identify the different actions studied there. In order to perform this classification, this paper explores new designs and architectures for models inspired by the ones which have yielded the best results in the literature. More specifically, different configurations of Convolutional Neural Networks (CNN) and Long-Short Term Memory (LSTM) have been tested, but on real-life conditions instead of laboratory ones. It is worth mentioning that the hybrid models formed from these techniques yielded the best results, with a peak accuracy of 94.80% on the dataset used.

Blind Modalities for Human Activity Recognition.

Human Activity Recognition (HAR) has attracted considerable interest due to its ability to facilitate automation in various application areas, including but not limited to smart homes, active assisted living, and security. At present, optical modalities such as RGB, depth, and thermal imaging are prevalent in the field due to the effectiveness of deep learning algorithms like Convolutional Neural Networks (CNNs) and the abundance of publicly available image data. However, unconventional modalities such as radar, WiFi, seismic and environmental sensors are emerging as potential alternatives due to their capacity for contactless long-range sensing in spatially constrained environments and preservation of visual privacy. This work gives an overview of the HAR modalities landscape and discusses works that apply these emerging modalities in new and unconventional ways to inform researchers and practitioners about challenges and opportunities in the field of HAR.

A Jeap-BiLSTM Neural Network for Action Recognition

Human action recognition in videos is an important task in computer vision with applications in fields such as surveillance, human–computer interaction, and sports analysis. However, it is a challenging task due to the complex background changes and redundancy of long-term video information. In this paper, we propose a novel bi-directional long short-term memory method with attention pooling based on joint motion and difference entropy (JEAP-BiLSTM) to address these challenges. To obtain discriminative features, we introduce a joint entropy map that measures both the entropy of motion and the entropy of change. The Bi-LSTM method is then applied to capture visual and temporal associations in both forward and backward directions, enabling efficient capture of long-term temporal correlation. Furthermore, attention pooling is used to highlight the region of interest and to mitigate the effects of background changes in video information. Experiments on the UCF101 and HMDB51 datasets demonstrate that the proposed JEAP-BiLSTM method achieves recognition rates of 96.4% and 75.2%, respectively, outperforming existing methods. Our proposed method makes significant contributions to the field of human action recognition by effectively capturing both spatial and temporal patterns in videos, addressing background changes, and achieving state-of-the-art performance.

Wearable sensor-based human activity recognition with ensemble learning: a comparison study

<p>The spectacular growth of wearable sensors has provided a key contribution to the field of human activity recognition. Due to its effective and versatile usage and application in various fields such as smart homes and medical areas, human activity recognition has always been an appealing research topic in artificial intelligence. From this perspective, there are a lot of existing works that make use of accelerometer and gyroscope sensor data for recognizing human activities. This paper presents a comparative study of ensemble learning methods for human activity recognition. The methods include random forest, adaptive boosting, gradient boosting, extreme gradient boosting, and light gradient boosting machine (LightGBM). Among the ensemble learning methods in comparison, light gradient boosting machine and random forest demonstrate the best performance. The experimental results revealed that light gradient boosting machine yields the highest accuracy of 94.50% on UCI-HAR dataset and 100% on single accelerometer dataset while random forest records the highest accuracy of 93.41% on motion sense dataset.</p>

Review on Human Action Recognition in Smart Living: Sensing Technology, Multimodality, Real-Time Processing, Interoperability, and Resource-Constrained Processing.

Smart living, a concept that has gained increasing attention in recent years, revolves around integrating advanced technologies in homes and cities to enhance the quality of life for citizens. Sensing and human action recognition are crucial aspects of this concept. Smart living applications span various domains, such as energy consumption, healthcare, transportation, and education, which greatly benefit from effective human action recognition. This field, originating from computer vision, seeks to recognize human actions and activities using not only visual data but also many other sensor modalities. This paper comprehensively reviews the literature on human action recognition in smart living environments, synthesizing the main contributions, challenges, and future research directions. This review selects five key domains, i.e., Sensing Technology, Multimodality, Real-time Processing, Interoperability, and Resource-Constrained Processing, as they encompass the critical aspects required for successfully deploying human action recognition in smart living. These domains highlight the essential role that sensing and human action recognition play in successfully developing and implementing smart living solutions. This paper serves as a valuable resource for researchers and practitioners seeking to further explore and advance the field of human action recognition in smart living.