Human Activity Classification Research Articles

Classification of human actions using 3D skeleton data: A performance comparison between classical machine learning and deep learning models

Classification of human actions using 3D skeleton data: A performance comparison between classical machine learning and deep learning models

Enhanced federated recognition mechanism based on spatial-temporal model with split learning for multi-view human activity classification in edge intelligent network

Enhanced federated recognition mechanism based on spatial-temporal model with split learning for multi-view human activity classification in edge intelligent network

An Accurate Classification for Human Sport Activity Recognition from Sensor Data Using Hybrid Machine Learning Models

The significance of human sports activities recognition (HSAR) in many computing applications, including sports analytics, smart surroundings, and healthcare monitoring is huge. The current use of vision sensors for HSAR is a difficult task due to the intricate movements involved in sports and fitness workouts, as well as fluctuations in illumination conditions. Therefore, a comprehensive review of machine learning algorithms (such as XGBoost, Random Forest (RF), Decision Trees (DT), and K-Nearest Neighbors (KNN)) and their usage for human sports activities classification based on sensor data called ML-HSAR is presented in this article. A dataset containing 19 different human activities gathered from 8 different people through different types of sensors was used for this work; the dataset contains activities like sitting and standing, as well as other activities that require serious movements, such as running, cycling, and playing basketball. This study primarily aims to determine the performance of the considered ML models in accurately classifying these activities found in the dataset. The training of the algorithms and the subsequent evaluation were done on the segmented sensor data, effectively leveraging the dataset’s temporal aspect. In addition to the presented results for each of these algorithms, the limitations of each algorithm and their strengths in handling the complexities involved in HSAR were also presented. The contribution of this analysis is towards the understanding of the most appropriate ML models for HSAR tasks; it also offers valuable insights for future research and practical applications.

Optimal Sensor Placement and Multimodal Fusion for Human Activity Recognition in Agricultural Tasks

This study examines the impact of sensor placement and multimodal sensor fusion on the performance of a Long Short-Term Memory (LSTM)-based model for human activity classification taking place in an agricultural harvesting scenario involving human-robot collaboration. Data were collected from twenty participants performing six distinct activities using five wearable inertial measurement units placed at various anatomical locations. The signals collected from the sensors were first processed to eliminate noise and then input into an LSTM neural network for recognizing features in sequential time-dependent data. Results indicated that the chest-mounted sensor provided the highest F1-score of 0.939, representing superior performance over other placements and combinations of them. Moreover, the magnetometer surpassed the accelerometer and gyroscope, highlighting its superior ability to capture crucial orientation and motion data related to the investigated activities. However, multimodal fusion of accelerometer, gyroscope, and magnetometer data showed the benefit of integrating data from different sensor types to improve classification accuracy. The study emphasizes the effectiveness of strategic sensor placement and fusion in optimizing human activity recognition, thus minimizing data requirements and computational expenses, and resulting in a cost-optimal system configuration. Overall, this research contributes to the development of more intelligent, safe, cost-effective adaptive synergistic systems that can be integrated into a variety of applications.

Smartphone-sensor-based human activities classification for forensics: a machine learning approach

The accurate classification of human activities in crime scenes during forensics (criminalistics) is of utmost importance in classifying suspicious and unlawful activities, easing their acceptability and interpretability by judges during legal procedures in courts or by other non-experts in the field of forensics. This paper implements machine learning (ML) algorithms: support vector machine (SVM) and decision tree (DT), to demonstrate with a high accuracy, how data emanating from smartphones’ sensors reveal and isolate relevant information about static and dynamic human activities in criminalistics. Smartphones’ data from five different sensors (accelerometer, gravity, orientation, Gyroscope and light), related to ten recurrent crime scenes activities, grouped into three classes of events (normal, felony and none-felony events) are classified by the proposed algorithms, with novelty being the classification decisions based on the entire period of the events and not instantaneous decision makings. Three independent data-subsets were made, with permutations done between them and at each time, two sets used for training and the third set used for testing. Time- and frequency-domain features were initially used separately and then combined for the model training and testing. The best average training accuracies of 100% and 97.8% were obtained for the DT and SVM, respectively, and the testing accuracies of 89.1% were obtained for both algorithms. We therefore believe that these results will serve as a solid persuasive and convincing argument to judges and non-experts of the field of forensics to accept and easily interpret computer-aided classification of suspicious activities emanating from criminalistic studies.

SRCNN: Stacked-Residual Convolutional Neural Network for Improving Human Activity Classification Based on Micro-Doppler Signatures of FMCW Radar

Current methods for daily human activity classification primarily rely on optical images from cameras or wearable sensors. Despite their high detection reliability, camera-based approaches suffer from several drawbacks, such as low-light conditions, limited range, and privacy concerns. To address these limitations, this article proposes the use of a frequency-modulated continuous wave radar sensor for activity recognition. A stacked-residual convolutional neural network (SRCNN) is introduced to classify daily human activities based on the micro-Doppler features of returned radar signals. The model employs a two-layer stacked-residual structure to reuse former features, thereby improving the classification accuracy. The model is fine-tuned with different hyperparameters to find a trade-off between classification accuracy and inference time. Evaluations are conducted through training and testing on both simulated and measured datasets. As a result, the SRCNN model with six stacked-residual blocks and 64 filters achieves the best performance, with accuracies exceeding 95% and 99% at 0 dB and 10 dB, respectively. Remarkably, the proposed model outperforms several state-of-the-art CNN models in terms of classification accuracy and execution time on the same datasets.

Mitigating human fall injuries: A novel system utilizing 3D 4-stream convolutional neural networks and image fusion

Unintentional human falls, especially in seniors, lead to serious injuries, fatalities, and reduced standard of life. Vision-based fall detection methods have demonstrated their usefulness in timely fall response, helping to lessen such injuries. This paper presents an automated vision-based fall detection system that triggers immediate fall reporting. By incorporating human segmentation and image fusion in the pre-processing stage, the system enhances the accuracy of human action classification, thereby ensuring precise fall alerts. It further employs the innovative 4-stream 3D convolutional neural network (4S-3DCNN) model to learn different but consecutive spatial and temporal features. The system processes video input or live surveillance, segmenting human presence every 32 frames using a fine-tuned deep-learning model and applying a three-level image fusion to accentuate movement differences. This technique produces four pre-processed images, input to the 4S-3DCNN model for classification. Consecutive detection of “Falling” and “Fallen” actions triggers an alert for immediate intervention. The original 4S-3DCNN model is an end-to-end trained deep learning model with a fully connected layer serving as a classifier. The research also evaluates the performance of combining the 4S-3DCNN model with Autoencoders and Support Vector Machines (SVM) networks as classifiers. The SVM classifier demonstrated ideal fall detection performance with 100% accuracy using the MCFD, URFD, and Le2i FDD datasets. The proposed system is vital for detecting and preventing falls and reducing healthcare expenses and productivity losses.

Deep learning and quantization for accurate and efficient multi-target radar inference of moving targets

Real-time, radar-based human activity and target classification is useful for wide-area ground surveillance. However, the feasibility of deploying deep learning (DL) models in radar-based systems with limited computational resources remains unexplored. This paper investigated the effect of quantization on model throughput and accuracy for deployment in radar systems. A seven-layer residual network was proposed to classify ground-moving targets and achieved a test accuracy of 87.72%. The model was then quantized to 16-bit and 8-bit precision, resulting in a 3.8 times speedup in inference throughput, with less than a 0.4% drop in test and validation accuracy. The results showed that quantization can improve inference throughput with a negligible decrease in target classification accuracy. The increase in throughput and reduction in computational expense that comes with quantization promotes the feasibility of the deployment of DL models in systems with limited computational resources. The findings of this paper hold significant promise for the successful use of quantized models in modern radar systems, while adhering to stringent size, weight and power consumption constraints.

Enhance micro-Doppler signatures-based human activity classification accuracy of FMCW radar using the threshold method

Nowadays, radar-based human activity classification is being widely adopted in healthcare systems due to its benefits in terms of personal privacy compliance, non-contact sensing, and being unaffected by weather conditions. This study proposes a threshold method in the pre-processing stage to improve human activity classification accuracy by determining the region of meaningful information (RMI) on the spectrogram. Initially, a mask function, which is created by a certain threshold value, is applied to the input spectrogram to highlight the RMI from the micro-Doppler (m-D) signatures. Only the highlighted RMI on the spectrogram is retained as input to the classifiers. Then, five Convolutional Neural Networks (CNNs) of varying complexity are employed to extract features, identify activities, and assess the effectiveness of the suggested approach. The experimental results demonstrate that the suggested approach has enhanced classification accuracy by up to 11% when compared to the original unprocessed dataset.

Multi-head CNN-based activity recognition and its application on chest-mounted sensor-belt

In recent years, a great deal of research has been done on the identification, monitoring, and classification of human activities. Human activity recognition (HAR) is a term commonly used to describe the automatic identification of physical activities. For activity recognition, there are primarily vision-based and sensor-based methods available. The computer vision-based method is generally effective in lab settings, but because of clutter, fluctuating light levels, and contrast, it may not perform well in real-world scenarios. Continuous monitoring and analysis of physiological signals obtained from heterogeneous sensors attached to an individual’s body is required to realise sensor-based HAR systems. Most of the previous research in human activity recognition (HAR) is biased along with feature engineering and pre-processing which requires a good amount of domain knowledge. Application-specific modelling and time-taking methods are involved in these approaches. In this work, the multi-head convolutional neural network-based human activity recognition framework is proposed where automatic feature extraction and classification are involved in the form of an end-to-end classification approach. Experiments of this approach are performed by taking raw wearable sensor data with few pre-processing steps and without the involvement of a handcrafted feature extraction technique. 99.23% and 93.55% accuracy are obtained on the WISDM and UCI-HAR datasets which denoted the much improvement in the assessment of HAR over other similar approaches. The model is also tested on locally collected data from a chest mounted belt with fabric sensors and an accuracy of 87.14% has been achieved on that data.

Визуализация и классификация движений человека на основе скелетной структуры: Нейросетевой подход к анализу спортивных упражнений и сравнение методологий

The authors of the paper review and compare different existing approaches to Human Action Recognition (HAR), analyze the advantages and disadvantages of platforms for extracting human skeletal structure from video stream, and evaluate the importance of visual representation in the motion analysis process. This paper presents an example implementation of one of the approaches to HAR based on the use of interpretability and visual expressiveness inherent in skeletal structures. In this work, an ad hoc network with Long Short-Term Memory (LSTM) for human activity classification is designed and implemented, which has been trained and tested in the domain of sports exercises. LSTM incorporation of memory cells and gating mechanisms not only mitigates the vanishing gradient problem but also enables LSTMs to selectively retain and utilize relevant information over extended sequences, making them highly effective in tasks with complex temporal dependencies. The problem with a fading gradient is quite common in deep neural networks and is that if the error is back propagated during the training of the network, the gradient can decrease strongly as it travels through the layers of the network to the initial layers. This can lead to the fact that the weights in the initial layers are practically not updated, which makes training of these layers impossible or slows down its process. The resulting solution can be used to create a real-time virtual fitness assistant. The resulting solution can be used to create a real-time virtual fitness assistant. In addition, this approach will make it possible to create interactive training applications with visualization of human skeletal structure, motion analysis and monitoring systems in the field of medicine and rehabilitation, as well as for the development of security systems with access control based on the analysis of visual data on the movement of human body parts.

A Systematic Literature Review on Human Activity Recognition

Human Activity Recognition (HAR) plays a significant role in several fields by automatically identifying and monitoring human activities using advanced techniques. It enhances safety, improves healthcare services, optimizes fitness routines, and enables context-aware applications in various fields. HAR contributes to a more efficient and intelligent interaction between humans and technology. It has emerged as an essential research domain with applications in healthcare, smart environments, and human-computer interaction. This study aims to provide a comprehensive survey of the evolving landscape of HAR, including key methodologies, techniques, and trends in existing research. The study discusses various applications of HAR and their significance in modern smart environments. The survey also highlights different types of HAR and data collection techniques. Additionally, it explores various methods for analyzing the collected data and provides a comprehensive analysis of existing human activity classification datasets. It offers valuable insights into understanding the strengths and limitations of various HAR techniques. The study also discusses various challenges and future directions for HAR.

WIFI based human activity recognition using multi-head adaptive attention mechanism

Human Activity Recognition (HAR) utilizing Channel State Information (CSI) extracted from WiFi signals has garnered substantial interest across various domains and applications. This field’s potential paths and applications extend beyond CSI-based HAR and include smart homes, assisted living, security, gaming, surveillance, and context-aware computing. The ability of deep learning algorithms to effectively process and interpret CSI data opens up new possibilities for accurate and robust human activity recognition in real-world scenarios. However, traditional Recurrent Neural Networks (RNN) models, such as Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU), rely solely on their internal memory cells to maintain information over time. Important details might be diluted or lost within the memory cells in complex CSI sequences. To address this limitation, we propose a lightweight approach that incorporates a multi-head adaptive attention weight mechanism MHAAM into the HAR framework. The multi-head attention mechanism allows the model to attend to different informative patterns within the CSI data simultaneously, capturing fine-grained temporal dependencies and improving the model’s ability to recognize complex activities. The implemented models effectively filter out noise and irrelevant information by assigning higher weights to informative CSI features, further enhancing activity classification accuracy. Experimental evaluations and comparative analyses of HAR for seven activities demonstrate that attention-based RNN models with multi-head attention consistently outperform traditional RNN models. The multi-head attention mechanism achieves improved generalization and testing for seven common human activities and environments, leading to a higher complex human activity classification accuracy of up to 98.5%.

Action Recognition in videos using VGG19 pre-trained based CNN-RNN Deep Learning Model

Automatic identification and classification of human actions is one the important and challenging tasks in the field of computer vision that has appealed many researchers since last two decays. It has wide range of applications such as security and surveillance, sports analysis, video analysis, human computer interaction, health care, autonomous vehicles and robotic. In this paper we developed and trained a VGG19 based CNN-RNN deep learning model using transfer learning for classification or prediction of actions and its performance is evaluated on two public actions datasets; KTH and UCF11. The models achieved significant accuracies on these datasets that are equal to 90% and 95% respectively on KTH and UCF11 which beats some of the accuracies achieved by handcraftedfeature based and deep learning based methods on these datasets.

Multiple Human Activities Classification Based on Dynamic On-Body Propagation Characteristics Using Transfer Learning

Multiple Human Activities Classification Based on Dynamic On-Body Propagation Characteristics Using Transfer Learning

Skeleton-based Human Activity Classification in Sparse Image Sequences

Research results on human activity classification in video are described, based on initial human skeleton estimation in video frames. Both single person actions and two-person interactions are considered. The initial skeleton data is estimated in selected video frames by OpenPose, HRNet or other dedicated library. Important contributions of presented work are computational steps of skeleton tracking and -refinement, and relational feature extraction from pairs of skeleton joints. It is shown, that this feature engineering significantly increases the classification accuracy. Regarding the final neural network encoder-classifier, two different architectures are designed and tested. The first solution is a lightweight MLP network, implementing the idea of a "mixture of pose experts". Several pose classifiers (experts) are trained independently on different time periods (snapshots) of single-person visual actions (or 2-person interactions), while the final classification is a time-related pooling of weighted expert classifications. All pose experts use the same deep encoding network. The second (middle weight) solution is based on a LSTM network.Both solutions are trained and tested on the action set of the well-known NTU RGB+D dataset, although only 2D data are used.Our results show comparable performance with some of the best reported STM- and CNN-based classifiers for this dataset. We conclude that by reducing the noise of skeleton data, highly successful lightweight- and midweight-approaches to visual activity recognition in image sequences can be achieved.

IoT-assisted Human Activity Recognition Using Bat Optimization Algorithm with Ensemble Voting Classifier for Disabled Persons

Internet of Things (IoT)-based human action recognition (HAR) has made a significant contribution to scientific studies. Furthermore, hand gesture recognition is a subsection of HAR, and plays a vital role in interacting with deaf people. It is the automatic detection of the actions of one or many subjects using a series of observations. Convolutional neural network structures are often utilized for finding human activities. With this intention, this study presents a new bat optimization algorithm with an ensemble voting classifier for human activity recognition (BOA-EVCHAR) technique to help disabled persons in the IoT environment. The BOA-EVCHAR technique makes use of the ensemble classification concept to recognize human activities proficiently in the IoT environment. In the presented BOA-EVCHAR approach, data preprocessing is generally achieved at the beginning level. For the identification and classification of human activities, an ensemble of two classifiers namely long short-term memory (LSTM) and deep belief network (DBN) models is utilized. Finally, the BOA is used to optimally select the hyperparameter values of the LSTM and DBN models. To elicit the enhanced performances of the BOA-EVCHAR technique, a series of experimentation analyses were performed. The extensive results of the BOA-EVCHAR technique show a superior value of 99.31% on the HAR process.

التعرف على الفعل البشري باستخدام البنية الهجينة لشبكة CNN والمحول

This work presents a Deep learning and Vision Transformer hybrid sequence model for the classification and identification of Human Motion Actions. The deep learning model works by extracting Spatial-temporal features from the features of every video, and then we use a CNN model that takes these inputs as spatial features map from videos and outputs them as a sequence of features. These sequences will be temporally fed into the Vision Transformer (ViT) which classifies the videos used into 7 different classes: Jump, Walk, Wave1, wave2, Bend, Jack, and powerful jump. The model was trained and tested on the Weismann dataset and the results showed that such a model was accurately capable of identifying the human actions. Keywords: Deep Learning, Vision Transformer, Human Motion Action Detection, Spatial features, CNN.

Human Activity Classification Based on Cognitive Doppler Radar to Optimize Carrier Frequency and Sampling Rate Using Reinforcement Learning

Human Activity Classification Based on Cognitive Doppler Radar to Optimize Carrier Frequency and Sampling Rate Using Reinforcement Learning

Attention-enhanced gated recurrent unit for action recognition in tennis.

Human Action Recognition (HAR) is an essential topic in computer vision and artificial intelligence, focused on the automatic identification and categorization of human actions or activities from video sequences or sensor data. The goal of HAR is to teach machines to comprehend and interpret human movements, gestures, and behaviors, allowing for a wide range of applications in areas such as surveillance, healthcare, sports analysis, and human-computer interaction. HAR systems utilize a variety of techniques, including deep learning, motion analysis, and feature extraction, to capture and analyze the spatiotemporal characteristics of human actions. These systems have the capacity to distinguish between various actions, whether they are simple actions like walking and waving or more complex activities such as playing a musical instrument or performing sports maneuvers. HAR continues to be an active area of research and development, with the potential to enhance numerous real-world applications by providing machines with the ability to understand and respond to human actions effectively. In our study, we developed a HAR system to recognize actions in tennis using an attention-based gated recurrent unit (GRU), a prevalent recurrent neural network. The combination of GRU architecture and attention mechanism showed a significant improvement in prediction power compared to two other deep learning models. Our models were trained on the THETIS dataset, one of the standard medium-sized datasets for fine-grained tennis actions. The effectiveness of the proposed model was confirmed by three different types of image encoders: InceptionV3, DenseNet, and EfficientNetB5. The models developed with InceptionV3, DenseNet, and EfficientNetB5 achieved average ROC-AUC values of 0.97, 0.98, and 0.81, respectively. While, the models obtained average PR-AUC values of 0.84, 0.87, and 0.49 for InceptionV3, DenseNet, and EfficientNetB5 features, respectively. The experimental results confirmed the applicability of our proposed method in recognizing action in tennis and may be applied to other HAR problems.