Activity Recognition Model Research Articles

PAR-Net: An Enhanced Dual-Stream CNN-ESN Architecture for Human Physical Activity Recognition.

Physical exercise affects many facets of life, including mental health, social interaction, physical fitness, and illness prevention, among many others. Therefore, several AI-driven techniques have been developed in the literature to recognize human physical activities. However, these techniques fail to adequately learn the temporal and spatial features of the data patterns. Additionally, these techniques are unable to fully comprehend complex activity patterns over different periods, emphasizing the need for enhanced architectures to further increase accuracy by learning spatiotemporal dependencies in the data individually. Therefore, in this work, we develop an attention-enhanced dual-stream network (PAR-Net) for physical activity recognition with the ability to extract both spatial and temporal features simultaneously. The PAR-Net integrates convolutional neural networks (CNNs) and echo state networks (ESNs), followed by a self-attention mechanism for optimal feature selection. The dual-stream feature extraction mechanism enables the PAR-Net to learn spatiotemporal dependencies from actual data. Furthermore, the incorporation of a self-attention mechanism makes a substantial contribution by facilitating targeted attention on significant features, hence enhancing the identification of nuanced activity patterns. The PAR-Net was evaluated on two benchmark physical activity recognition datasets and achieved higher performance by surpassing the baselines comparatively. Additionally, a thorough ablation study was conducted to determine the best optimal model for human physical activity recognition.

Prediction and simulation of wearable sensor devices for sports injury prevention based on BP neural network

In the research of sports injury prevention, the recognition of sports action plays an important role in the action recognition model and the prediction evaluation model. In view of the above problems, this paper constructs a new mathematical model through the idea of BP neural network. The model combines wearable technology and can improve the recognition accuracy of sports actions. The model uses the BP network classifier, which can be used in data. Processes such as feature extraction improve efficiency and reliability. In this paper, the algorithm simulation is carried out, and the experiments are carried out for three movements: running, running and static. The results show that for the recognition of running and running action, when the hidden layer node is 11, the BP neural network classifier shows the best recognition effect. For static motion, the recognition effect of each classifier is basically the same. This paper analyzes the wearable sports action recognition system, including perception layer, application layer and service layer, to realize the recognition and classification of sports actions, predict actions in advance and prevent sports injuries. Finally, this paper analyzes the causes of sports injury, puts forward specific measures to prevent sports injury, and further reduces sports injury events through wearable devices.

StresSense: Real-Time detection of stress-displaying behaviors

BackgroundWrist-worn gadgets like smartphones are ideal for unobtrusively gathering user data, in various fields such as health and fitness monitoring, communication, and productivity enhancement. They seamlessly integrate into users' daily lives, providing valuable insights and features without the need for constant attention or disruption. In sensitive domains like mental health, these devices provide user-friendly, privacy-protected means of diagnosis and treatment, offering a secure and cost-effective avenue for seeking help. ObjectivesThis study addresses the limitations of traditional mental health assessment techniques, such as intrusive sensing and subjective self-reporting, by harnessing the unobtrusive data collection capabilities of smartphones. Equipped with accelerometers and other sensors, these devices offer a novel approach to mental health research. Our objective was to develop methods for real-time detection of stress and boredom behavior markers using smart devices and machine learning algorithms. MethodologyBy leveraging data from accelerometers (A), gyroscopes (G), and magnetometers (M), we compiled a dataset indicative of stress-related behaviors and trained various machine-learning models for predictive accuracy. The methodology involved collecting data from motion sensors (A, G, and M) on the dominant arm's wrist-worn smartphone, followed by data preprocessing, transformation from time series format, and training a Deep Neural Network (DNN) model for activity recognition. FindingsRemarkably, the DNN achieved an accuracy of 93.50% on test data, outperforming traditional and ensemble machine learning methods across different window sizes, and demonstrated real-time accuracy of 77.78%, validating its practical application. ConclusionIn conclusion, this research presents a novel dataset for detecting stress and boredom behaviors using smartphones, reducing reliance on costly devices and offering a more objective assessment. It also proposes a DNN-based method for wrist-worn devices to accurately identify complex activities associated with stress and boredom, with benefits in terms of privacy and user convenience. This advancement represents a significant contribution to the field of mental health research, providing a less intrusive and more user-friendly approach to monitoring mental well-being.

An Optimized Deep LSTM Model for Human Action Recognition

An Optimized Deep LSTM Model for Human Action Recognition

Exploring biometric domain adaptation in human action recognition models for unconstrained environments

AbstractIn conventional machine learning (ML), a fundamental assumption is that the training and test sets share identical feature distributions, a reasonable premise drawn from the same dataset. However, real-world scenarios often defy this assumption, as data may originate from diverse sources, causing disparities between training and test data distributions. This leads to a domain shift, where variations emerge between the source and target domains. This study delves into human action recognition (HAR) models within an unconstrained, real-world setting, scrutinizing the impact of input data variations related to contextual information and video encoding. The objective is to highlight the intricacies of model performance and interpretability in this context. Additionally, the study explores the domain adaptability of HAR models, specifically focusing on their potential for re-identifying individuals within uncontrolled environments. The experiments involve seven pre-trained backbone models and introduce a novel analytical approach by linking domain-related (HAR) and domain-unrelated (re-identification (re-ID)) tasks. Two key analyses addressing contextual information and encoding strategies reveal that maintaining the same encoding approach during training results in high task correlation while incorporating richer contextual information enhances performance. A notable outcome of this study is the comprehensive evaluation of a novel transformer-based architecture driven by a HAR backbone, which achieves a robust re-ID performance superior to state-of-the-art (SOTA). However, it faces challenges when other encoding schemes are applied, highlighting the role of the HAR classifier in performance variations.

Design of a Low-Cost and Device-Free Human Activity Recognition Model for Smart LED Lighting Control

Design of a Low-Cost and Device-Free Human Activity Recognition Model for Smart LED Lighting Control

Domain Adaptation for Sensor-Based Human Activity Recognition with a Graph Convolutional Network

Sensor-based human activity recognition (HAR) plays a fundamental role in various mobile application scenarios, but the model performance of HAR heavily relies on the richness of the dataset and the completeness of data annotation. To address the shortage of comprehensive activity types in collected datasets, we adopt the domain adaptation technique with a graph neural network-based approach by incorporating an adaptive learning mechanism to enhance the action recognition model’s generalization ability, especially when faced with limited sample sizes. To evaluate the effectiveness of our proposed approach, we conducted experiments using three well-known datasets: MHealth, PAMAP2, and TNDA. The experimental results demonstrate the efficacy of our approach in sensor-based HAR tasks, achieving impressive average accuracies of 98.88%, 98.58%, and 97.78% based on the respective datasets. Furthermore, we conducted transfer learning experiments to address the domain adaptation problem. These experiments revealed that our proposed model exhibits exceptional transferability and distinguishing ability, even in scenarios with limited available samples. Thus, our approach offers a practical and viable solution for sensor-based HAR tasks.

MHAiR: A Dataset of Audio-Image Representations for Multimodal Human Actions

Audio-image representations for a multimodal human action (MHAiR) dataset contains six different image representations of the audio signals that capture the temporal dynamics of the actions in a very compact and informative way. The dataset was extracted from the audio recordings which were captured from an existing video dataset, i.e., UCF101. Each data sample captured a duration of approximately 10 s long, and the overall dataset was split into 4893 training samples and 1944 testing samples. The resulting feature sequences were then converted into images, which can be used for human action recognition and other related tasks. These images can be used as a benchmark dataset for evaluating the performance of machine learning models for human action recognition and related tasks. These audio-image representations could be suitable for a wide range of applications, such as surveillance, healthcare monitoring, and robotics. The dataset can also be used for transfer learning, where pre-trained models can be fine-tuned on a specific task using specific audio images. Thus, this dataset can facilitate the development of new techniques and approaches for improving the accuracy of human action-related tasks and also serve as a standard benchmark for testing the performance of different machine learning models and algorithms.

Domain adaptation framework for personalized human activity recognition models

Domain adaptation framework for personalized human activity recognition models

Novel Human Activity Recognition and Recommendation Models for Maintaining Good Health of Mobile Users

With the continuous improvement of the living standard, people have changed their concept from disease treatment to health management. However, most of the current health management software makes recommendations based on users’ static information, with low updating frequency. The effect of targeted suggestions becomes weak with time, and it is hard for the recommendation effect to be satisfactory. Based on the use of smartphones for recognizing human activities in real-time, firstly, a novel 'CNN+GRU' model is proposed in this paper, utilizing both convolutional neural networks (CNNs) and gated recurrent units (GRUs). 'CNN+GRU' can improve the recognition speed and extract the features in sensor data more accurately by achieving in the conducted experiments an average accuracy of 91.27%, thus outperforming other models compared. Secondly, another model, named SimilRec, is proposed for physical activity recommendation to users based on their health profile, the similarities between their current physical activity sequence, and the historical physical activity sequence of other (similar) users.

Enhanced Recognition of Human Activity using Hybrid Deep Learning Techniques

In the domain of deep learning, Human Activity Recognition (HAR) models stand out, surpassing conventional methods. These cutting-edge models excel in autonomously extracting vital data features and managing complex sensor data. However, the evolving nature of HAR demands costly and frequent retraining due to subjects, sensors, and sampling rate variations. To address this challenge, we introduce Cross-Domain Activities Analysis (CDAA) combined with a clustering-based Gated Recurrent Unit (GRU) model. CDAA reimagines motion clusters, merging origin and destination movements while quantifying domain disparities. Expanding our horizons, we incorporate image datasets, leveraging Convolutional Neural Networks (CNNs). The innovative aspects of the proposed hybrid GRU_CNN model, showcasing its superiority in addressing specific challenges in human activity recognition, such as subject and sensor variations. This approach consistently achieves 98.5% accuracy across image, UCI-HAR, and PAMAP2 datasets. It excels in distinguishing activities with similar postures. Our research not only pushes boundaries but also reshapes the landscape of HAR, opening doors to innovative applications in healthcare, fitness tracking, and beyond.

Video Abnormal Action Recognition Based on Multimodal Heterogeneous Transfer Learning

Human abnormal action recognition is crucial for video understanding and intelligent surveillance. However, the scarcity of labeled data for abnormal human actions often hinders the development of high-performance models. Inspired by the multimodal approach, this paper proposes a novel approach that leverages text descriptions associated with abnormal human action videos. Our method exploits the correlation between the text domain and the video domain in the semantic feature space and introduces a multimodal heterogeneous transfer learning framework from the text domain to the video domain. The text of the videos is used for feature encoding and knowledge extraction, and knowledge transfer and sharing are realized in the feature space, which is used to assist in the training of the abnormal action recognition model. The proposed method reduces the reliance on labeled video data, improves the performance of the abnormal human action recognition algorithm, and outperforms the popular video-based models, particularly in scenarios with sparse data. Moreover, our framework contributes to the advancement of automatic video analysis and abnormal action recognition, providing insights for the application of multimodal methods in a broader context.

WiLDAR: WiFi Signal-Based Lightweight Deep Learning Model for Human Activity Recognition

WiLDAR: WiFi Signal-Based Lightweight Deep Learning Model for Human Activity Recognition

P2ANet: A Large-Scale Benchmark for Dense Action Detection from Table Tennis Match Broadcasting Videos

While deep learning has been widely used for video analytics, such as video classification and action detection, dense action detection with fast-moving subjects from sports videos is still challenging. In this work, we release yet another sports video benchmarkP2ANetforPingPong-Action detection, which consists of 2,721 video clips collected from the broadcasting videos of professional table tennis matches in World Table Tennis Championships and Olympiads. We work with a crew of table tennis professionals and referees on a specially designed annotation toolbox to obtain fine-grained action labels (in 14 classes) for every ping-pong action that appeared in the dataset, and formulate two sets of action detection problems—action localizationandaction recognition. We evaluate a number of commonly seen action recognition (e.g., TSM, TSN, Video SwinTransformer, and Slowfast) and action localization models (e.g., BSN, BSN++, BMN, TCANet), usingP2ANetfor both problems, under various settings. These models can only achieve 48% area under the AR-AN curve for localization and 82% top-one accuracy for recognition since the ping-pong actions are dense with fast-moving subjects but broadcasting videos are with only 25 FPS. The results confirm thatP2ANetis still a challenging task and can be used as a special benchmark for dense action detection from videos. We invite readers to examine our dataset by visiting the following link:https://github.com/Fred1991/P2ANET.

Comparative performance of machine learning models for the classification of human gait

The efficacy of human activity recognition (HAR) models mostly relies on the characteristics derived from domain expertise. The input of the classification algorithm consists of many characteristics that are utilized to accurately and effectively classify human physical activities. In contemporary research, machine learning techniques have been increasingly employed to automatically extract characteristics from unprocessed sensory input to develop models for Human Activity Recognition (HAR) and classify various activities. The primary objective of this research is to compare and contrast several machine learning models and determine a reliable and precise classification model for classifying activities. This study does a comparison analysis in order to assess the efficacy of 10 distinct machine learning models using frequently used datasets in the field of HAR. In this work, three benchmark public human walking datasets are being used. The research is conducted based on eight evaluating parameters. Based on the study conducted, it was seen that the machine learning classification models Random Forest, Extra Tree, and Light Gradient Boosting Machine had superior performance in all the eight evaluating parameters compared to specific datasets. Consequently, it can be inferred that machine learning significantly enhances performance within the area of Human Activity Recognition (HAR). This study can be utilized to provide suitable model selection for HAR-based datasets. Furthermore, this research can be utilized to facilitate the identification of various walking patterns for bipedal robotic systems.

Semantic-aware Video Representation for Few-shot Action Recognition.

Recent work on action recognition leverages 3D features and textual information to achieve state-of-the-art performance. However, most of the current few-shot action recognition methods still rely on 2D frame-level representations, often require additional components to model temporal relations, and employ complex distance functions to achieve accurate alignment of these representations. In addition, existing methods struggle to effectively integrate textual semantics, some resorting to concatenation or addition of textual and visual features, and some using text merely as an additional supervision without truly achieving feature fusion and information transfer from different modalities. In this work, we propose a simple yet effective Semantic-Aware Few-Shot Action Recognition (SAFSAR) model to address these issues. We show that directly leveraging a 3D feature extractor combined with an effective feature-fusion scheme, and a simple cosine similarity for classification can yield better performance without the need of extra components for temporal modeling or complex distance functions. We introduce an innovative scheme to encode the textual semantics into the video representation which adaptively fuses features from text and video, and encourages the visual encoder to extract more semantically consistent features. In this scheme, SAFSAR achieves alignment and fusion in a compact way. Experiments on five challenging few-shot action recognition benchmarks under various settings demonstrate that the proposed SAFSAR model significantly improves the state-of-the-art performance.

Designing an Athletic Training Assistance System under Deep Learning Guided by Kinesiology Theory

Abstract The combined application of computer artificial intelligence technology and sports promotes the positive development of the sports industry. The combined application of computer artificial intelligence technology and sports promotes the positive development of the sports industry. After completing the model construction, the structure of the external limb sports training auxiliary machinery is designed, and the system structure of the sports training auxiliary machinery is constructed by combining the needs of sports training. The optimization of external limb sports training auxiliary machinery was achieved by performing and solving mathematical modeling of a single limb. The performance of the action recognition model and the sports auxiliary machinery is examined through experiments, in which the action recognition model based on deep learning in this paper has the best effect, which is 95.75%, and the loss values of the model after convergence are all below 0.2. In the empirical testing of the performance of the sports assistive machine, the obtained target trajectory tracking error is within ±3mm and ±2mm, respectively, and the performance of the deep learning-based action recognition model and the external limb sports training assistive machine is excellent, which can effectively help the sports training. The purpose of this study is to provide design direction and practical significance for the design of a sports training assistance system.

Study on the Practical Effect of Modernization of Ancient Chinese Educational Concepts Empowered by Intelligent Algorithms

Abstract It takes ten years to grow trees and a hundred years to nurture people. Education, as the most indispensable link in establishing morality and cultivating people, has a very important role in modern society. The study explores the application of ancient Chinese educational concepts to modern educational models, proposes a modernized teaching model based on ancient educational concepts, and uses the YOLO model to construct a classroom teaching expression and action recognition model under ancient educational concepts. The model is used in the actual classroom to analyze the expressions and behaviors of students in the classroom to explore their classroom concentration, which is used to analyze the quality of classroom teaching. By comparing the differences and changes in the performance of the experimental group and the control group, the effect of the modernized teaching model based on the ancient education concept on actual teaching is explored. Through the model test, the classroom students’ concentration score is 72.71. The pre-test P-value of the experimental group and the control group is 0.846, and the post-test P-value is 0.000, which indicates that there is a significant difference in the performance of the two groups after the experiment. The increase in the performance of the students in the experimental group was 31.390 points, while in the control group, it was 5.175 points. The P-values of the students in the experimental group on the five dimensions of math level are less than 0.05, and the control group is greater than 0.05, which indicates that the modernized teaching model based on the ancient concept of education has a significant positive effect on improving the performance of the students.

Application of 3D recognition algorithm based on spatio-temporal graph convolutional network in basketball pose estimation

In recent years, human motion recognition in computer vision has become a hot research direction in this field. Based on 2D human motion recognition technology, real-time extraction of motion features from 2D planes is used to recognize human movements. This method can only learn the position contour and color information of the image. It cannot directly reflect the motion situation, which results in low recognition accuracy and efficiency. In response to this issue, this study proposes a combination method of motion recognition and 3D pose estimation to recognize and classify basketball movements. First, the 2D skeleton model is obtained by extracting the feature information in the video action, which is converted into a 3D model, and the model is replaced by the time-space convolutional network to establish a human action recognition model. The experiment showed that when the number of iterations reached 6, the accuracy of the spatio-temporal graph convolutional network algorithm model reached 92%. Comparing the accuracy rates of different algorithm models, the average accuracy rates of convolutional neural network, long short memory network, graph convolution, long short model of action recognition and graph convolution model of action recognition were 61.6%, 65.4%, 72.5%, 76.8% and 90.3% respectively. The results show that the proposed 3D recognition algorithm can accurately recognize different basketball movements. This study can provide reference for basketball coaches and athletes in basketball training.

Leveraging Smartphone Sensor Data and Machine Learning Model for Human Activity Recognition and Fall Classification

This study investigates the application of Machine Learning (ML) algorithms to classify human physical activities and detect different types of falls using smartphone sensor data. Smartphones equipped with various sensors offer new opportunities for healthcare and human activity monitoring, which are crucial for health and well-being, aligning with the United Nations’ sustainable development goals (SDG). The study used the ”Sensor Data Collector” Android application to collect smartphone sensor data. The data collection experiments involve ten healthy participants engaged in diverse physical activities and different fall events, utilizing accelerometers and gyroscope sensors. Different types of fall detection include forward, backward, left lateral, and right lateral fall, which are important from a seriousness of health point of view. Participants placed their smartphones in the front pocket of their trousers to ensure a fixed position during the data collection process. The collected datasets are preprocessed using a moving average filter to denoise and smoothen the data. Different statistical features and extracted and ML algorithms, namely Support Vector Machine (SVM), Random Forest (RF), Naive Bayes (NB), and K-Nearest Neighbors (KNN), are employed to create robust classifiers. SVM and RF performed exceptionally well in classifying normal physical activities, achieving a 99.09% accuracy. SVM excelled in daily walking activity classification with a 99.59% accuracy, while the K-NN model performed best for crucial different types of fall classification, achieving an accuracy of 85.71%. This research emphasizes the potential of ML models for accurate physical activity and various types of fall classification, expanding the horizons of smartphone-based healthcare applications and aligning with sustainable development goals.