Action Recognition Method Research Articles

Brain-inspired multimodal motion and fine-grained action recognition

IntroductionTraditional action recognition methods predominantly rely on a single modality, such as vision or motion, which presents significant limitations when dealing with fine-grained action recognition. These methods struggle particularly with video data containing complex combinations of actions and subtle motion variations.MethodsTypically, they depend on handcrafted feature extractors or simple convolutional neural network (CNN) architectures, which makes effective multimodal fusion challenging. This study introduces a novel architecture called FGM-CLIP (Fine-Grained Motion CLIP) to enhance fine-grained action recognition. FGM-CLIP leverages the powerful capabilities of Contrastive Language-Image Pretraining (CLIP), integrating a fine-grained motion encoder and a multimodal fusion layer to achieve precise end-to-end action recognition. By jointly optimizing visual and motion features, the model captures subtle action variations, resulting in higher classification accuracy in complex video data.Results and discussionExperimental results demonstrate that FGM-CLIP significantly outperforms existing methods on multiple fine-grained action recognition datasets. Its multimodal fusion strategy notably improves the model's robustness and accuracy, particularly for videos with intricate action patterns.

High-accuracy detection method for miner action recognition with shuffle attention SlowFast networks

High-accuracy detection method for miner action recognition with shuffle attention SlowFast networks

A method of athlete foul action recognition based on DTW algorithm

A method of athlete foul action recognition based on DTW algorithm

Graph-Based Methods for Multimodal Indoor Activity Recognition: A Comprehensive Survey

Graph-Based Methods for Multimodal Indoor Activity Recognition: A Comprehensive Survey

Accurate recognition of human abnormal behaviours using adaptive 3D residual attention network with gated recurrent units (GRU) in the video sequences

ABSTRACT Abnormal or violent behaviour by individuals with mental disorders presents significant risks to public safety, necessitating advanced systems capable of detecting such behaviours in real time. Traditional single-sensing methods for human activity recognition often struggle with issues like signal noise, dropped data, and limited scalability, which hinder their ability to accurately detect abnormal behaviours in dynamic and complex environments. This paper introduces a novel solution that addresses these challenges by proposing an adaptive 3D residual attention network (A3D-RAN) combined with Gated Recurrent Units (GRUs). The A3D-RAN utilises an adaptive attention mechanism to focus on the most relevant regions in video sequences, while residual connections improve feature reuse and maintain gradient flow, enabling fine-grained detail capture. GRUs are integrated to efficiently model long-term temporal dependencies, ensuring a more comprehensive understanding of human behaviour across time. Through extensive experimentation on real-world datasets, our model achieved a remarkable accuracy of 97%, significantly surpassing the 78% accuracy of standalone A3D-RAN implementations. Moreover, the robustness of the model under challenging conditions – such as occlusions and lighting variations – demonstrates its potential for real-world surveillance applications. By employing the Improved War Strategy Optimization (IWSO) Algorithm for parameter tuning, we further enhanced performance, reaching an unprecedented accuracy of 99%. This breakthrough underscores the practical value of our approach in improving public safety and security through accurate and timely detection of abnormal behaviours in surveillance systems.

Application of action recognition and tactical optimization methods for rope skipping competitions based on artificial intelligence

To solve the problems that action recognition methods in rope skipping competitions rely on manual annotation and are prone to misjudgment in complex movements, this study implemented an AI-based rope skipping action recognition and tactical optimization method, using artificial intelligence technology to achieve efficient and accurate action recognition and tactical adjustment. The feature extraction of video frames is performed through Convolutional Neural Network (CNN), and the processed feature sequence is sent to Long Short-Term Memory (LSTM) network for processing, so as to achieve accurate recognition of rope skipping actions. To optimize the competition strategy, the Deep Q Network (DQN) is used to optimize the tactical execution. Experimental results show that the proposed model can recognize common rope skipping movements such as single jump, double-leg jump and cross jump with an average accuracy of 98.4%; the tactical strategy optimized by reinforcement learning significantly improves the performance of athletes, the jumping frequency increases by 4.59% and the error rate decreases by 0.986%. This study not only provides an intelligent evaluation and optimization solution for rope skipping competitions, but also has certain reference significance for action recognition and tactical decision-making in other sports.

Self-supervised representation learning for robust fine-grained human hand action recognition in industrial assembly lines

Humans are still indispensable on industrial assembly lines, but in the event of an error, they need support from intelligent systems. In addition to the objects to be observed, it is equally important to understand the fine-grained hand movements of a human to be able to track the entire process. However, these deep-learning-based hand action recognition methods are very label intensive, which cannot be offered by all industrial companies due to the associated costs. This work therefore presents a self-supervised learning approach for industrial assembly processes that allows a spatio-temporal transformer architecture to be pre-trained on a variety of information from real-world video footage of daily life. Subsequently, this deep learning model is adapted to the industrial assembly task at hand using only a few labels. Well-known real-world datasets best suited for representation learning of such hand actions in a regression tasks are outlined and to what extent they optimize the subsequent supervised trained classification task. This subsequent fine-tuning is supplemented by concept drift detection, which makes the resulting productively employed models more robust against concept drift and future changing assembly movements.

Player Performance Analysis in Table Tennis Through Human Action Recognition

This paper aims to enhance the effectiveness of table tennis coaching and player performance analysis through human action recognition by using deep learning. In the field of video analysis, human action recognition has emerged as a highly researched area. Beyond post-session analysis, it has the potential for real-time applications, such as providing instant feedback or comparing ideal motions with actual player movements. However, the complexity of human actions presents significant challenges. To address these issues, in this paper, we combine the latest computer vision and deep learning algorithms to accurately identify and classify a few table tennis strokes in human action recognition. Through an in-depth review of existing methods, we develop a high-precision offline method for player action recognition. Our experimental results show that the proposed method achieves an average accuracy of 99.85% in recognizing six distinct table tennis actions based on our own dataset.

An Action Evaluation Method for Virtual Reality Simulation Power Training Based on an Improved Dynamic Time Warping Algorithm

To address the shortcomings in action evaluation within VR simulation power training, this paper introduces a novel action recognition and evaluation method based on dynamic recognition of finger keypoints combined with an improved Dynamic Time Warping (DTW) algorithm. By constructing an action recognition model centered on hand keypoints, the proposed method integrates distance similarity and cosine similarity to account comprehensively for both numerical differences and directional consistency of action features. This approach effectively tackles the challenges of feature extraction and recognition for complex actions in VR power training. Furthermore, a scoring mechanism based on the improved DTW algorithm is proposed, which employs Gaussian-weighted feature-derivative Euclidean distance combined with cosine similarity. This method significantly reduces computational complexity while improving scoring accuracy and consistency. Experimental results indicated that the improved DTW algorithm outperformed traditional methods in terms of classification accuracy and robustness. In particular, cosine similarity demonstrated superior performance in capturing dynamic variations and assessing the consistency of fine hand movements. This study provides an essential technical reference for action evaluation in VR simulation power training and offers a scientific basis for advancing the intelligence and digitalization of power VR training environments.

Enhancing Healthcare Monitoring: A Deep Learning Approach to Human Activity Recognition using Wearable Sensors

Wearable devices and deep learning methods for Human Activity Recognition (HAR) have attracted a lot of interest because they could change healthcare monitoring. This study presents a CNN-LSTM model to accurately and reliably detect human movements from smartphone sensor data. The proposed model takes advantage of both the strengths of Long Short-Term Memory (LSTM) networks for modeling time and Convolutional Neural Networks (CNNs) for extracting features from space. This enables determining how the input data change over time and space. This study examines whether this method can work and is practical in real-life healthcare settings, focused on uses such as watching patients from distance, caring for the elderly, and therapy. The proposed model was evaluated on publicly accessible standard datasets. Various architectural configurations and hyperparameters were examined to determine their performance. The proposed CNN-LSTM model performed well and has great potential for practical use in activity tracking and environment understanding systems.

The research on dance motion quality evaluation based on spatiotemporal convolutional neural networks

With the rapid development of artificial intelligence and computer vision technology, dance action recognition and quality assessment have become a challenging and important research field. Traditional dance movement recognition methods often rely on manual feature extraction and expert scoring, which have problems such as strong subjectivity, low efficiency, and difficulty in handling complex movements. To address these issues, this study proposes a dance action recognition and quality assessment method based on Spatio Temporal Convolutional Neural Network (ST-CNN). Specifically, it includes a feature extraction network based on 3D spatiotemporal description operators and a multi-scale aggregated long short memory network for extracting spatial features of single frame images and fusing video sequence features composed of all frames. This approach can effectively capture dynamic changes in dance movements by combining information from spatial and temporal dimensions, thereby achieving more accurate recognition and evaluation. In addition, to address the high storage and computational requirements of convolutional neural networks during training, we integrated dance video sequences into an IoT architecture to address parallel recognition and evaluation of multiple videos. Finally, we validated the proposed method on the public datasets NTU-RGB 60 and NTU-RGB 120, and the experimental results demonstrated the effectiveness of the method.

A Review on Computer Vision-Based Methods for Abnormal Human Action Recognition

Human behavior recognition constitutes a crucial research domain within both computer vision and behavior recognition. As a branch of human behavior recognition, abnormal behavior recognition has witnessed rapid advancement in recent years, which is capable of enhancing the governance level of public safety. To investigate the theoretical and technical progress of abnormal behavior recognition in public places within the realm of computer vision, this paper initially delineates the definition of abnormal behavior in public places. Secondly, as computer vision and pattern recognition technologies have progressed, algorithms are now divided into two distinct categories: traditional methods and those leveraging deep learning techniques. The identification of atypical human behavior can be divided into two approaches depending on the detection of body key points: one approach uses skeletal key points, while the other focuses on analyzing temporal and spatial features. Finally, this paper conducts a review of the mainstream datasets of abnormal human behavior both at home and abroad, analyzes the performance of related algorithms on the datasets, and gives future research directions as well as optimization suggestions.

A Novel CNN-BiLSTM-GRU Hybrid Deep Learning Model for Human Activity Recognition

Human Activity Recognition (HAR) is critical in a variety of disciplines, including healthcare and robotics. This paper presents a new Convolutional Neural Network with Bidirectional Long Short-Term Memory and along with Gated Recurrent Unit (CNN-BiLSTM-GRU)hybrid deep learning model designed for Human Activity Recognition (HAR) that makes use of data from wearable sensors and mobile devices. Surprisingly, the model achieves an amazing accuracy rate of 99.7% on the difficult Wireless Sensor Data Mining (WISDM) dataset, demonstrating its ability to properly identify human behaviors. This study emphasizes parameter optimization, with a focus on batch size 0.3 as a significant component in improving the model’s robustness. Furthermore, the findings of this study have far-reaching implications for bipedal robotics, where precise HAR (Human Activity Recognition) is critical to improving human–robot interaction quality and overall work efficiency. These discoveries not only strengthen Human Activity Recognition (HAR) techniques, but also provide practical benefits in real-world applications, particularly in the robotics and healthcare areas. This study thus makes a significant contribution to the continuous development of Human Activity Recognition methods and their actual applications, emphasizing their important role in stimulating innovation and efficiency across a wide range of industries.

Non-Contact Cross-Person Activity Recognition by Deep Metric Ensemble Learning.

In elderly monitoring or indoor intrusion detection, the recognition of human activity is a key task. Owing to several strengths of Wi-Fi-based devices, including their non-contact and privacy protection, these devices have been widely applied in the area of smart homes. By the deep learning technique, numerous Wi-Fi-based activity recognition methods can realize satisfied recognitions, however, these methods may fail to recognize the activities of an unknown person without the learning process. In this study, using channel state information (CSI) data, a novel cross-person activity recognition (CPAR) method is proposed by a deep learning approach with generalization capability. Combining one of the state-of-the-art deep neural networks (DNNs) used in activity recognition, i.e., attention-based bi-directional long short-term memory (ABLSTM), the snapshot ensemble is the first to be adopted to train several base-classifiers for enhancing the generalization and practicability of recognition. Second, to discriminate the extracted features, metric learning is further introduced by using the center loss, obtaining snapshot ensemble-used ABLSTM with center loss (SE-ABLSTM-C). In the experiments of CPAR, the proposed SE-ABLSTM-C method markedly improved the recognition accuracies to an application level, for seven categories of activities.

A unified framework for unsupervised action learning via global-to-local motion transformer

Human action recognition remains challenging due to the inherent complexity arising from the combination of diverse granularity of semantics, ranging from the local motion of body joints to high-level relationships across multiple people. To learn this multi-level characteristic of human action in an unsupervised manner, we propose a novel pretraining strategy along with a transformer-based model architecture named GL-Transformer++. Prior methods in unsupervised action recognition or unsupervised group activity recognition (GAR) have shown limitations, often focusing solely on capturing a partial scope of the action, such as the local movements of each individual or the broader context of the overall motion. To tackle this problem, we introduce a novel pretraining strategy named multi-interval pose displacement prediction (MPDP) that enables the model to learn the diverse extents of the action. In the architectural aspect, we incorporate the global and local attention (GLA) mechanism within the transformer blocks to learn local dynamics between joints, global context of each individual, as well as high-level interpersonal relationships in both spatial and temporal manner. In fact, the proposed method is a unified approach that demonstrates efficacy in both action recognition and GAR. Particularly, our method presents a new and strong baseline, surpassing the current SOTA GAR method by significant margins: 29.6% in Volleyball and 60.3% and 59.9% on the xsub and xset settings of the Mutual NTU dataset, respectively.

Advancing Incremental Few-Shot Video Action Recognition with Cluster Compression and Generative Separation

Few-Shot Class Incremental Learning (FSCIL) is a trending topic in deep learning, addressing the need for models to incrementally learn novel classes, particularly in real-world scenarios where continuously emerging classes come with limited labeled samples. However, the majority of FSCIL research has been dedicated to image classification and object recognition tasks, with limited attention given to video action classification. In this paper, we present a new Cluster Compression and Generative Separation (CCGS) method for Incremental Few-Shot Video Action Recognition (iFSVAR), which introduces contrastive learning to boost the degree of class separation in the base session. Simultaneously, it creates numerous fine-grained classes with diverse semantics, effectively filling the unallocated representation space. Experimental results on UCF101, Kinetics, and Something-Something-V2[Formula: see text]demonstrate the effectiveness of the framework.

Depth Video-Based Secondary Action Recognition in Vehicles via Convolutional Neural Network and Bidirectional Long Short-Term Memory with Spatial Enhanced Attention Mechanism

Secondary actions in vehicles are activities that drivers engage in while driving that are not directly related to the primary task of operating the vehicle. Secondary Action Recognition (SAR) in drivers is vital for enhancing road safety and minimizing accidents related to distracted driving. It also plays an important part in modern car driving systems such as Advanced Driving Assistance Systems (ADASs), as it helps identify distractions and predict the driver’s intent. Traditional methods of action recognition in vehicles mostly rely on RGB videos, which can be significantly impacted by external conditions such as low light levels. In this research, we introduce a novel method for SAR. Our approach utilizes depth-video data obtained from a depth sensor located in a vehicle. Our methodology leverages the Convolutional Neural Network (CNN), which is enhanced by the Spatial Enhanced Attention Mechanism (SEAM) and combined with Bidirectional Long Short-Term Memory (Bi-LSTM) networks. This method significantly enhances action recognition ability in depth videos by improving both the spatial and temporal aspects. We conduct experiments using K-fold cross validation, and the experimental results show that on the public benchmark dataset Drive&Act, our proposed method shows significant improvement in SAR compared to the state-of-the-art methods, reaching an accuracy of about 84% in SAR in depth videos.

Wearable Activity Recognition for Advancing Elderly Care with Modified Transformer Model

Technological devices such as smartphones can be utilized in tracking various human activities or movements through built-in accelerometers and gyroscopes. Data obtained from these inertial sensors can be utilized in various applications to assist people, including healthcare, human-computer interaction, and sports. As a result, further developments in the effective classification of time series data through machine and deep learning is highly valued and actively pursued. In this study, the transformer model, a deep learning architecture designed for sequential data such as natural language processing (NLP), has been utilized for analysis of time-series motion readings from wearable accelerometers. The transformer model in this study has been refined by incorporating a Long Short-Term Memory (LSTM) recurrent neural network (RNN) architecture. By leveraging the HAR70+ dataset with a wide range of activities, the modified transformer model in this study obtained a best accuracy of 95.85%, demonstrating that it can match the performance of state-of-the-art wearable activity recognition methods using Deep Neural Networks (DNN) and LSTM. Hence, the findings presented in this study suggest the future relevance of improved transformer or deep learning models to enhance the quality of life for seniors.

The Application of Deep Learning in Sports Training

In response to the problems of overfitting, susceptibility to interference information, and insufficient feature expression ability in existing deep learning methods for sports action recognition, the author proposes a deep learning sports action recognition method that integrates attention mechanism. This method proposes a video data augmentation algorithm in data preprocessing to reduce the risk of model overfitting. Then, during the video frame sampling process, the existing sampling algorithms are improved to effectively suppress the influence of interference information. In the special section, the network residual consolidation is proposed to improve the feature extraction capacity of the structure. The Long Term Time Transform (LSTM) network is used to solve the problem of the global correlation of the spatial correlation, and the classification algorithm is achieved by Softmax. and the classification algorithm is proposed. The experimental results show that the recognition rates of this method on UCFYouTube, KTH, and HMDB-51 data are 96.73%, 98.07%, and 64.82%, respectively.

Identifying Truck Activity from Global Positioning System Data Based on Satellite Imagery

Road freight transportation plays a crucial role in fostering economic growth and enhancing logistics efficiency. Existing research primarily focuses on analyzing freight truck activity characteristics based on global positioning system (GPS) data of freight truck trajectories. Those studies proposed a variety of identification methods from different perspectives, but they mainly rely on rule-based methods and subjectively defined key threshold parameters, ignoring the complexity of truck freight activities and relying on long sequences of trajectory data. To bridge this gap, we propose the satellite image augmented truck activity recognition method, a structured framework that utilizes satellite imagery to identify truck activities from GPS data. Firstly, we categorize truck activities into three types: loading/unloading, resting, and driving. We extract geographic entities from the area of interest data to create the satellite image dataset of truck activity. Secondly, we evaluate the recognition accuracy of benchmark image classification models on our dataset, and ResNet50 achieves 98.04% accuracy on the test set. Thirdly, we apply the image classification model to identify truck activity from GPS trajectory data and use the recognition results of neighboring points to improve the identification results. The framework delineated in this paper entails the extraction of truck activities from GPS data at the satellite image level. Our approach holds promise in facilitating logistics operation fleets and traffic management authorities to maintain near real-time surveillance of the truck status, thereby fostering enhancements in both the efficiency and safety of freight transport.