Activity Recognition Techniques Research Articles

One-Shot Action Recognition via Multi-Scale Spatial-Temporal Skeleton Matching.

One-shot skeleton action recognition, which aims to learn a skeleton action recognition model with a single training sample, has attracted increasing interest due to the challenge of collecting and annotating large-scale skeleton action data. However, most existing studies match skeleton sequences by comparing their feature vectors directly which neglects spatial structures and temporal orders of skeleton data. This paper presents a novel one-shot skeleton action recognition technique that handles skeleton action recognition via multi-scale spatial-temporal feature matching. We represent skeleton data at multiple spatial and temporal scales and achieve optimal feature matching from two perspectives. The first is multi-scale matching which captures the scale-wise semantic relevance of skeleton data at multiple spatial and temporal scales simultaneously. The second is cross-scale matching which handles different motion magnitudes and speeds by capturing sample-wise relevance across multiple scales. Extensive experiments over three large-scale datasets (NTU RGB+D, NTU RGB+D 120, and PKU-MMD) show that our method achieves superior one-shot skeleton action recognition, and outperforms SOTA consistently by large margins.

Improved sand cat swarm optimization with deep learning based enhanced malicious activity recognition for cybersecurity

The main concept of a smart city is to join manual items with electronics, software, sensors, and network connectivity for data contact via Internet of Things (IoT) gadgets. IoT improves production and efficiency by cleverly utilizing remote management, but security as well as privacy risk, rises. Cyber threats are progressing every day, affecting inadequate actions of safety and privacy. In a smart city, the exposed action of an individual or business can keep the whole city in danger. Owing to the trust of numerous modules of smart cities on data and communication techniques, cybersecurity tasks like information leakage and mischievous cyber-attacks. Malicious Activity Detection in the area of cybersecurity for smart cities is dominant in protecting the consistent organization of urban atmospheres. Leveraging innovative machine learning (ML) and deep learning (DL) approaches, mainly anomaly detection methods permits the identification of abnormal patterns and probable threats within the massive streams of data produced by smart city methods. So, this paper designs a Fusion of Optimization Algorithms and DL for Enhanced Malicious Activity Recognition (FOADL-EMAR) technique for cybersecurity in smart cities. The purpose of the FOADL-EMAR technique is to recognize the presence of malicious activities using optimal DL techniques in the environment of a smart city. To accomplish this, the FOADL-EMAR technique uses a Z-score data normalization model to measure the input dataset into a useful format. In addition, the FOADL-EMAR technique uses an improved sand cat swarm optimization (ISCSO) algorithm that can be applied to select an optimal subset of features. An ensemble of three DL techniques is used namely long short-term memory (LSTM), deep neural networks (DNNs), and extreme learning machine (ELM) for malicious activity detection. The Harris Hawks Optimization (HHO) approach has been applied to improve the detection results of the ensemble models. The simulation analysis of the FOADL-EMAR methodology occurs utilizing a benchmark database. The experimentation values demonstrate the improved detection outcomes of the FOADL-EMAR algorithm over other existing methods in terms of different measures.

Unlocking the potential of RNN and CNN models for accurate rehabilitation exercise classification on multi-datasets

AbstractPhysical rehabilitation is crucial in healthcare, facilitating recovery from injuries or illnesses and improving overall health. However, a notable global challenge stems from the shortage of professional physiotherapists, particularly acute in some developing countries, where the ratio can be as low as one physiotherapist per 100,000 individuals. To address these challenges and elevate patient care, the field of physical rehabilitation is progressively integrating Computer Vision and Human Activity Recognition (HAR) techniques. Numerous research efforts aim to explore methodologies that assist in rehabilitation exercises and evaluate patient movements, which is crucial as incorrect exercises can potentially worsen conditions. This study investigates applying various deep-learning models for classifying exercises using the benchmark KIMORE and UI-PRMD datasets. Employing Bi-LSTM, LSTM, CNN, and CNN-LSTM, alongside a Random Search for architectural design and Hyper-parameter tuning, our investigation reveals the (CNN) model as the top performer. After applying cross-validation, the technique achieves remarkable mean testing accuracy rates of 93.08% on the KIMORE dataset and 99.7% on the UI-PRMD dataset. This marks a slight improvement of 0.75% and 0.1%, respectively, compared to previous techniques. In addition, expanding beyond exercise classification, this study explores the KIMORE dataset’s utility for disease identification, where the (CNN) model consistently demonstrates an outstanding accuracy of 89.87%, indicating its promising role in both exercises and disease identification within the context of physical rehabilitation.

IoT-assisted Human Activity Recognition Using Bat Optimization Algorithm with Ensemble Voting Classifier for Disabled Persons

Internet of Things (IoT)-based human action recognition (HAR) has made a significant contribution to scientific studies. Furthermore, hand gesture recognition is a subsection of HAR, and plays a vital role in interacting with deaf people. It is the automatic detection of the actions of one or many subjects using a series of observations. Convolutional neural network structures are often utilized for finding human activities. With this intention, this study presents a new bat optimization algorithm with an ensemble voting classifier for human activity recognition (BOA-EVCHAR) technique to help disabled persons in the IoT environment. The BOA-EVCHAR technique makes use of the ensemble classification concept to recognize human activities proficiently in the IoT environment. In the presented BOA-EVCHAR approach, data preprocessing is generally achieved at the beginning level. For the identification and classification of human activities, an ensemble of two classifiers namely long short-term memory (LSTM) and deep belief network (DBN) models is utilized. Finally, the BOA is used to optimally select the hyperparameter values of the LSTM and DBN models. To elicit the enhanced performances of the BOA-EVCHAR technique, a series of experimentation analyses were performed. The extensive results of the BOA-EVCHAR technique show a superior value of 99.31% on the HAR process.

Temporal Fusion of Time-Distributed VGG-16 and LSTM for Precise Action Recognition in Video Sequences

Autonomous systems play a vital role in creating an ambient environment, but for comprehensive human-robot interaction, the system must interpret and identify specific human behaviours. The demand for recognizing human activity from videos has significantly increased across diverse domains, such as medicine, intelligent video surveillance, human-robot interaction, and ambient-assisted environments. Deep learning has emerged as a pioneering technique for human activity recognition from videos due to its ability to learn features from input videos automatically. This research proposes a novel approach for recognizing activities in videos by leveraging the combined power of the VGG-16 convolutional neural network (CNN) and the Long Short-Term Memory (LSTM). To assess the efficiency of our suggested methodology, experiments are conducted on the widely used benchmark dataset UCF-50. The experimental results demonstrate that our VGG-16 and LSTM-based model outperforms state-of-the-art methods for activity recognition, achieving higher accuracy and robustness across various activity categories.

A Comparative Study of Machine Learning Techniques for Human Activity Recognition

A Comparative Study of Machine Learning Techniques for Human Activity Recognition

A Survey on 3D Skeleton-Based Action Recognition Using Learning Method.

Three-dimensional skeleton-based action recognition (3D SAR) has gained important attention within the computer vision community, owing to the inherent advantages offered by skeleton data. As a result, a plethora of impressive works, including those based on conventional handcrafted features and learned feature extraction methods, have been conducted over the years. However, prior surveys on action recognition have primarily focused on video or red-green-blue (RGB) data-dominated approaches, with limited coverage of reviews related to skeleton data. Furthermore, despite the extensive application of deep learning methods in this field, there has been a notable absence of research that provides an introductory or comprehensive review from the perspective of deep learning architectures. To address these limitations, this survey first underscores the importance of action recognition and emphasizes the significance of 3-dimensional (3D) skeleton data as a valuable modality. Subsequently, we provide a comprehensive introduction to mainstream action recognition techniques based on 4 fundamental deep architectures, i.e., recurrent neural networks, convolutional neural networks, graph convolutional network, and Transformers. All methods with the corresponding architectures are then presented in a data-driven manner with detailed discussion. Finally, we offer insights into the current largest 3D skeleton dataset, NTU-RGB+D, and its new edition, NTU-RGB+D 120, along with an overview of several top-performing algorithms on these datasets. To the best of our knowledge, this research represents the first comprehensive discussion of deep learning-based action recognition using 3D skeleton data.

SMDF: Spatial Mass Distribution Features and Deep Learning-Based Technique for Human Activity Recognition

SMDF: Spatial Mass Distribution Features and Deep Learning-Based Technique for Human Activity Recognition

An improved human activity recognition technique based on convolutional neural network

A convolutional neural network (CNN) is an important and widely utilized part of the artificial neural network (ANN) for computer vision, mostly used in the pattern recognition system. The most important applications of CNN are medical image analysis, image classification, object recognition from videos, recommender systems, financial time series analysis, natural language processing, and human–computer interfaces. However, after the technological advancement in the power of computing ability and the emergence of huge quantities of labeled data provided through enhanced algorithms, nowadays, CNN is widely used in almost every area of study. One of the main uses of wearable technology and CNN within medical surveillance is human activity recognition (HAR), which must require constant tracking of everyday activities. This paper provides a comprehensive study of the application of CNNs in the classification of HAR tasks. We describe their enhancement, from their antecedents up to the current state-of-the-art systems of deep learning (DL). We have provided a comprehensive working principle of CNN for HAR tasks, and a CNN-based model is presented to perform the classification of human activities. The proposed technique interprets data from sensor sequences of inputs by using a multi-layered CNN that gathers temporal and spatial data related to human activities. The publicly available WISDM dataset for HAR has been used to perform this study. This proposed study uses the two-dimensional CNN approach to make a model for the classification of different human activities. A recent version of Python software has been used to perform the study. The rate of accuracy for HAR through the proposed model in this experiment is 97.20%, which is better than the previously estimated state-of-the-art technique. The findings of the study imply that using DL methods for activity recognition might greatly increase accuracy and increase the range of applications where HAR can be used successfully. We have also described the future research trends in the field of HAR in this article.

Action Recognition in Dark Videos Using Spatio-Temporal Features and Bidirectional Encoder Representations From Transformers

Several research works have been developed in the area of action recognition. Unfortunately, when these algorithms are applied to low-light or dark videos, their performances are highly affected and found to be very poor or fall rapidly. To address the issue of improving the performance of action recognition in dark or low-light videos; in this article, we have developed an efficient deep 3D CNN based action recognition model. The proposed algorithm follows two-stages for action recognition. In the first stage, the low-light videos are enhanced using Zero-Reference Deep Curve Estimation (Zero-DCE), followed by the min-max sampling algorithm. In the latter stage, we propose an action classification network to recognize the actions in the enhanced videos. In the proposed action classification network, we explored the capabilities of the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$R(2+1)D$</tex-math></inline-formula> for spatio-temporal feature extraction. The model's overall generalization performance depends on how well it can capture long-range temporal structure in videos, which is essential for action recognition. So we have used a Graph convolutional network (GCN) on the top of R(2+1)D as our video feature encoder which captures long-term temporal dependencies of the extracted features. Finally, a Bidirectional Encoder Representations from Transformers (BERT) is adhered to classify the actions from the 3D features extracted from the enhanced video scenes. The effectiveness of the proposed action recognition scheme is verified on ARID V1.0 and ARID V1.5 datasets. It is observed that the proposed algorithm is able to achieve <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$96.60\%$</tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$99.88\%$</tex-math></inline-formula> as Top-1 and Top-5 accuracy, respectively, on ARID V1.0 dataset. Similarly, on ARID V1.5, the proposed algorithm is able to achieve <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$86.93\%$</tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$99.35\%$</tex-math></inline-formula> as Top-1 and Top-5 accuracies, respectively. To corroborate our findings, we have compared the results obtained by the proposed scheme with those of fifteen state-of-the-art action recognition techniques.

STMultiple: Sparse Transformer Based on RFID for Multi-Object Activity Recognition

Wireless sensing techniques for Human Activity Recognition (HAR) have been widely studied in recent years. At present, research on HAR based on Radio Frequency Identification (RFID) is changing from the tag attachment method to the tag non-attachment method. Affected by multipath, the current solutions in tag non-attachment scenarios mainly focus on single-object activity recognition, which is not suitable for multi-object scenarios. To address these issues, we propose STMultiple, a novel tag non-attachment activity recognition model for multi-object. The model first preprocesses the raw signal with filter and phase calibration, then it applies dilated convolution in the frequency domain to extract multi-object activity features, finally the feature pyramid structure and ProbSparse are used to optimize the vanilla Transformer-Encoder to enhance the activity recognition ability. Extensive experiments show that STMultiple can achieve recognition accuracy of up to 97.93% and down to about 90% in challenging environments ranging from two to five users, which has excellent performance compared to several state-of-the-art methods.

A survey on intelligent human action recognition techniques

A survey on intelligent human action recognition techniques

Attention-Based Multihead Deep Learning Framework for Online Activity Monitoring With Smartwatch Sensors

The expeditious propagation of Internet of Things (IoT) technologies implanted in different smart devices such as smartphones and smartwatches have a ubiquitous consequence on the modern population. These devices are employed to collect data and to aid in tracking and analyzing the users’ daily activities using various human activity monitoring and recognition (HAR) techniques. However, most current HAMR approaches rely on exploratory case-based shallow feature learning architectures, which endeavor to recognize activities correctly in real-world situations. To address this issue, we offer a unique strategy for HAMR that leverages the attention mechanism with multi-head convolutional neural networks (CNNs) and Long-Short-Term-Memory (LSTM). The accuracy of activity detection is improved in the presented method by integrating attention into multi-head CNNs followed by LSTM for better feature extraction and selection. Verification investigations are carried out using data from the University of California (UCI) repository, which is publicly available. The results show that our proposed framework is more accurate than current frameworks using both the 10-fold and leave-one-subject-out cross-validation. Finally, the proposed method can recognize human activity in real-time, regardless of the type of smart device.

Deep Learning Techniques for Radar-Based Continuous Human Activity Recognition

Human capability to perform routine tasks declines with age and age-related problems. Remote human activity recognition (HAR) is beneficial for regular monitoring of the elderly population. This paper addresses the problem of the continuous detection of daily human activities using a mm-wave Doppler radar. In this study, two strategies have been employed: the first method uses un-equalized series of activities, whereas the second method utilizes a gradient-based strategy for equalization of the series of activities. The dynamic time warping (DTW) algorithm and Long Short-term Memory (LSTM) techniques have been implemented for the classification of un-equalized and equalized series of activities, respectively. The input for DTW was provided using three strategies. The first approach uses the pixel-level data of frames (UnSup-PLevel). In the other two strategies, a convolutional variational autoencoder (CVAE) is used to extract Un-Supervised Encoded features (UnSup-EnLevel) and Supervised Encoded features (Sup-EnLevel) from the series of Doppler frames. The second approach for equalized data series involves the application of four distinct feature extraction methods: i.e., convolutional neural networks (CNN), supervised and unsupervised CVAE, and principal component Analysis (PCA). The extracted features were considered as an input to the LSTM. This paper presents a comparative analysis of a novel supervised feature extraction pipeline, employing Sup-ENLevel-DTW and Sup-EnLevel-LSTM, against several state-of-the-art unsupervised methods, including UnSUp-EnLevel-DTW, UnSup-EnLevel-LSTM, CNN-LSTM, and PCA-LSTM. The results demonstrate the superiority of the Sup-EnLevel-LSTM strategy. However, the UnSup-PLevel strategy worked surprisingly well without using annotations and frame equalization.

ARCTIC: A knowledge distillation approach via attention-based relation matching and activation region constraint for RGB-to-Infrared videos action recognition

The recognition effect of existing infrared-based action recognition is greatly reduced when clear appearance and texture are required. To address this limitation, the amalgamation of RGB data presents an opportunity to compensate for this deficiency. However, effectively leveraging RGB data to bridge the gap between RGB and infrared modalities poses a significant challenge. In this paper, we propose a knowledge distillation method based on attention relation matching and activation region consistency constraint (ARCTIC) for RGB-to-Infrared action recognition, which guides infrared data for recognition by acquiring the information of RGB data modality. To enhance the precision of knowledge screening across different modalities at various feature levels and ensure accurate information transfer, we constructed attention-based relation matching modules between different feature layers of the teacher and student networks. To minimize the dissimilarity between the two modalities and obtain more advantageous complementary information, we consider the spatial activation region consistency constraint to maintain the consistency between the most salient features of the teacher network and the student network, and employ knowledge distillation loss to favor the selection of accurate predictions while judiciously mitigating the occurrence of erroneous logical outputs. Our experimentation substantiates that the ARCTIC method surpasses the performance of state-of-the-art action recognition techniques across the NTU RGB+D and PKU-MMD datasets.

Overcoming Data Scarcity in Human Activity Recognition.

Wearable sensors have become increasingly popular in recent years, with technological advances leading to cheaper, more widely available, and smaller devices. As a result, there has been a growing interest in applying machine learning techniques for Human Activity Recognition (HAR) in healthcare. These techniques can improve patient care and treatment by accurately detecting and analyzing various activities and behaviors. However, current approaches often require large amounts of labeled data, which can be difficult and time-consuming to obtain. In this study, we propose a new approach that uses synthetic sensor data generated by 3D engines and Generative Adversarial Networks to overcome this obstacle. We evaluate the synthetic data using several methods and compare them to real-world data, including classification results with baseline models. Our results show that synthetic data can improve the performance of deep neural networks, achieving a better F1-score for less complex activities on a known dataset by 8.4% to 73% than state-of-the-art results. However, as we showed in a self-recorded nursing activity dataset of longer duration, this effect diminishes with more complex activities. This research highlights the potential of synthetic sensor data generated from multiple sources to overcome data scarcity in HAR.

Review of Multiple Human Action Recognition Techniques

During the last decades topics such as video analysis and image understanding have acquired a big importance due to its inclusion in applications such as security, intelligent spaces, assistive living and focused marketing. Human action detection is investigated in utilization of artificial intelligence and computer vision. Numerous effective action recognition strategies have demonstrated and the action information are successfully gained from motion videos and still pictures. In order to get equivalent actions, the proper activity information gained from various kind of media like video or picture might be connected. The majority of existing video activity action identification strategies experience the ill effects of inadequate recordings. In this review article we are going to discussed about some earlier human action recognition techniques. In past numbers of researchers, developers dose a nicest work in same specific domain, in this review article/Paper we are going to review their work. During this we are going to find out and conclude the result and working of different Human action recognition techniques. In this process we are going to serve our focus on some specific action resignation techniques like: Action Recognition System using 3D Convolutional Neural Networks, Using Deep Learning, Deep Convolutional Neural Networks, Image Processing.

Knowledge Mapping of Human Activity Recognition Techniques for Assistive Living

Purpose: Human Activity Recognition (HAR) is a subject of research that identifies an individual’s activities for assistive living. The proliferation of ICT and sensor technology prompted HAR to flourish beyond unfathomable levels, having immense human-centric applications. The development of accurate HAR systems involves complex statistical and computational tasks from signal acquisition to activity classification. This research aims to conduct a systematic review of recent techniques proposed for each stage of HAR application development. Methodology: The review is conducted following Kitchenham principles, using Scopus and Web of Science databases. Firstly, research questions were formulated, followed by the search strategy definition. Based on assessment criteria, 193 papers are shortlisted and thoroughly analyzed to extract research- related information. Results: The techniques identified in 193 articles are comprehensively mapped from four aspects: data acquisition, data preprocessing and feature engineering, learning algorithm, and evaluation. Each technique is examined for its strengths and limitations to assist application developers in selecting the best one for their needs. The prevailing challenges and upcoming research opportunities are thoroughly explored. Conclusion: The ever-expanding literature in the field necessitated an update to the status of HAR literature. Compared to other reviews that focused on specific methods, fields of application, and datatypes, to the best of our understanding, this is the first evaluation of its kind that provides a broader mapping of HAR approaches. The findings of this analysis will provide researchers and newcomers in the field an up-to-date and holistic view of the complete body of work in this area.

Two-Streams: Dark and Light Networks with Graph Convolution for Action Recognition from Dark Videos (Student Abstract)

In this article, we propose a two-stream action recognition technique for recognizing human actions from dark videos. The proposed action recognition network consists of an image enhancement network with Self-Calibrated Illumination (SCI) module, followed by a two-stream action recognition network. We have used R(2+1)D as a feature extractor for both streams with shared weights. Graph Convolutional Network (GCN), a temporal graph encoder is utilized to enhance the obtained features which are then further fed to a classification head to recognize the actions in a video. The experimental results are presented on the recent benchmark ``ARID" dark-video database.

PhaseAnti: An Anti-Interference WiFi-Based Activity Recognition System Using Interference-Independent Phase Component

Driven by a wide range of essential applications, significant achievements are made to explore WiFi-based Human Activity Recognition (HAR) techniques that utilize the information collected by commercial off-the-shelf WiFi infrastructures to infer human activities without the need for subjects to carry any devices. Although existing WiFi-based HAR systems achieve satisfactory performance in some instances, they are faced with a severe challenge that the impacts of ubiquitous Co-channel Interference (CCI) on WiFi signals are inevitable. This downgrades the performance of these HAR systems significantly. To address this challenge, we propose PhaseAnti, a novel WiFi-based HAR system to exploit the CCI-independent phase component, Nonlinear Phase Error Variation (NLPEV), of WiFi Channel State Information to cope with the negative effects of CCI. The stability of NLPEV data and the sensibility of this component to motions are rigorously analyzed. Furthermore, validated by extensive properly designed experiments, this phase component across subcarriers is invariant under various CCI scenarios while sufficiently distinct for different motions. Therefore, the NLPEV data can be used and processed effectively to perform HAR in CCI scenarios. Extensive experiments with various daily activities in different indoor rooms demonstrate the superior effectiveness and generalizability of the proposed PhaseAnti system under various CCI scenarios.