Human Action Recognition Method Research Articles

A novel deep learning architecture and MINIROCKET feature extraction method for human activity recognition using ECG, PPG and inertial sensor dataset

A novel deep learning architecture and MINIROCKET feature extraction method for human activity recognition using ECG, PPG and inertial sensor dataset

Multi‐modal fusion method for human action recognition based on IALC

In occlusion and interaction scenarios, human action recognition (HAR) accuracy is low. To address this issue, this paper proposes a novel multi-modal fusion framework for HAR. In this framework, a module called improved attention long short-term memory (IAL) is proposed, which combines the improved SE-ResNet50 (ISE-ResNet50) with long short-term memory (LSTM). IAL can extract the video sequence features and the skeleton sequence features of human behaviour. To improve the performance of HAR at a high semantic level, the obtained multi-modal sequence features are fed into a couple hidden Markov model (CHMM), and a multi-modal IAL+CHMM method called IALC is developed based on a probability graph model. To test the performance of the proposed method, experiments are conducted on the HMDB51, UCF101, Kinetics 400k, and ActivityNet datasets, and the obtained recognition accuracy are 86.40%, 97.78%, 81.12%, and 69.36% on the four datasets, respectively. The experimental results show that when the environment is complex, the proposed multi-modal fusion method for HAR based on the IALC can achieve more accurate target recognition results.

Employment of Ensemble Machine Learning Methods for Human Activity Recognition.

The endeavor to detect human activities and behaviors is targeted as a real-time detection mechanism that tends to predict the form of human motions and actions. Though sensors like accelerometer and gyroscopes are noticeable in human motion detection, categorizing unique and individual human gestures require software-based assistance. With the widespread implementation of machine learning algorithms, human actions can be distinguished into multiple classes. Several state-of-the-art machine learning algorithms can be applied to this specified field which will give suitable outcomes, yet due to the bulk of the dataset, complexity can be made apparent, which will reduce the efficiency of the model. In our proposed research, ensemble learning methods have been established by assembling several trained and tuned machine learning models. The adopted dataset for the model has been preprocessed through PCA (principal component analysis), SMOTE oversampling (synthetic minority oversampling technique), and K-means clustering, which reduced the dataset to essentials, keeping the weight of the features intact and reducing complexity. Maximum accuracy of 99.36% was achieved from both stacking and voting ensemble methods.

Device-Free Multi-Location Human Activity Recognition Using Deep Complex Network.

Wi-Fi-based human activity recognition has attracted broad attention for its advantages, which include being device-free, privacy-protected, unaffected by light, etc. Owing to the development of artificial intelligence techniques, existing methods have made great improvements in sensing accuracy. However, the performance of multi-location recognition is still a challenging issue. According to the principle of wireless sensing, wireless signals that characterize activity are also seriously affected by location variations. Existing solutions depend on adequate data samples at different locations, which are labor-intensive. To solve the above concerns, we present an amplitude- and phase-enhanced deep complex network (AP-DCN)-based multi-location human activity recognition method, which can fully utilize the amplitude and phase information simultaneously so as to mine more abundant information from limited data samples. Furthermore, considering the unbalanced sample number at different locations, we propose a perception method based on the deep complex network-transfer learning (DCN-TL) structure, which effectively realizes knowledge sharing among various locations. To fully evaluate the performance of the proposed method, comprehensive experiments have been carried out with a dataset collected in an office environment with 24 locations and five activities. The experimental results illustrate that the approaches can achieve 96.85% and 94.02% recognition accuracy, respectively.

A novel fuzzy clustering-based method for human activity recognition in cloud-based industrial IoT environment

A novel fuzzy clustering-based method for human activity recognition in cloud-based industrial IoT environment

GRU with Dual Attentions for Sensor-Based Human Activity Recognition

Human Activity Recognition (HAR) is nowadays widely used in intelligent perception and medical detection, and the use of traditional neural networks and deep learning methods has made great progress in this field in recent years. However, most of the existing methods assume that the data has independent identical distribution (I.I.D.) and ignore the data variability of different individual volunteers. In addition, most deep learning models are characterized by many parameters and high resources consumption, making it difficult to run in real time on embedded devices. To address these problems, this paper proposes a Gate Recurrent Units (GRU) network fusing the channel attention and the temporal attention for human activity recognition method without I.I.D. By using channel attention to mitigate sensor data bias, GRU and the temporal attention are used to capture important motion moments and aggregate temporal features to reduce model parameters. Experimental results show that our model outperforms existing methods in terms of classification accuracy on datasets without I.I.D., and reduces the number of model parameters and resources consumption, which can be easily used in low-resource embedded devices.

Multi-expert human action recognition with hierarchical super-class learning

In still image human action recognition, existing studies have mainly leveraged extra bounding box information along with class labels to mitigate the lack of temporal information in still images. However, preparing additional annotations such as human and object bounding boxes is time-consuming and also prone to human errors because these annotations are prepared manually. In this paper, we propose a two-phase multi-expert classification method for human action recognition by means of super-class learning and without any extra information. Specifically, a coarse-grained phase selects the most relevant fine-grained experts. Then, the fine-grained experts encode the intricate details within each super-class so that the inter-class variation increases. In the proposed approach, to choose the best configuration for each super-class and characterize inter-class dependency between different action classes, we propose a novel Graph-Based Class Selection (GCS) algorithm. Moreover, the proposed method copes with long-tailed distribution, which the existing studies have not addressed in action recognition. Extensive experimental evaluations are conducted on various public human action recognition datasets, including Stanford40, Pascal VOC 2012 Action, BU101+, and IHAR datasets. The experimental results demonstrate that the proposed method yields promising improvements. To be more specific, in IHAR, Sanford40, Pascal VOC 2012 Action, and BU101+ benchmarks, the proposed approach outperforms the state-of-the-art studies by 8.92%, 0.41%, 0.66%, and 2.11% with much less computational cost and without any auxiliary annotation information. Besides, it is proven that in addressing action recognition with long-tailed distribution, the proposed method outperforms its counterparts by a significant margin.

The Simulation of Aerobics Performance Based on Virtual Reality

The competition aerobics training project is based on the continuous summarization of experience, and on the basis of maintaining the characteristics of its own project, it also actively draws on the structure of other types of skills, and it is developed on the basis of further enriching the new “health, strength, and beauty” characteristics of competition aerobics training. This paper studies and analyzes the simulation of aerobics performance based on virtual reality. This paper firstly gives the definition of virtual reality and then studies and analyzes the action simulation recognition algorithm. The calculation methods of human action recognition usually include two types, the behavior recognition algorithm corresponding to the template and the behavior recognition algorithm based on the state space. And this paper studies the number of formation changes and the speed of the fitness aerobics complete set of movements. This paper analyzes the number and space usage of the aerobics complete sets of formation changing venues and also studies the positioning and moving paths of the aerobics complete sets of movement players. The experimental results show that the use of a complete set of three-dimensional space basically meets the requirements. The use of three-dimensional space in group B is more abundant than that in group A, and the total number of five types of space transformations is more than 10. The three groups and the two types of space have more conversions to reflect the sense of hierarchy, making the complete set of movements more active, but the number of conversions should be increased.

Action Recognition, Tracking, and Optimization Analysis of Training Process Based on the Support Vector Regression Model.

In order to study the action recognition, tracking, and optimization of the training process based on the support vector regression model, a method of human action recognition based on support vector machine optimization is proposed. This method uses the improved strategy of support vector machine to realize the action recognition through the human action recognition based on the optimization of the vector machine. During the recognition, the DAG SVM strategy is improved according to the recognition accuracy of the classifier, and when outputting the result, output the recognition result and the corresponding confidence level, and use the confidence level to process the recognition result. Finally, through the experimental results, it is realized that the recognition rate based on support vector optimization is 98.7%, indicating that this method is effective and can improve the accuracy and efficiency of human body action recognition.

Multilabel Classification Methods for Human Activity Recognition: A Comparison of Algorithms.

As the world’s population is aging, and since access to ambient sensors has become easier over the past years, activity recognition in smart home installations has gained increased scientific interest. The majority of published papers in the literature focus on single-resident activity recognition. While this is an important area, especially when focusing on elderly people living alone, multi-resident activity recognition has potentially more applications in smart homes. Activity recognition for multiple residents acting concurrently can be treated as a multilabel classification problem (MLC). In this study, an experimental comparison between different MLC algorithms is attempted. Three different techniques were implemented: RAkEL, classifier chains, and binary relevance. These methods are evaluated using the ARAS and CASAS public datasets. Results obtained from experiments have shown that using MLC can recognize activities performed by multiple people with high accuracy. While RAkEL had the best performance, the rest of the methods had on-par results.

Human activity recognition in WBAN using ensemble model

PurposeFor achieving the profitable human activity recognition (HAR) method, this paper solves the HAR problem under wireless body area network (WBAN) using a developed ensemble learning approach. The purpose of this study is,to solve the HAR problem under WBAN using a developed ensemble learning approach for achieving the profitable HAR method. There are three data sets used for this HAR in WBAN, namely, human activity recognition using smartphones, wireless sensor data mining and Kaggle. The proposed model undergoes four phases, namely, “pre-processing, feature extraction, feature selection and classification.” Here, the data can be preprocessed by artifacts removal and median filtering techniques. Then, the features are extracted by techniques such as “t-Distributed Stochastic Neighbor Embedding”, “Short-time Fourier transform” and statistical approaches. The weighted optimal feature selection is considered as the next step for selecting the important features based on computing the data variance of each class. This new feature selection is achieved by the hybrid coyote Jaya optimization (HCJO). Finally, the meta-heuristic-based ensemble learning approach is used as a new recognition approach with three classifiers, namely, “support vector machine (SVM), deep neural network (DNN) and fuzzy classifiers.” Experimental analysis is performed.Design/methodology/approachThe proposed HCJO algorithm was developed for optimizing the membership function of fuzzy, iteration limit of SVM and hidden neuron count of DNN for getting superior classified outcomes and to enhance the performance of ensemble classification.FindingsThe accuracy for enhanced HAR model was pretty high in comparison to conventional models, i.e. higher than 6.66% to fuzzy, 4.34% to DNN, 4.34% to SVM, 7.86% to ensemble and 6.66% to Improved Sealion optimization algorithm-Attention Pyramid-Convolutional Neural Network-AP-CNN, respectively.Originality/valueThe suggested HAR model with WBAN using HCJO algorithm is accurate and improves the effectiveness of the recognition.

A Novel Feature-Selection Method for Human Activity Recognition in Videos

Human Activity Recognition (HAR) is the process of identifying human actions in a specific environment. Recognizing human activities from video streams is a challenging task due to problems such as background noise, partial occlusion, changes in scale, orientation, lighting, and the unstable capturing process. Such multi-dimensional and none-linear process increases the complexity, making traditional solutions inefficient in terms of several performance indicators such as accuracy, time, and memory. This paper proposes a technique to select a set of representative features that can accurately recognize human activities from video streams, while minimizing the recognition time and memory. The extracted features are projected on a canvas, which keeps the synchronization property of the spatiotemporal information. The proposed technique is developed to select the features that refer only to progression of changes. The original RGB frames are preprocessed using background subtraction to extract the subject. Then the activity pattern is extracted through the proposed Growth method. Three experiments were conducted; the first experiment was a baseline to compare the classification task using the original RGB features. The second experiment relied on classifying activities using the proposed feature-selection method. Finally, the third experiment provided a sensitivity analysis that compares between the effect of both techniques on time and memory resources. The results indicated that the proposed method outperformed original RBG feature-selection method in terms of accuracy, time, and memory requirements.

Review on recent Computer Vision Methods for Human Action Recognition


 
 
 The subject of human activity recognition is considered an important goal in the domain of computer vision from the beginning of its development and has reached new levels. It is also thought of as a simple procedure. Problems arise in fast-moving and advanced scenes, and the numerical analysis of artificial intelligence (AI) through activity prediction mistreatment increased the attention of researchers to study. Having decent methodological and content related variations, several datasets were created to address the evaluation of these ways. Human activities play an important role but with challenging characteristic in various fields. Many applications exist in this field, such as smart home, helpful AI, HCI (Human-Computer Interaction), advancements in protection in applications such as transportation, education, security, and medication management, including falling or helping elderly in medical drug consumption. The positive impact of deep learning techniques on many vision applications leads to deploying these ways in video processing. Analysis of human behavior activities involves major challenges when human presence is concerned. One individual can be represented in multiple video sequences through skeleton, motion and/or abstract characteristics. This work aims to address human presence by combining many options and utilizing a new RNN structure for activities. The paper focuses on recent advances in machine learning-assisted action recognition.
 Existing modern techniques for the recognition of actions and prediction similarly because the future scope for the analysis is mentioned accuracy within the review paper.
 
 

Using a Selective Ensemble Support Vector Machine to Fuse Multimodal Features for Human Action Recognition.

The traditional human action recognition (HAR) method is based on RGB video. Recently, with the introduction of Microsoft Kinect and other consumer class depth cameras, HAR based on RGB-D (RGB-Depth) has drawn increasing attention from scholars and industry. Compared with the traditional method, the HAR based on RGB-D has high accuracy and strong robustness. In this paper, using a selective ensemble support vector machine to fuse multimodal features for human action recognition is proposed. The algorithm combines the improved HOG feature-based RGB modal data, the depth motion map-based local binary pattern features (DMM-LBP), and the hybrid joint features (HJF)-based joints modal data. Concomitantly, a frame-based selective ensemble support vector machine classification model (SESVM) is proposed, which effectively integrates the selective ensemble strategy with the selection of SVM base classifiers, thus increasing the differences between the base classifiers. The experimental results have demonstrated that the proposed method is simple, fast, and efficient on public datasets in comparison with other action recognition algorithms.

Deep learning method for human activity recognition using heaped LSTM and image pattern of activity

Deep learning method for human activity recognition using heaped LSTM and image pattern of activity

A Novel Selective Ensemble Learning Method for Smartphone Sensor-Based Human Activity Recognition Based on Hybrid Diversity Enhancement and Improved Binary Glowworm Swarm Optimization

Human activity recognition (HAR) is gaining interest with many important applications including ubiquitous computing, health-care services and detection of diseases. Smartphone sensors have high acceptance and adherence in daily life and they provide an alternative and economic way for activity recognition. To improve the performance of smartphone sensor-based HAR, a novel smartphone sensor-based HAR method (hybrid diversity enhancement with selective ensemble learning, HDESEN) that utilizes selective ensemble learning with differentiated extreme learning machines (ELMs) is proposed, where hybrid diversity enhancement is proposed to boost the diversity of base models and an improved binary glowworm swarm optimization (IBGSO) is employed to effectively enhance the learning process by choosing a superior subset for ensemble instead of all. Firstly, statistical features in the time domain and frequency domain are extracted and integrated from smartphone sensors and then three filter-based feature selection methods are utilized for desirable base models. Secondly, to enhance the diversity of the base models, three types of diversities are introduced to construct different base models, respectively. Among them, Bootstrap is introduced to design distinctive training data subsets for differential base models, random subspace and optimized subspace are proposed to obtain different feature spaces for constructing base models. Thirdly, a pruning method based on glowworm swarm optimization (GSO) is proposed to find the optimal sub-ensemble from the pool of models from all diverse types to implement selective ensemble learning. The experimental results on tow publicly available datasets (UCI-HAR and WISDM) demonstrate the proposed HDESEN can reliably improve the performance of HAR and outperforms the relevant state-of-the-art approaches.

Human Complex Activity Recognition With Sensor Data Using Multiple Features

In order to achieve better results of human complex activity recognition, the optimal feature representation of human complex activities from sensor data are studied. Features from multiple sets that are obtained by the learned networks, the manual extraction, and the location information are fused to generate mixed features to achieve better results of human complex activity recognition. A method extracting multi-layer features from the hybrid CNN and BLSTM network is proposed. The output feature vector of the second layer of the CNN and that of the BLSTM are combined to generate multi-layer features. Meanwhile, a new feature selection method based on SFS and network weight analysis is proposed. The method is to select a series of features extracted from the segmented sensor data using manual methods as dominant features. The location information of complex activities after one-hot encoding is also used as feature, which is then fused with multi-layer features and manual dominant features to generate mixed features. To further improve the recognition performance, the sensor data collected at different body positions are separated and used independently to train the hybrid CNN and BLSTM network to obtain their respective state information. The PAMAP2 and UT-Data datasets are used in our experiments to verify the proposed method. The results show that the method based on multiple features proposed in this paper dramatically outperforms the existing state-of-the-art methods for human complex activity recognition using sensor data.

Toward Vehicle Occupant-Invariant Models for Activity Characterization

With the advent of self-driving cars and the push by large companies into fully driverless transportation services, monitoring passenger behaviour in vehicles is becoming increasingly important for several reasons, such as ensuring safety and comfort. Although several human action recognition (HAR) methods have been proposed, developing a true HAR system remains a very challenging task. If the dataset used to train a model contains a small number of actors, the model can become biased towards these actors and their unique characteristics. This can cause the model to generalise poorly when confronted with new actors performing the same actions. This limitation is particularly acute when developing models to characterise the activities of vehicle occupants, for which data sets are short and scarce. In this study, we describe and evaluate three different methods that aim to address this actor bias and assess their performance in detecting in-vehicle violence. These methods work by removing specific information about the actor from the model’s features during training or by using data that is independent of the actor, such as information about body posture. The experimental results show improvements over the baseline model when evaluated with real data.

Real time dangerous action warning system based on graph convolution neural network

In order to solve a series of disadvantages of slow efficiency and high cost brought by the traditional human action recognition method based on manual feature extraction, in order to prove the feasibility of using graph convolution neural network and action recognition, a real-time dangerous action alarm system based on graph convolution neural network is designed and implemented. The system includes data collection, human posture estimation, action recognition, dangerous action alarm and other functions. The feasibility of using graph convolution neural network to realize real-time dangerous action alarm system is proved by the system. Even if the graph convolution neural network does not pass the training, it can extract good features only by using the original random initialization parameters. If certain annotation information is given, its effect can be greatly improved. After the system adopts the graph convolution neural network, the action recognition can be more accurate, more efficient, faster and more reliable than the traditional method.

A Shoe-Integrated Sensor System for Long- Term Center of Pressure Evaluation

In clinical, the center of pressure (CoP) is commonly used for accessing the stability of a person’s postural control, which is highly associated with various neurological diseases and movement disorders such as Alzheimer’s disease, Parkinson’s disease, chronic ankle instability. Such a disease usually has a long development or rehabilitation process which requires long-term CoP monitoring. The current CoP evaluation process does not meet the requirement, as it is often complicated and expensive through either the lab-based equipment or the clinical evaluation procedure. Different wearable sensor-based systems with less cost and restrictions have emerged, but their way of CoP calculation requires deliberate calibration of the positions of their sensors, which are not feasible in daily CoP monitoring. In this study, we developed a long-term CoP monitoring system in a smart-shoe form. First, a thin and flexible smart insole with optimal sensor locations was designed to be compact and energy sufficient for a whole-day usage. Then, a user-friendly app on the smartphone with a cloud-based data managing system was developed for applications in both clinical and home environments. Additionally, a simplified CoP estimation model was created without the need for calibration. Lastly, a machine learning-based human activity recognition method was incorporated to make the CoP detection process more automatic. Through a thorough validation test with the clinical level lab equipment, our system can generate the CoP measurements with high accuracy.