Activity Recognition Algorithms Research Articles

Activity recognition using a combination of high gain observer and deep learning computer vision algorithms

Inertial sensors have become increasingly popular in human activity classification due to their ease of use and affordability. This paper proposes a novel algorithm for human activity recognition that is a combination of a high-gain observer and deep learning computer vision classification algorithms. The nonlinear high-gain observer designed using Lyapunov analysis accurately estimates the attitude of the chest of a human subject using measurements from a single Inertial Measurement Unit (IMU). The signals processed by the observer are then converted into spectrograms to obtain “images” of the frequency response of the signals. The images for activities from a dataset of 7 human subjects are annotated and used for training/ fine-tuning of several well-known deep learning algorithms for image processing. The results from the best combination of our algorithms shows an exceptional accuracy of 98% for activity recognition. Using deep learning computer vision algorithms, this paper shows how to perform transfer learning from networks pre-trained on millions of images, thus showing how we can train a powerful deep learning network for activity recognition even with just small datasets. The algorithm that uses the high gain observer is shown to perform significantly better than an algorithm based on raw accelerometer and gyro signals.

Real world validation of activity recognition algorithm and development of novel behavioral biomarkers of falls in aged control and movement disorder patients.

The use of wearable sensors in movement disorder patients such as Parkinson's disease (PD) and normal pressure hydrocephalus (NPH) is becoming more widespread, but most studies are limited to characterizing general aspects of mobility using smartphones. There is a need to accurately identify specific activities at home in order to properly evaluate gait and balance at home, where most falls occur. We developed an activity recognition algorithm to classify multiple daily living activities including high fall risk activities such as sit to stand transfers, turns and near-falls using data from 5 inertial sensors placed on the chest, upper-legs and lower-legs of the subjects. The algorithm is then verified with ground truth by collecting video footage of our patients wearing the sensors at home. Our activity recognition algorithm showed >95% sensitivity in detection of activities. Extracted features from our home monitoring system showed significantly better correlation (~69%) with prospectively measured fall frequency of our subjects compared to the standard clinical tests (~30%) or other quantitative gait metrics used in past studies when attempting to predict future falls over 1 year of prospective follow-up. Although detecting near-falls at home is difficult, our proposed model suggests that near-fall frequency is the most predictive criterion in fall detection through correlation analysis and fitting regression models.

Individual Surveillance Around Parked Aircraft at Nighttime: Thermal Infrared Vision-Based Human Action Recognition

The COVID-19 crisis has led to an unusually large number of commercial aircraft being currently parked or stored. For airlines, airports, and civil aviation authorities around the world, monitoring, and protecting these parked aircraft to prevent them from causing human-made damage are becoming urgent problems that are receiving increasing attention. In this study, we use thermal infrared monitoring videos to establish a framework for individual surveillance around parked aircraft by proposing a human action recognition (HAR) algorithm. As the focus of this article, the proposed HAR algorithm seamlessly integrates a preprocessing module in which a novel data structure is constructed to introduce spatiotemporal information of the action; a convolutional neural network-based module for spatial feature extraction; a triple-layer convolutional long short-term memory network for temporal feature extraction; and two fully connected layers for classification. Moreover, because no infrared dataset is available for the HAR task on airport grounds at nighttime, we present a dataset called IIAR-30, which consists of eight action categories that frequently occur on airport grounds and 2000 video clips. The experimental results on the IIAR-30 dataset demonstrated that the recognition accuracy of the proposed method was higher than 96%. We also further evaluated the effectiveness of the proposed method by comparing it with five baselines and four other methods.

Improved Shark Smell Optimization Algorithm for Human Action Recognition

Improved Shark Smell Optimization Algorithm for Human Action Recognition

Algorithm for Skeleton Action Recognition by Integrating Attention Mechanism and Convolutional Neural Networks

Algorithm for Skeleton Action Recognition by Integrating Attention Mechanism and Convolutional Neural Networks

A review on learning-based algorithms for human activity recognition

A review on learning-based algorithms for human activity recognition

A Review on Learning-based Algorithms for Human Activity Recognition

A Review on Learning-based Algorithms for Human Activity Recognition

Use of Clustering Algorithms for Sensor Placement and Activity Recognition in Smart Homes

This work presents a novel method for motion sensor placement within smart homes. Using recordings from 3D depth cameras within six real homes, clusters are created with the resident’s tracked location. The resulting clusters identify the possible position of a sensor and its field of view. By using a sequence of clusters as input to a Recurrent Neural Network, we evaluate our method on the task of activity recognition and prediction. These results are compared to using sensor events as input sequence, from motion sensors that were installed empirically in the same homes. Different clustering methods are investigated and all outperform the installed motion sensors, achieving a significant increase of prediction accuracy and F1-score.

Adaptive Weighted Flow Net Algorithm for Human Activity Recognition Using Depth Learned Features

Human Activity Recognition (HAR) from video data collections is the core application in vision tasks and has a variety of utilizations including object detection applications, video-based behavior monitoring, video classification, and indexing, patient monitoring, robotics, and behavior analysis. Although many techniques are available for HAR in video analysis tasks, most of them are not focusing on behavioral analysis. Hence, a new HAR system analysis the behavioral activity of a person based on the deep learning approach proposed in this work. The most essential aim of this work is to recognize the complex activities that are useful in many tasks that are based on object detection, modelling of individual frame characteristics, and communication among them. Moreover, this work focuses on finding out the human actions from various video resolutions, invariant human poses, and nearness of multi objects. First, we identify the key and essential frames of each activity using histogram differences. Secondly, Discrete Wavelet Transform (DWT) is used in this system to extract coefficients from the sequence of key-frames where the activity is localized in space. Finally, an Adaptive Weighted Flow Net (AWFN) algorithm is proposed in this work for effective video activity recognition. Moreover, the proposed algorithm has been evaluated by comparing it with the existing Visual Geometry Group (VGG-16) convolution neural networks for making performance comparisons. This work focuses on competent deep learning-based feature extraction to discriminate the activities for performing the classification accuracy. The proposed model has been evaluated with VGG-16 using a combination of regular UCF-101 activity datasets and also in very challenging Low-quality videos such as HMDB51. From these investigations, it is proved that the proposed AWFN approach gives higher detection accuracy of 96%. It is approximately 0.3% to 7.88% of higher accuracy than state-of-art methods.

Retracted: Construction of Intelligent Nursing System Based on Visual Action Recognition Algorithm.

[This retracts the article DOI: 10.1155/2022/8311371.].

Research on Human Activity Recognition Algorithm Based on LSTM-1DCNN

Research on Human Activity Recognition Algorithm Based on LSTM-1DCNN

Data Valuation Algorithm for Inertial Measurement Unit-Based Human Activity Recognition.

This paper proposes a data valuation algorithm for inertial measurement unit-based human activity recognition (IMU-based HAR) data based on meta reinforcement learning. Unlike previous studies that received feature-level input, the algorithm in this study added a feature extraction structure to the data valuation algorithm, and it can receive raw-level inputs and achieve excellent performance. As IMU-based HAR data are multivariate time-series data, the proposed algorithm incorporates an architecture capable of extracting both local and global features by inserting a transformer encoder after the one-dimensional convolutional neural network (1D-CNN) backbone in the data value estimator. In addition, the 1D-CNN-based stacking ensemble structure, which exhibits excellent efficiency and performance on IMU-based HAR data, is used as a predictor to supervise model training. The Berg balance scale (BBS) IMU-based HAR dataset and the public datasets, UCI-HAR, WISDM, and PAMAP2, are used for performance evaluation in this study. The valuation performance of the proposed algorithm is observed to be excellent on IMU-based HAR data. The rate of discovering corrupted data is higher than 96% on all datasets. In addition, classification performance is confirmed to be improved by the suppression of discovery of low-value data.

Developing a novel hybrid method based on dispersion entropy and adaptive boosting algorithm for human activity recognition

BackgroundWith the rapid development of technology, human activity recognition (HAR) from sensor data has become a key element for many real-world applications, such as healthcare, disease diagnosis and smart home systems. Although there have been several studies conducted on HAR, traditional methods remain inadequate in balancing efficiency, accuracy and speed. Moreover, existing studies have not identified a solution to managing imbalanced data in different activities groups of HAR, although that is major issue in determining satisfactory performance. Methodsthis study proposes a new hybrid approach involving hierarchical dispersion entropy (HDE) and Adaptive Boosting with convolutional neural networks (AdaB_CNN) for classifying human activities, such as running downstairs/upstairs, walking and other daily activities, from sensor data. The proposed model is comprised of the following steps: firstly, HAR data are segmented into intervals using a sliding window technique, and then the segmented data are decomposed into different frequency bands. Following this, the dispersion entropy of different frequency bands is computed to produce a feature vector set. Then, the extracted features are reduced using Joint Approximate Diagonalization of Eigenmatrices (JADE) to further eliminate redundant information. The final feature vector set is then fed into the AdaB_CNN method to classify human activities. ResultsThe proposed approach is tested on three publicly available datasets: WISDM, UCI_HAR 2012, and PAMAP2. The experimental results demonstrate that the proposed model attains a superior performance in HAR to most current methods. ConclusionsThe findings reveal that the proposed HDE based AdaB_CNN model has the capability to efficiently recognize different activities from sensor technologies. It has the potential to be implemented in a hardware system to classify human activity.

Benchmarking Classification Algorithms for Radar-Based Human Activity Recognition

Linked to the increasing availability of datasets for radar-based human activity recognition (HAR), in this Student Highlights contribution, we report on a classification project that a group of 23 graduate students performed at TU Delft. The students were asked to work in groups of 2–3 members and to use the publicly available University of Glasgow dataset to develop the best classification pipeline as possible. This involved development and justification of both choices for the preprocessing techniques on the radar data (e.g., time–frequency distributions and cleaning of the signatures), and for the classification algorithms (e.g., the type of the algorithm, the hyperparameters’ selection, the training–validation–testing split). While this student activity was performed at a small scale and with educational rather than research aims, we are happy to report it to the AESS readership, as we believe that such initiatives with open datasets sharing and classification algorithm benchmarking are beneficial for the wider radar research community. Furthermore, a list of publicly available datasets for radar-based HAR that can be used for similar initiatives is also reported in this article.

A Survey of Human Action Recognition and Posture Prediction

Human action recognition and posture prediction aim to recognize and predict respectively the action and postures of persons in videos. They are both active research topics in computer vision community, which have attracted considerable attention from academia and industry. They are also the precondition for intelligent interaction and human-computer cooperation, and they help the machine perceive the external environment. In the past decade, tremendous progress has been made in the field, especially after the emergence of deep learning technologies. Hence, it is necessary to make a comprehensive review of recent developments. In this paper, firstly, we attempt to present the background, and then discuss research progresses. Secondly, we introduce datasets, various typical feature representation methods, and explore advanced human action recognition and posture prediction algorithms. Finally, facing the challenges in the field, this paper puts forward the research focus, and introduces the importance of action recognition and posture prediction by taking interactive cognition in self-driving vehicle as an example.

Acceptability of In-Home Ambient Sensors for Activity Recognition and Assessment Post-Stroke

<h3>Research Objectives</h3> To explore the acceptability of an in-home ambient sensor for activity recognition and assessment post-stroke. <h3>Design</h3> Single arm pilot study. The DNS-6 depth sensor (Foresite Healthcare) was installed in the kitchen of enrolled participants for 3 months. Data collected from the depth sensor was used to develop, train, and test an algorithm for activity recognition and assessment. <h3>Setting</h3> Community homes. <h3>Participants</h3> Twenty individuals with chronic stroke were recruited via a stroke registry at the University of Missouri. Individuals were eligible if they had experienced a stroke at least 6 months prior, were able to ambulate with or without an assistive device, and reported some difficulty using their arm in everyday activities. <h3>Interventions</h3> Not applicable. <h3>Main Outcome Measures</h3> Semi-structured interviews were conducted with each study participant at the end of the 3-month period. Questions were focused on the study participant's attitudes and reactions to having the depth sensor in their home. <h3>Results</h3> Most study participants "forgot" that the sensor was installed in their home and went about their daily activities like normal. None of the study participants expressed concerns about the overall aesthetic of the depth sensor in their home. Some study participants expressed concerns about data privacy; however, none felt that their privacy or confidentiality was broken during their time in the study. All study participants agreed that the benefits of the depth sensor and data provided for clinicians outweighed any risks or privacy concerns. Some study participants wanted to see greater integration with other technologies such as video games and wearable devices to track activity in the home and community. <h3>Conclusions</h3> Despite the privacy concerns, all participants were able to understand the anticipated benefits and clinical implications of this sensor. This is consistent with past literature as most participants in Demiris et. al saw value and purpose of monitoring (2009). Future research will integrate the depth sensor and developed algorithm into a larger home-based sensor system for individuals post-stroke and other conditions. <h3>Author(s) Disclosures</h3> None

Lightweight Semantic-Guided Neural Networks Based on Single Head Attention for Action Recognition

Skeleton-based action recognition can achieve a relatively high performance by transforming the human skeleton structure in an image into a graph and applying action recognition based on structural changes in the body. Among the many graph convolutional network (GCN) approaches used in skeleton-based action recognition, semantic-guided neural networks (SGNs) are fast action recognition algorithms that hierarchically learn spatial and temporal features by applying a GCN. However, because an SGN focuses on global feature learning rather than local feature learning owing to the structural characteristics, there is a limit to an action recognition in which the dependency between neighbouring nodes is important. To solve these problems and simultaneously achieve a real-time action recognition in low-end devices, in this study, a single head attention (SHA) that can overcome the limitations of an SGN is proposed, and a new SGN-SHA model that combines SHA with an SGN is presented. In experiments on various action recognition benchmark datasets, the proposed SGN-SHA model significantly reduced the computational complexity while exhibiting a performance similar to that of an existing SGN and other state-of-the-art methods.

SlowFast Action Recognition Algorithm Based on Faster and More Accurate Detectors

Object detection algorithms play a crucial role in other vision tasks. This paper finds that the action recognition algorithm SlowFast’s detection algorithm FasterRCNN (Region Convolutional Neural Network) has disadvantages in terms of both detection accuracy and speed and the traditional IOU (Intersection over Union) localization loss is difficult to make the detection model converge to the minimum stability point. To solve the above problems, the article uses YOLOv3 (You Only Look Once), YOLOX, and CascadeRCNN to improve the detection accuracy and speed of the SlowFast. This paper proposes a new localization loss function that adopts the Lance and Williams distance as a new penalty term. The new loss function is more sensitive when the distance difference is smaller, and this property is very suitable for the late convergence of the detection model. The experiments were conducted on the VOC (Visual Object Classes) dataset and the COCO dataset. In the final videos test, YOLOv3 improved the detection speed by 10.5 s. CascadeRCNN improved by 3.1%AP compared to FasterRCNN in the COCO dataset. YOLOX’s performance on the COCO dataset is also mostly better than that of FasterRCNN. The new LIOU (Lance and Williams Distance Intersection over Union) localization loss function performs better than other loss functions in the VOC dataset. It can be seen that improving the detection algorithm of the SlowFast seems to be crucial and the proposed loss function is indeed effective.

Development of an intrinsic health risk prediction model for camera-based monitoring of older adults living alone

The number of older adults in Korea is increasing, along with the number of depressed older patients. The causes of depression in older adults include social isolation with negligible interaction with others, irregular nutritional habits, and self-negligence, i.e., they do not engage in any activity. These factors, self-negligence, social isolation, and irregular nutritional habits, are defined as inherent health risks, and in this study, we detected them. These factors can only be derived through long-term monitoring, but the current monitoring system for older adults is severely limited as it focuses only on emergencies, such as “falls.” Therefore, in this study, the goal was to perform long-term monitoring using a camera. In order to capture the physical characteristics of the older adults, the ETRI-Activity3D data were used for training, and the skeleton-based action recognition algorithm Posec3d was used. By defining 90 frames as the time taken for one action, we built a monitoring system to enable long-term monitoring of older adult by performing multiple action detection in one video. A reliable monitoring system, with 98% accuracy, 98% precision, 99% recall, and 98% F1, was successfully established for health monitoring of older adults. This older adult monitoring technology is expected to improve the quality of medical services in a medical environment as well as the objective, activities of daily living test, which does not depend on the observer through daily life detection.

Representative‐discriminativedictionary learning algorithm for human action recognition using smartphone sensors

SUMMARYWith the advancement of mobile computing, understanding, and interpretation of human activities has become increasingly popular as an innovative human computer interaction application over the past few decades. This article presents a new scheme for action recognition based on sparse representation theory using a novel dictionary learning algorithm. This system employs two types of inertial signals from smartphones namely, accelerometer and gyroscope sensory data. Attainment of higher values of classification accuracy depends on the creation of effective dictionaries that completely retain the important features of every action while maintaining the least correlation with the features of other actions. Accordingly, in this research, we propose a new algorithm for learning dictionaries with two levels of dictionary training that aims at learning a compact, representative, and discriminative dictionary for each class. Unlike typical dictionary learning algorithms that aim at the creation of dictionaries that best represents the features of each class, our proposed algorithm incorporates a discriminative criterion that eventually produces better classification results. To validate the proposed framework, all the experiments were performed using three publicly available datasets.