Activity Recognition Algorithms Research Articles

ENGA: Elastic Net-Based Genetic Algorithm for human action recognition

ENGA: Elastic Net-Based Genetic Algorithm for human action recognition

Activity Recognition Algorithms based on Sensor Data for Wearable Walking-Assistive Robots

Activity Recognition Algorithms based on Sensor Data for Wearable Walking-Assistive Robots

A Novel Multiperson Activity Recognition Algorithm Based on Point Clouds Measured by Millimeter-Wave MIMO Radar

Radar is widely used in human activity recognition (HAR). The radar-based HAR algorithms can be roughly categorized into the feature map-based and the point cloud-based. Between the two, the point cloud-based algorithms are more suitable for multi-person activity recognition (MPAR) tasks due to the spatial properties of point clouds, and are therefore increasingly attracting attentions. However, most existing point cloud-based algorithms make decisions by utilizing either Doppler or coordinate features of point clouds, which are not sufficient to characterize the activity; the remaining algorithms have a large randomness in extracting both features. Neither of these algorithms can fully exploit the advantage of feature fusion, which is the key to recognition performance. To address this shortcoming, this paper proposes a novel MPAR algorithm. First, a feature mapping approach is proposed and defined in equation form, the Doppler, range, azimuth, and elevation features of the point clouds are calculated and accumulated sequentially, so we can obtain the four time-domain feature maps. Second, with the four feature maps as inputs, a four-channel CNN classification model with channel attention is trained for MPAR tasks. Datasets of multi-person activities are collected respectively under the indoor circumstance and aquatic circumstance using the millimeter-wave multiple-input-multiple-output (MIMO) radar platform. The dataset-based evaluation performance shows that the proposed algorithm achieves accuracy results of 96.09% for the indoor-MPAR task and 93.97% for the aquatic-MPAR task, and outperforms three conventional point cloud-based algorithms in terms of the overall MPAR accuracy, the generalization ability to the aquatic activity recognition, and the robustness of distance and headcount.

TwSense: Highly Robust Through-the-Wall Human Detection Method Based on COTS Wi-Fi Device

With the popularization of Wi-Fi router devices, the application of device-free sensing has garnered significant attention due to its potential to make our lives more convenient. Wi-Fi signal-based through-the-wall human detection offers practical applications, such as emergency rescue and elderly monitoring. However, the accuracy of through-the-wall human detection is hindered by signal attenuation caused by wall materials and multiple propagation paths of interference. Therefore, through-the-wall human detection presents a substantial challenge. In this paper, we proposed a highly robust through-the-wall human detection method based on a commercial Wi-Fi device (TwSense). To mitigate interference from wall materials and other environmental factors, we employed the robust principal component analysis (OR-PCA) method to extract the target signal of Channel State Information (CSI). Subsequently, we segmented the action-induced Doppler shift feature image using the K-means clustering method. The features of the images were extracted using the Histogram of Oriented Gradients (HOG) algorithm. Finally, these features were fed into an SVM classifier (G-SVM) optimized by a grid search algorithm for action classification and recognition, thereby enhancing human detection accuracy. We evaluated the robustness of the entire system. The experimental results demonstrated that TwSense achieved the highest accuracy of 96%.

Human activity recognition algorithm in video sequences based on the fusion of multiple features for realistic and multi-view environment

Human activity recognition algorithm in video sequences based on the fusion of multiple features for realistic and multi-view environment

Progressive Instance-Aware Feature Learning for Compositional Action Recognition.

In order to enable the model to generalize to unseen "action-objects" (compositional action), previous methods encode multiple pieces of information (i.e., the appearance, position, and identity of visual instances) independently and concatenate them for classification. However, these methods ignore the potential supervisory role of instance information (i.e., position and identity) in the process of visual perception. To this end, we present a novel framework, namely Progressive Instance-aware Feature Learning (PIFL), to progressively extract, reason, and predict dynamic cues of moving instances from videos for compositional action recognition. Specifically, this framework extracts features from foreground instances that are likely to be relevant to human actions (Position-aware Appearance Feature Extraction in Section III-B1), performs identity-aware reasoning among instance-centric features with semantic-specific interactions (Identity-aware Feature Interaction in Section III-B2), and finally predicts instances' position from observed states to force the model into perceiving their movement (Semantic-aware Position Prediction in Section III-B3). We evaluate our approach on two compositional action recognition benchmarks, namely, Something-Else and IKEA-Assembly. Our approach achieves consistent accuracy gain beyond off-the-shelf action recognition algorithms in terms of both ground truth and detected position of instances.

A Prior Knowledge-Guided Graph Convolutional Neural Network for Human Action Recognition in Solar Panel Installation Process

Human action recognition algorithms have garnered significant research interest due to their vast potential for applications. Existing human behavior recognition algorithms primarily focus on recognizing general behaviors using a large number of datasets. However, in industrial applications, there are typically constraints such as limited sample sizes and high accuracy requirements, necessitating algorithmic improvements. This article proposes a graph convolution neural network model that combines prior knowledge supervision and attention mechanisms, designed to fulfill the specific action recognition requirements for workers installing solar panels. The model extracts prior knowledge from training data, improving the training effectiveness of action recognition models and enhancing the recognition reliability of special actions. The experimental results demonstrate that the method proposed in this paper surpasses traditional models in terms of recognizing solar panel installation actions accurately. The proposed method satisfies the need for highly accurate recognition of designated person behavior in industrial applications, showing promising application prospects.

System-Identification-Based Activity Recognition Algorithms With Inertial Sensors.

This paper focuses on activity recognition using a single wearable inertial measurement sensor placed on the subject's chest. The ten activities that need to be identified include lying down, standing, sitting, bending and walking, among others. The activity recognition approach is based on using and identifying a transfer function associated with each activity. The appropriate input and output signals for each transfer function are first determined based on the norms of the sensor signals excited by that specific activity. Then the transfer function is identified using training data and a Wiener filter based on the auto-correlation and cross-correlation of the output and input signals. The activity occurring in real-time is recognized by computing and comparing the input-output errors associated with all the transfer functions. The performance of the developed system is evaluated using data from a group of Parkinson's disease subjects, including data obtained in a clinical setting and data obtained through remote home monitoring. On average, the developed system provides better than 90% accuracy in identifying each activity as it occurs. Activity recognition is particularly useful for PD patients in order to monitor their level of activity, characterize their postural instability and recognize high risk-activities in real-time that could lead to falls.

Combination of Different-Granularity Beliefs for Sensor-Based Human Activity Recognition

Human Activity Recognition (HAR) based on sensor information has become a hot topic of research due to its wide range of applications in health-care, fitness and smart homes. However, the classification of activities with similar sensor signals such as standing and sitting is usually more challenging for the design of efficient activity recognition algorithms. Considering the characteristic of human activities with different granularity, which can provide complementary knowledge for individual granularity activity recognition, we propose a novel approach that combines different-granularity beliefs based on belief functions (BFs) theory for HAR. Specifically, at first, two support vector machines (SVM) are trained to acquire different-granularity basic belief assignments (BBA) of human activity, respectively. Then, a prior knowledge library (PKL) is established and the least square method is employed to solve the transformation matrix for mapping coarse-grained BBAs to fine-grained BBAs. Finally, the classic DSm (DSmC) rule is adopted to combine the two sets of fine-grained BBAs and further make decisions. Several experiments are conducted to illustrate the performance of the proposed method using two widely used UCI datasets: Smartphone and mHealth. The experimental results show that the proposed method efficiently improves the classification accuracy with respect to other related methods.

Anomalous Behavior Detection with Spatiotemporal Interaction and Autoencoder Enhancement

To reduce the cargo loss rate caused by abnormal consumption behavior in smart retail cabinets, two problems need to be solved. The first is that the diversity of consumers leads to a diversity of actions contained in the same behavior, which makes the accuracy of consumer behavior identification low. Second, the difference between normal interaction behavior and abnormal interaction behavior is small, and anomalous features are difficult to define. Therefore, we propose an anomalous behavior detection algorithm with human–object interaction graph convolution and confidence-guided difference enhancement. Aiming to solve the problem of low accuracy of consumer behavior recognition, including interactive behavior, the human–object interaction graph convolutional network is used to recognize action and extract video frames of abnormal human behavior. To define anomalies, we detect anomalies by delineating anomalous areas of the anomaly video frames. We use a confidence-guided anomaly enhancement module to perform confidence detection on the encoder-extracted coded features using a confidence full connection layer. The experimental results showed that the action recognition algorithm had good generalization ability and accuracy, and the screened video frames have obvious destruction characteristics, and the area under the receiver operating characteristic (AUROC) curve reached 82.8% in the detection of abnormal areas. Our research provides a new solution for the detection of abnormal behavior that destroys commodity packaging, which has considerable application value.

Using Smart Home Technologies to Promote Physical Activity Among the General and Aging Populations: Scoping Review.

Health-monitoring smart homes are becoming popular, with experts arguing that 9-to-5 health care services might soon become a thing of the past. However, no review has explored the landscape of smart home technologies that aim to promote physical activity and independent living among a wide range of age groups. This review aims to map published studies on smart home technologies aimed at promoting physical activity among the general and aging populations to unveil the state of the art, its potential, and the research gaps and opportunities. Articles were retrieved from 6 databases (PubMed, CINAHL, Scopus, IEEE Xplore, ACM Library, and Web of Science). The criteria for inclusion were that the articles must be user studies that dealt with smart home or Active Assisted Living technologies and physical activity, were written in English, and were published in peer-reviewed journals. In total, 3 researchers independently and collaboratively assessed the eligibility of the retrieved articles and elicited the relevant data and findings using tables and charts. This review synthesized 20 articles that met the inclusion criteria, 70% (14/20) of which were conducted between 2018 and 2020. Three-quarters of the studies (15/20, 75%) were conducted in Western countries, with the United States accounting for 25% (5/20). Activities of daily living were the most studied (9/20, 45%), followed by physical activity (6/20, 30%), therapeutic exercise (4/20, 20%), and bodyweight exercise (1/20, 5%). K-nearest neighbor and naïve Bayes classifier were the most used machine learning algorithms for activity recognition, with at least 10% (2/20) of the studies using either algorithm. Ambient and wearable technologies were equally studied (8/20, 40% each), followed by robots (3/20, 15%). Activity recognition was the most common goal of the evaluated smart home technologies, with 55% (11/20) of the studies reporting it, followed by activity monitoring (7/20, 35%). Most studies (8/20, 40%) were conducted in a laboratory setting. Moreover, 25% (5/20) and 10% (2/20) were conducted in a home and hospital setting, respectively. Finally, 75% (15/20) had a positive outcome, 15% (3/20) had a mixed outcome, and 10% (2/20) had an indeterminate outcome. Our results suggest that smart home technologies, especially digital personal assistants, coaches, and robots, are effective in promoting physical activity among the young population. Although only few studies were identified among the older population, smart home technologies hold bright prospects in assisting and aiding older people to age in place and function independently, especially in Western countries, where there are shortages of long-term care workers. Hence, there is a need to do more work (eg, cross-cultural studies and randomized controlled trials) among the growing aging population on the effectiveness and acceptance of smart home technologies that aim to promote physical activity.

From Lab to Real World: Assessing the Effectiveness of Human Activity Recognition and Optimization through Personalization.

Human activity recognition (HAR) algorithms today are designed and evaluated on data collected in controlled settings, providing limited insights into their performance in real-world situations with noisy and missing sensor data and natural human activities. We present a real-world HAR open dataset compiled from a wristband equipped with a triaxial accelerometer. During data collection, participants had autonomy in their daily life activities, and the process remained unobserved and uncontrolled. A general convolutional neural network model was trained on this dataset, achieving a mean balanced accuracy (MBA) of 80%. Personalizing the general model through transfer learning can yield comparable and even superior results using fewer data, with the MBA improving to 85%. To emphasize the issue of insufficient real-world training data, we conducted training of the model using the public MHEALTH dataset, resulting in 100% MBA. However, upon evaluating the MHEALTH-trained model on our real-world dataset, the MBA drops to 62%. After personalizing the model with real-world data, an improvement of 17% in the MBA is achieved. This paper showcases the potential of transfer learning to make HAR models trained in different contexts (lab vs. real-world) and on different participants perform well for new individuals with limited real-world labeled data available.

Motion Stimulation for Compositional Action Recognition

Recognizing the unseen combinations of action and different objects, namely (zero-shot) compositional action recognition, is extremely challenging for conventional action recognition algorithms in real-world applications. Previous methods focus on enhancing the dynamic clues of objects that appear in the scene by building region features or tracklet embedding from ground-truths or detected bounding boxes. These methods rely heavily on manual annotation or the quality of detectors, which are inflexible for practical applications. In this work, we aim to mining the temporal clues from moving objects or hands without explicit supervision. Thus, we propose a novel Motion Stimulation (MS) block, which is specifically designed to mine dynamic clues of the local regions autonomously from adjacent frames. Furthermore, MS consists of the following three steps: motion feature extraction, motion feature recalibration, and action-centric excitation. The proposed MS block can be directly and conveniently integrated into existing video backbones to enhance the ability of compositional generalization for action recognition algorithms. Extensive experimental results on three action recognition datasets, the Something-Else, IKEA-Assembly and EPIC-KITCHENS datasets, indicate the effectiveness and interpretability of our MS block.

Deep Learning-Based Action Detection in Untrimmed Videos: A Survey.

Understanding human behavior and activity facilitates advancement of numerous real-world applications, and is critical for video analysis. Despite the progress of action recognition algorithms in trimmed videos, the majority of real-world videos are lengthy and untrimmed with sparse segments of interest. The task of temporal activity detection in untrimmed videos aims to localize the temporal boundary of actions and classify the action categories. Temporal activity detection task has been investigated in full and limited supervision settings depending on the availability of action annotations. This article provides an extensive overview of deep learning-based algorithms to tackle temporal action detection in untrimmed videos with different supervision levels including fully-supervised, weakly-supervised, unsupervised, self-supervised, and semi-supervised. In addition, this article reviews advances in spatio-temporal action detection where actions are localized in both temporal and spatial dimensions. Action detection in online setting is also reviewed where the goal is to detect actions in each frame without considering any future context in a live video stream. Moreover, the commonly used action detection benchmark datasets and evaluation metrics are described, and the performance of the state-of-the-art methods are compared. Finally, real-world applications of temporal action detection in untrimmed videos and a set of future directions are discussed.

Reconstruction of physical dance teaching content and movement recognition based on a machine learning model

With the technological development of movement recognition based on machine learning model algorithms, the content and movements for physical dance teaching are also seeking changes and innovations. In this paper, a set of three-dimensional convolutional neural network recognition algorithms based on a machine learning model is constructed through the collection to recognition of sports dance movement data. By collecting the skeleton information of typical movements of physical dance, a typical movement dataset of physical dance is constructed, which is recognized by the improved 3D convolutional neural network recognition algorithm under the machine learning model, and the method is validated on the public dataset. The experimental results show that the 3D CNNs in this paper can produce relatively satisfactory results for sports dance action recognition with high accuracy of action recognition, which verifies the feasibility of the 3D convolutional neural network action recognition algorithm under the machine learning model for the acquisition to recognition of sports dance actions. It illustrates that the future can be better to open a new direction of physical dance education content through machine learning models in this form.

Global spatio-temporal synergistic topology learning for skeleton-based action recognition

Compared to RGB video-based action recognition, skeleton-based action recognition algorithm has attracted much more attention due to being more lightweight, better generalization and robustness. The extraction of temporal and spatial features is a crucial factor for skeleton-based action recognition. However, existing feature extraction methods suffer from two limitations: (1) the isolated extraction of temporal and spatial feature cannot capture temporal feature connections among non-adjacent joints and (2) convolution-limited perceptual fields cannot capture global temporal features of joints effectively. In this work, we propose a global spatio-temporal synergistic feature learning module (GSTL), which generates global spatio-temporal synergistic topology of joints by spatio-temporal feature fusion. By further combining the GSTL with a temporal modeling unit, we develop a powerful global spatio-temporal synergistic topology learning network (GSTLN), and it achieves competitive performance with fewer parameters on three challenge datasets: NTU RGB + D, NTU RGB + D 120, and NW-UCLA.

Examination of physical activity development in early childhood: protocol for a longitudinal cohort study of mother-toddler dyads

BackgroundPhysical activity (PA) development in toddlers (age 1 and 2 years) is not well understood, partly because of a lack of analytic tools for accelerometer-based data processing that can accurately evaluate PA among toddlers. This has led to a knowledge gap regarding how parenting practices around PA, mothers’ PA level, mothers’ parenting stress, and child developmental and behavioral problems influence PA development in early childhood.MethodsThe Child and Mother Physical Activity Study is a longitudinal study to observe PA development in toddlerhood and examine the influence of personal and parental characteristics on PA development. The study is designed to refine and validate an accelerometer-based machine learning algorithm for toddler activity recognition (Aim 1), apply the algorithm to compare the trajectories of toddler PA levels in males and females age 1–3 years (Aim 2), and explore the association between gross motor development and PA development in toddlerhood, as well as how parenting practices around PA, mothers’ PA, mothers’ parenting stress, and child developmental and behavioral problems are associated with toddlerhood PA development (Exploratory Aims 3a-c).DiscussionThis study will be one of the first to use longitudinal data to validate a machine learning activity recognition algorithm and apply the algorithm to quantify free-living ambulatory movement in toddlers. The study findings will help fill a significant methodological gap in toddler PA measurement and expand the body of knowledge on the factors influencing early childhood PA development.

DeepHAR: a deep feed-forward neural network algorithm for smart insole-based human activity recognition

Health monitoring, rehabilitation, and fitness are just a few domains where human activity recognition can be applied. In this study, a deep learning approach has been proposed to recognise ambulation and fitness activities from data collected by five participants using smart insoles. Smart insoles, consisting of pressure and inertial sensors, allowed for seamless data collection while minimising user discomfort, laying the baseline for the development of a monitoring and/or rehabilitation system for everyday life. The key objective has been to enhance the deep learning model performance through several techniques, including data segmentation with overlapping technique (2 s with 50% overlap), signal down-sampling by averaging contiguous samples, and a cost-sensitive re-weighting strategy for the loss function for handling the imbalanced dataset. The proposed solution achieved an Accuracy and F1-Score of 98.56% and 98.57%, respectively. The Sitting activities obtained the highest degree of recognition, closely followed by the Spinning Bike class, but fitness activities were recognised at a higher rate than ambulation activities. A comparative analysis was carried out both to determine the impact that pre-processing had on the proposed core architecture and to compare the proposed solution with existing state-of-the-art solutions. The results, in addition to demonstrating how deep learning solutions outperformed those of shallow machine learning, showed that in our solution the use of data pre-processing increased performance by about 2%, optimising the handling of the imbalanced dataset and allowing a relatively simple network to outperform more complex networks, reducing the computational impact required for such applications.

Activity recognition using a combination of high gain observer and deep learning computer vision algorithms

Inertial sensors have become increasingly popular in human activity classification due to their ease of use and affordability. This paper proposes a novel algorithm for human activity recognition that is a combination of a high-gain observer and deep learning computer vision classification algorithms. The nonlinear high-gain observer designed using Lyapunov analysis accurately estimates the attitude of the chest of a human subject using measurements from a single Inertial Measurement Unit (IMU). The signals processed by the observer are then converted into spectrograms to obtain “images” of the frequency response of the signals. The images for activities from a dataset of 7 human subjects are annotated and used for training/ fine-tuning of several well-known deep learning algorithms for image processing. The results from the best combination of our algorithms shows an exceptional accuracy of 98% for activity recognition. Using deep learning computer vision algorithms, this paper shows how to perform transfer learning from networks pre-trained on millions of images, thus showing how we can train a powerful deep learning network for activity recognition even with just small datasets. The algorithm that uses the high gain observer is shown to perform significantly better than an algorithm based on raw accelerometer and gyro signals.

Real world validation of activity recognition algorithm and development of novel behavioral biomarkers of falls in aged control and movement disorder patients.

The use of wearable sensors in movement disorder patients such as Parkinson's disease (PD) and normal pressure hydrocephalus (NPH) is becoming more widespread, but most studies are limited to characterizing general aspects of mobility using smartphones. There is a need to accurately identify specific activities at home in order to properly evaluate gait and balance at home, where most falls occur. We developed an activity recognition algorithm to classify multiple daily living activities including high fall risk activities such as sit to stand transfers, turns and near-falls using data from 5 inertial sensors placed on the chest, upper-legs and lower-legs of the subjects. The algorithm is then verified with ground truth by collecting video footage of our patients wearing the sensors at home. Our activity recognition algorithm showed >95% sensitivity in detection of activities. Extracted features from our home monitoring system showed significantly better correlation (~69%) with prospectively measured fall frequency of our subjects compared to the standard clinical tests (~30%) or other quantitative gait metrics used in past studies when attempting to predict future falls over 1 year of prospective follow-up. Although detecting near-falls at home is difficult, our proposed model suggests that near-fall frequency is the most predictive criterion in fall detection through correlation analysis and fitting regression models.