Activity Recognition Research Articles

An IoT solution that detects human physical activities to support health assessment based on sensors and supervised learning algorithms

At the moment, human physical activities can be detected by activity recognition systems to help with health status evaluation. One of the key issues in human activity recognition (HAR) systems is cost, speed, and accuracy. With the advancement of mobile devices and increasingly sophisticated microprocessors, along with the integration of various sensors including accelerometers, human activity recognition has become easier and faster. However, the accuracy of activity classification based on accelerometer data depends on many different factors, which can lead to potential deviations in results. Through various experiments, we have found that selecting optimal features is crucial for the performance of classification. In this paper, we propose a simple feature set that yields very promising results with an accuracy of up to 99%.

Cross-Domain Human Activity Recognition Using Low-Resolution Infrared Sensors.

This paper investigates the feasibility of cross-domain recognition for human activities captured using low-resolution 8 × 8 infrared sensors in indoor environments. To achieve this, a novel prototype recurrent convolutional network (PRCN) was evaluated using a few-shot learning strategy, classifying up to eleven activity classes in scenarios where one or two individuals engaged in daily tasks. The model was tested on two independent datasets, with real-world measurements. Initially, three different networks were compared as feature extractors within the prototype network. Following this, a cross-domain evaluation was conducted between the real datasets. The results demonstrated the model's effectiveness, showing that it performed well regardless of the diversity of samples in the training dataset.

Cross-view action recognition understanding from exocentric to egocentric perspective

Understanding action recognition in egocentric videos has emerged as a vital research topic with numerous practical applications. With the limitation in the scale of egocentric data collection, learning robust deep learning-based action recognition models remains difficult. Transferring knowledge learned from the large-scale exocentric data to the egocentric data is challenging due to the difference in videos across views. Our work introduces a novel cross-view learning approach to action recognition (CVAR) that effectively transfers knowledge from the exocentric to the selfish view. First, we present a novel geometric-based constraint into the self-attention mechanism in Transformer based on analyzing the camera positions between two views. Then, we propose a new cross-view self-attention loss learned on unpaired cross-view data to enforce the self-attention mechanism learning to transfer knowledge across views. Finally, to further improve the performance of our cross-view learning approach, we present the metrics to measure the correlations in videos and attention maps effectively. Experimental results on standard egocentric action recognition benchmarks, i.e., Charades-Ego, EPIC-Kitchens-55, and EPIC-Kitchens-100, have shown our approach’s effectiveness and state-of-the-art performance.

Hypergraph-Based Multi-View Action Recognition Using Event Cameras.

Action recognition from video data forms a cornerstone with wide-ranging applications. Single-view action recognition faces limitations due to its reliance on a single viewpoint. In contrast, multi-view approaches capture complementary information from various viewpoints for improved accuracy. Recently, event cameras have emerged as innovative bio-inspired sensors, leading to advancements in event-based action recognition. However, existing works predominantly focus on single-view scenarios, leaving a gap in multi-view event data exploitation, particularly in challenges like information deficit and semantic misalignment. To bridge this gap, we introduce HyperMV, a multi-view event-based action recognition framework. HyperMV converts discrete event data into frame-like representations and extracts view-related features using a shared convolutional network. By treating segments as vertices and constructing hyperedges using rule-based and KNN-based strategies, a multi-view hypergraph neural network that captures relationships across viewpoint and temporal features is established. The vertex attention hypergraph propagation is also introduced for enhanced feature fusion. To prompt research in this area, we present the largest multi-view event-based action dataset THUMV-EACT-50, comprising 50 actions from 6 viewpoints, which surpasses existing datasets by over tenfold. Experimental results show that HyperMV significantly outperforms baselines in both cross-subject and cross-view scenarios, and also exceeds the state-of-the-arts in frame-based multi-view action recognition.

Tennis teaching assistance model based on double chain shared unsupervised action recognition algorithm

The traditional self-supervised methods based on skeleton data frequently categorize the various enhancements of a given sample as positive examples, while the remaining samples are designated as negative examples. This approach results in a significant imbalance in the ratio of positive to negative samples, which in turn constrains the efficacy of samples with identical semantic information. Therefore, to further improve the training quality in tennis teaching and enhance the accuracy of action recognition, a dual chain sharing unsupervised action recognition algorithm has been proposed. This study first designs a skeleton topology data augmentation method based on the physical connections of human joints to obtain advanced semantic embeddings. Then, an improved positive sample expansion strategy is utilized to enhance the diversity and quality of training data. Next, an unsupervised learning mechanism is employed to autonomously learn and recognize complex patterns of tennis movements. To measure the performance of the model, tests were conducted on its accuracy, F1 score, recognition time, and fitting degree. The experimental results showed that the proposed algorithm could reach its optimal state after 23 iterations of training, and its F1 value reached 0.918, with an average accuracy of 92.9 % and an average recognition time of 7.4 s. The research algorithms were superior to the multi-input branch graph convolutional network action recognition algorithms used for comparison, pull-push contrastive loss action recognition algorithms, and multi-granularity anchor contrastive representation learning action recognition algorithms. Its accuracy was leading by 8.6 %-23.5 %, and the recognition time has been reduced by 2.3s-7.4 s. In addition, in terms of the fitting degree of action recognition, compared with other methods, the proposed method had a fitting degree of up to 95.7 %, which was at least 8.1 % higher. This research method can improve the timeliness of feedback from trainers by accurately and quickly recognizing movements, helping coaches develop more targeted training plans, thereby improving the efficiency and quality of tennis teaching.

Ensem-DeepHAR: Identification of human activity in smart environments using ensemble of deep learning methods and motion sensor data

Recognizing human activity plays a crucial role in many applications such as medical care services in smart healthcare environments. Inertial or motion sensors can measure physiognomies such as acceleration and angular velocity of body movement while performing the activities and we can use them to learn the models capable of activity recognition. Over the past decades, many state-of-the-art activity recognition systems have been developed but there is still room to improve. In this paper, we have proposed a novel approach to identify human activity from motion sensor data by employing an enormous analysis of sensor data. Based on data analysis, we yielded quality data by preprocessing using a preprocessing chain for human activity recognition (PC-HAR) which also includes the Synthetic Minority Over-sampling Technique to balance the data of the dataset. As a recognition model, we proposed an ensemble of three different deep learning algorithms, namely, modified DeepConvLSTM, modified InceptionTime, and modified ResNet which is named ‘Ensem-DeepHAR’. The outcome of the proposed model is carried out by stacking predictions from each of the mentioned models and then a Random Forest as a meta-model uses those predictions to recognize the final activity. We evaluated our method on both person-dependent and person-independent cases and achieved 99.31 %, 99.08 %, and 97.52 % accuracies for the former case and 97.95 %, 98.11 %, and 99.51 % accuracies for the latter case using three common benchmark datasets: WISDM_ar_v1.1, PAMAP2, and UCI-HAR respectively. The various performance metrics and measures of experimental results establish the supremacy of the proposed model over the state-of-the-arts.

Fine-Grained Recognition of Manipulation Activities on Objects via Multi-Modal Sensing

Fine-Grained Recognition of Manipulation Activities on Objects via Multi-Modal Sensing

Adaptive morphological modeling and optimal estimation for dynamic LVET measurement from seismocardiographic signals

The dynamic left ventricular ejection time (LVET) represents a crucial cardiovascular health indicator. Seismocardiography (SCG) offers substantial advantages over conventional techniques, however, it is susceptible to motion artifacts and exhibits complex morphologies, posing challenges for accurate dynamic LVET measurement. This study proposes a synergistic approach that integrates adaptive morphological modeling with an optimal estimation technique to accurately measure dynamic LVET from SCG signals. Firstly, an active feature recognition method was proposed to automatically identify 10 morphological features, followed by modeling them using 10 Gaussian functions. Subsequently, the model results were fused with measured SCG through optimal estimation, involving adaptive updating of the measurement noise matrix. When evaluated against state-of-the-art algorithms on the open-source CEBS dataset (82997 heartbeats) and the experimental dataset (34181 heartbeats), the proposed algorithm exhibits outstanding performance, with a goodness of fit (R2) of 0.9272 and root-mean-square error (RMSE) of 13.808 ms, convincingly demonstrating its effectiveness and advancement.

The Application of Deep Learning in Sports Training

In response to the problems of overfitting, susceptibility to interference information, and insufficient feature expression ability in existing deep learning methods for sports action recognition, the author proposes a deep learning sports action recognition method that integrates attention mechanism. This method proposes a video data augmentation algorithm in data preprocessing to reduce the risk of model overfitting. Then, during the video frame sampling process, the existing sampling algorithms are improved to effectively suppress the influence of interference information. In the special section, the network residual consolidation is proposed to improve the feature extraction capacity of the structure. The Long Term Time Transform (LSTM) network is used to solve the problem of the global correlation of the spatial correlation, and the classification algorithm is achieved by Softmax. and the classification algorithm is proposed. The experimental results show that the recognition rates of this method on UCFYouTube, KTH, and HMDB-51 data are 96.73%, 98.07%, and 64.82%, respectively.

Visual-guided hierarchical iterative fusion for multi-modal video action recognition

Vision-Language models(VLMs) have shown promising improvements on various visual tasks. Most existing VLMs employ two separate transformer-based encoders, each dedicated to modeling visual and language features independently. Because the visual features and language features are unaligned in the feature space, it is challenging for the multi-modal encoder to learn vision-language interactions. In this paper, we propose a V,isual-guided Hierarchical Iterative Fusion (VgHIF) method for VLMs in video action recognition, which acquires more discriminative vision and language representation. VgHIF leverages visual features from different levels in visual encoder to interact with language representation. The interaction is processed by the attention mechanism to calculate the correlation between visual features and language representation. VgHIF learns grounded video-text representation and supports many different pre-trained VLMs in a flexible and efficient manner with a tiny computational cost. We conducted experiments on the Kinetics-400 Mini Kinetics 200 HMDB51, and UCF101 using VLMs: CLIP, X-CLIP, and ViFi-CLIP. The experiments were conducted under full supervision and few shot settings, and compared with the baseline multi-modal model without VgHIF, the Top-1 accuracy of the proposed method has been improved to varying degrees, and several groups of results have achieved comparable results with state-of-the-art performance, which strongly verified the effectiveness of the proposed method.

GRASNet: A novel graph neural network for improving human action recognition and well-being assessment in smart manufacturing

In the manufacturing environment, prioritizing the well-being of workers is both essential and challenging. Recognizing and monitoring workers’ actions in real time is crucial in ensuring safety and health. This study introduces innovative approaches leveraging smart manufacturing technologies and Graph Neural Networks (GNNs) to enhance human action recognition using accessible skeletal data. Traditional Neural Networks encounter limitations in processing readily accessible human-related data, such as image and skeletal data, prompting the development of the Graph Residual Attention SAGE Network (GRASNet). GRASNet exhibits superior capabilities in capturing intricate local connectivity within graph data, ensuring a delicate balance of complexity and efficiency. Furthermore, experimental validations on two manufacturing-related datasets, derived from the NTU RGB + D 120 benchmark dataset, underscore GRASNet’s ability for high-accuracy monitoring of human abnormal behaviors and health issues, positioning it ahead of other existing deep learning models.

Sports Player Action Recognition based on Deep Learning

A sports auxiliary evaluation system suitable for China’s national conditions was established using big data and sports identification technology. First of all, this paper extends the data of normative behavior and constructs a normative library of scores and comparisons. The acquisition of 3D data is emphasized. The method based on Fourier descriptors is used to locate the motion accurately. In this way, better gait recognition results can be obtained. The Fourier characteristics before and after wavelet transform are compared with the actual object characteristics, and the results show that the proposed algorithm can extract the features with high precision. This scheme can obtain a more accurate identification effect. The system proposed in this paper provides a powerful means for judges to score.

Lightweight skeleton-based action recognition model based on global–local feature extraction and fusion

Lightweight skeleton-based action recognition model based on global–local feature extraction and fusion

Human activity recognition using binary sensors: A systematic review

Human activity recognition (HAR) is an emerging area of study and research field that explores the development of automated systems to identify and categorize human activities using data collected from various sensors. In the field of Human Activity Recognition (HAR), binary sensors offer a distinct approach by providing simpler on/off readings to indicate the presence of events such as door openings or light switch activations. Compared to other sensors used for HAR, binary sensors have several advantages, including lower cost, low power consumption, ease of installation, and privacy preservation. For instance, they can be effectively used in smart homes to detect when someone enters or leaves a room without user input. This study presents a systematic review of the state-of-the-art methods and techniques for HAR using binary sensors. We comprehensively consider five crucial aspects: data collection methods, preprocessing techniques, feature extraction and fusion strategies, classification algorithms, and evaluation metrics. Furthermore, we identify the gaps and limitations of the existing studies and provide directions for future research. This comprehensive and up-to-date review can serve as a valuable reference for researchers and practitioners in the field of HAR using binary sensors.

Hybrid Deep Learning Approaches for Human Activity Recognition and Postural Transitions Using Mobile Device Sensors

Hybrid Deep Learning Approaches for Human Activity Recognition and Postural Transitions Using Mobile Device Sensors

CT-Net: an interpretable CNN-Transformer fusion network for fNIRS classification.

Functional near-infrared spectroscopy (fNIRS), an optical neuroimaging technique, has been widely used in the field of brain activity recognition and brain-computer interface. Existing works have proposed deep learning-based algorithms for the fNIRS classification problem. In this paper, a novel approach based on convolutional neural network and Transformer, named CT-Net, is established to guide the deep modeling for the classification of mental arithmetic (MA) tasks. We explore the effect of data representations, and design a temporal-level combination of two raw chromophore signals to improve the data utilization and enrich the feature learning of the model. We evaluate our model on two open-access datasets and achieve the classification accuracy of 98.05% and 77.61%, respectively. Moreover, we explain our model by the gradient-weighted class activation mapping, which presents a high consistent between the contributing value of features learned by the model and the mapping of brain activity in the MA task. The results suggest the feasibility and interpretability of CT-Net for decoding MA tasks.

Hybrid random projection: Integrating dense and sparse techniques for enhanced representation in high-dimensional data

This paper introduces a novel hybrid random projection method (HRP) that effectively combines the strengths of normal random projection (NRP) and plus-minus one random projection (PMRP). By incorporating a blending parameter, HRP optimises the contributions of the NRP and PMRP, reducing the dimensions of the data while ensuring that the structure of the data is preserved. Conventional methods, such as NRP and PMRP, are recognised for their benefits; however, they also have limitations. NRP is generally accurate but can encounter difficulties with certain data types or noise. The PMRP is simple, but occasionally fails to capture complex relationships within the data. The performance of HRP is investigated through comprehensive evaluations, including simulations and real-world data analyses from key machine learning datasets, such as period change, toxicity, MNIST, and human activity recognition (HAR). We examine various factors such as sample size, dimensions of the original and reduced data, and sparsity. Distance distortion is the primary metric utilised to measure performance, and indicates how well the dataset structure is maintained after dimension reduction. In every test, HRP consistently demonstrates the least distance distortion, performing superiorly to the NRP and PMRP. This is true for a range of scenarios and datasets. HRP represents a significant advancement in dimension reduction techniques. Its ability to minimise information loss during the reduction process demonstrates its potential as a powerful tool for managing complex high-dimensional data in practical applications. This renders HRP an important development in the fields of machine learning, intelligent systems, and artificial intelligence, offering improved efficiency and precision in data analyses.

Graph-Structure-Based Multigranular Belief Fusion for Human Activity Recognition.

The belief functions (BFs) introduced by Shafer in the mid of 1970s are widely applied in information fusion to model epistemic uncertainty and to reason about uncertainty. Their success in applications is however limited because of their high-computational complexity in the fusion process, especially when the number of focal elements is large. To reduce the complexity of reasoning with BFs, we can envisage as a first method to reduce the number of focal elements involved in the fusion process to convert the original basic belief assignments (BBAs) into simpler ones, or as a second method to use a simple rule of combination with potentially a loss of the specificity and pertinence of the fusion result, or to apply both methods jointly. In this article, we focus on the first method and propose a new BBA granulation method inspired by the community clustering of nodes in graph networks. This article studies a novel efficient multigranular belief fusion (MGBF) method. Specifically, focal elements are regarded as nodes in the graph structure, and the distance between nodes will be used to discover the local community relationship of focal elements. Afterward, the nodes belonging to the decision-making community are specially selected, and then the derived multigranular sources of evidence can be efficiently combined. To evaluate the effectiveness of the proposed graph-based MGBF, we further apply this new approach to combine the outputs of convolutional neural networks + attention (CNN + Attention) in the human activity recognition (HAR) problem. The experimental results obtained with real datasets prove the potential interest and feasibility of our proposed strategy with respect to classical BF fusion methods.

An Accurate Classification for Human Sport Activity Recognition from Sensor Data Using Hybrid Machine Learning Models

The significance of human sports activities recognition (HSAR) in many computing applications, including sports analytics, smart surroundings, and healthcare monitoring is huge. The current use of vision sensors for HSAR is a difficult task due to the intricate movements involved in sports and fitness workouts, as well as fluctuations in illumination conditions. Therefore, a comprehensive review of machine learning algorithms (such as XGBoost, Random Forest (RF), Decision Trees (DT), and K-Nearest Neighbors (KNN)) and their usage for human sports activities classification based on sensor data called ML-HSAR is presented in this article. A dataset containing 19 different human activities gathered from 8 different people through different types of sensors was used for this work; the dataset contains activities like sitting and standing, as well as other activities that require serious movements, such as running, cycling, and playing basketball. This study primarily aims to determine the performance of the considered ML models in accurately classifying these activities found in the dataset. The training of the algorithms and the subsequent evaluation were done on the segmented sensor data, effectively leveraging the dataset’s temporal aspect. In addition to the presented results for each of these algorithms, the limitations of each algorithm and their strengths in handling the complexities involved in HSAR were also presented. The contribution of this analysis is towards the understanding of the most appropriate ML models for HSAR tasks; it also offers valuable insights for future research and practical applications.

A Lightweight Deep Human Activity Recognition Algorithm Using Multiknowledge Distillation

A Lightweight Deep Human Activity Recognition Algorithm Using Multiknowledge Distillation