Activity Recognition System Research Articles

Position-Agnostic Smartphone Placement Detection for Improved Reliability in Human Activity Recognition

This research aims to solve the problem of position-independent activity recognition, a critical aspect in accurately identifying human activities using smartphones. Our study addresses this challenge by employing Convolutional Neural Networks to classify activities such as walking, sitting, running, and more, regardless of the smartphone’s position on the body. Leveraging a real-world publicly available dataset, we demonstrate 98% accuracy obtained solely from accelerometer data, surpassing state-of-the-art techniques by 5.77%. This advancement holds promise for enhancing smartphone-based human activity recognition, particularly in security-related applications like adaptive user authentication. Overall, our research demonstrates progress toward improving the reliability and adaptability of activity recognition systems across diverse contexts.

Quantifying the Relationship Between At-Home Shoulder Physiotherapy Participation and Outcome: What can a Watch Tell Us?

Exercise-based physiotherapy is an established treatment of rotator cuff injury. Objective assessment of at-home exercise is critical to understand its relationship with clinical outcomes. This study uses the Smart Physiotherapy Activity Recognition System to measure at-home physiotherapy participation in patients with rotator cuff injury based on inertial sensor data captured from smart watches. Relationships between participation and clinical outcomes, long-term durability of outcome improvements, and factors predictive of participation were evaluated. Patients participated in a 12-week rotator cuff physiotherapy program in a prospective single-center study. Patients wore smart watches during supervised weekly in-clinic physiotherapy sessions and while performing exercises at home. Demographic information and rotator-cuff diagnosis were collected at baseline and assessed as predictors of physiotherapy participation. Outcome measures (pain, disability [Disabilities of the Arm, Shoulder and Hand], strength, range of motion) were collected over duration of treatment and at 12-month follow-up (pain and disability). Machine learning algorithms identified and classified periods of exercise to evaluate participation and adherence. One hundred ten patients enrolled and initiated treatment, with 92 patients included in the analysis. All outcomes showed significant improvements from baseline at each time point. Mean total weekly at-home participation decreased from 35.6 ± 28.9 minutes in weeks 0 to 4 to 28.9 ± 25.7 minutes in weeks 8 to 12 (t = 2.23, P = 0.023). For the full cohort, significant relationships were found between physiotherapy participation and disability, manual strength, external rotation, internal rotation, and abduction. Significant predictors of participation included greater age, being unmarried, diagnosed rotator cuff tear, and measures of self-efficacy, social support, and comorbidity. Higher participation rates led to significant improvements in outcomes for partial thickness/no-tear patients but not for full-thickness tears. Machine learning methods applied to data collected from smart watches enabled objective assessment of physiotherapy participation in the home setting. Although most patients improved with physiotherapy, patients with full-thickness rotator cuff tears were not similarly responsive to higher exercise volumes.

Towards a low-cost and privacy-preserving indoor activity recognition system using wifi channel state information

Towards a low-cost and privacy-preserving indoor activity recognition system using wifi channel state information

Feasibility of YOLOX computer model-based assessment of knee function compared with manual assessment for people with severe knee osteoarthritis

ObjectiveThis study aimed to assess the feasibility of computer model-based evaluation of knee joint functional capacity in comparison with manual assessment.MethodsThis study consisted of two phases: (1) developing an automatic knee joint action recognition and classification system on the basis of improved YOLOX and (2) analyzing the feasibility of assessment by the software system and doctors, identifying the knee joint function of patients, and determining the accuracy of the software system. We collected 40–50 samples for use in clinical experiments. The datasets used in this study were collected from patients admitted to the Joint Surgery Center. In this study, the knee joint assessment items included stair climbing, walking on uneven surfaces, and knee joint function. To assess the computer model’s automatic evaluation of knee joint function, MedCalc 20 statistical software was used to analyze the consistency of the Lequesne functional index between the computer model’s automated determinations and manual independent assessments.ResultsThe weighted kappa coefficients between the doctors’ assessments and the software system’s assessments were 0.76 (95% confidence intervals:0.59 ~ 0.92) for climbing up and down stairs, 0.64 (95% confidence intervals:0.45 ~ 0.82) for walking on uneven floors, and 0.68 (95% confidence intervals:0.53 ~ 0.84) for the Lequesne functional index, indicating good consistency between the assessments of the software system and doctors.ConclusionThis paper introduces an automatic knee joint action recognition and classification method based on improved YOLOX. By comparing the results obtained by orthopedic doctors and the software system, the feasibility of this software system was validated in the clinic.

A graph neural network model application in point cloud structure for prolonged sitting detection system based on smartphone sensor data

AbstractThe prolonged sitting inherent in modern work and study environments poses significant health risks, necessitating effective monitoring solutions. Traditional human activity recognition systems often fall short in these contexts owing to their reliance on structured data, which may fail to capture the complexity of human movements or accommodate the often incomplete or unstructured nature of healthcare data. To address this gap, our study introduces a novel application of graph neural networks (GNNs) for detecting prolonged sitting periods using point cloud data from smartphone sensors. Unlike conventional methods, our GNN model excels at processing the unordered, three‐dimensional structure of sensor data, enabling more accurate classification of sedentary activities. The effectiveness of our approach is demonstrated by its superior ability to identify sitting, standing, and walking activities—critical for assessing health risks associated with prolonged sitting. By providing real‐time activity recognition, our model offers a promising tool for healthcare professionals to mitigate the adverse effects of sedentary behavior.

Action Recognition in Basketball with Inertial Measurement Unit-Supported Vest.

In this study, an action recognition system was developed to identify fundamental basketball movements using a single Inertial Measurement Unit (IMU) sensor embedded in a wearable vest. This study aims to enhance basketball training by providing a high-performance, low-cost solution that minimizes discomfort for athletes. Data were collected from 21 collegiate basketball players, and movements such as dribbling, passing, shooting, layup, and standing still were recorded. The collected IMU data underwent preprocessing and feature extraction, followed by the application of machine learning algorithms including KNN, decision tree, Random Forest, AdaBoost, and XGBoost. Among these, the XGBoost algorithm with a window size of 250 and a 75% overlap yielded the highest accuracy of 96.6%. The system demonstrated superior performance compared to other single-sensor systems, achieving an overall classification accuracy of 96.9%. This research contributes to the field by presenting a new dataset of basketball movements, comparing the effectiveness of various feature extraction and machine learning methods, and offering a scalable, efficient, and accurate action recognition system for basketball.

An Approach to Fall Detection Using Statistical Distributions of Thermal Signatures Obtained by a Stand-Alone Low-Resolution IR Array Sensor Device.

Infrared array sensor-based fall detection and activity recognition systems have gained momentum as promising solutions for enhancing healthcare monitoring and safety in various environments. Unlike camera-based systems, which can be privacy-intrusive, IR array sensors offer a non-invasive, reliable approach for fall detection and activity recognition while preserving privacy. This work proposes a novel method to distinguish between normal motion and fall incidents by analyzing thermal patterns captured by infrared array sensors. Data were collected from two subjects who performed a range of activities of daily living, including sitting, standing, walking, and falling. Data for each state were collected over multiple trials and extended periods to ensure robustness and variability in the measurements. The collected thermal data were compared with multiple statistical distributions using Earth Mover's Distance. Experimental results showed that normal activities exhibited low EMD values with Beta and Normal distributions, suggesting that these distributions closely matched the thermal patterns associated with regular movements. Conversely, fall events exhibited high EMD values, indicating greater variability in thermal signatures. The system was implemented using a Raspberry Pi-based stand-alone device that provides a cost-effective solution without the need for additional computational devices. This study demonstrates the effectiveness of using IR array sensors for non-invasive, real-time fall detection and activity recognition, which offer significant potential for improving healthcare monitoring and ensuring the safety of fall-prone individuals.

Mitigating Data Leakage in a WiFi CSI Benchmark for Human Action Recognition.

Human action recognition using WiFi channel state information (CSI) has gained attention due to its non-intrusive nature and potential applications in healthcare, smart environments, and security. However, the reliability of methods developed for CSI-based action recognition is often contingent on the quality of the datasets and evaluation protocols used. In this paper, we uncovered a critical data leakage issue, which arises from improper data partitioning, in a widely used WiFi CSI benchmark dataset. Specifically, the benchmark fails to separate individuals between the training and test sets, leading to inflated performance metrics as models inadvertently learn individual-specific features rather than generalizable action patterns. We analyzed this issue in depth, retrained several benchmarked models using corrected data partitioning methods, and demonstrated a significant drop in accuracy when individuals were properly separated across training and testing. Our findings highlight the importance of rigorous data partitioning in CSI-based action recognition and provide recommendations for mitigating data leakage in future research. This work contributes to the development of more robust and reliable human action recognition systems using WiFi CSI.

Human activity recognition utilizing optimized attention induced Multihead Convolutional Neural Network with Mobile Net V1 from Mobile health data.

Human Activity Recognition (HAR) systems are designed to continuously monitor human behaviour, mainly in the areas of entertainment and surveillance in intelligent home environments. In this manuscript, Human Activity Recognition utilizing optimized Attention Induced Multi head Convolutional Neural Network with Mobile Net V1 from Mobile Health Data (HAR-AMCNN-MNV1) is proposed. The input data is collected through MHEALTH and UCI HAR datasets. Neural Spectrospatial Filtering (NSF) is used for avoiding accurate labelling and reduces errors. Afterwards, Variational Density Peak Clustering Algorithm (VDPCA) is used for segmenting the data. Feature Extraction and Classification is done by Attention Induced Multi head Convolutional Neural Network with Mobile Net V1 (AMCNN-MNV1). AMCNN is used for extracting Hand-crafted features. AMCNN-MNV1 effectively classifies the human activities as Sitting and relaxing (Sit), Climbing stairs (CS), Walking (Walk), Standing still (Std), Waist bends forward (WBF), Frontal elevation of arms (FEA), Jogging (Jog), Knees bending (crouching) (KB), Cycling (Cycl), Lying down (Lay), Jump front & back (JFB) and Running (Run). Siberian Tiger Optimization Algorithm (STOA) is proposed to optimize the weight parameter of AMCNN-MNV1 classifier. The proposed method attains 21.19%, 23.45%, and 21.76% higher accuracy, 31.15%, 24.65% and 22.72% higher precision; 21.15%, 20.18%, and 21.28% higher recall evaluated to the existing methods.

Patient activity recognition for robust performance under varied illumination and cluttered background in indoor environment

Human Activity Recognition (HAR) holds significant importance in health and human-machine interaction. However, recognizing actions from 2D information faces challenges like occlusion, illumination variation, cluttered backgrounds, and view invariance. These hurdles are particularly pronounced in indoor patient monitoring settings due to fluctuating lighting conditions and cluttered backgrounds, which compromise the accuracy of activity recognition systems. A new architecture named illuminationRevive has been proposed to tackle this issue. Which utilizes an encoder-decoder convolutional neural network and image post-processing blocks to enhance the image's visual appearance. A new dataset comprising seven indoor physical activities has been proposed, created with contributions from thirty individuals aged 20 to 45. A hybrid fusion architecture is proposed to classify activities, integrating motion sequence information and body joint features. The proposed classification model incorporates generated Skeleton Motion History Images (SMHIs), collected human joint motion features from video frames, and novel kinematic and geometric features within window frames as inputs. The model can extract spatial and temporal feature vectors by integrating ResNet50-ViT (Residual Network-50 layers, Vision Transformer) and CNN-BiLSTM (Convolutional Neural Network-Bidirectional Long Short-Term Memory) layers. The suggested classification model was evaluated alongside state-of-the-art models using the LNMIIT-SMD (The LNM Institute of Information Technology-Skelton Motion Dataset) and established NTU-RGBD (Nanyang Technological University's Red Blue Green and Depth information) dataset. The evaluation aimed to assess the effectiveness of the proposed classification model architecture. Results demonstrate the superiority of the proposed model, achieving impressive accuracies of 98.21% on real-time data, 98.45% on the proposed dataset, and 97.12% on the NTU-RGBD dataset.

A Biolayer Interferometry-Based SARS-COV-2 Mpro-Targeted Active Ingredients Recognition System: Construction and Application in Ligand Screening From Herbal Medicines.

Drug discovery research targeting SARS-CoV-2 and other emerging pathogens remains critically important. Active compounds derived from plants frequently serve as lead compounds for further drug discovery; however, numerous unrelated chemical constituents in crude extracts may obscure the effective ingredients in LC-MS analysis. The aim of this study is to construct a biolayer interferometry (BLI)-based system for recognizing active ingredients that inhibit the main protease (Mpro) of SARS-CoV-2 and to identify the active chemical components binding to Mpro from herbal medicines. We developed a novel FRET fluorogenic probe by linking the amino acid sequences of the fluorescent proteins Lssmorange and mKate2 (Ls-mK). The interaction between traditional Chinese medicine and Mpro was analyzed using BLI. Ultrahigh performance liquid chromatography coupled with quadrupole time-of-flight mass spectrometry (UPLC-QTOF-MS) was employed to analyze the composition of herbal medicines. Fluorescence detection and spectroscopy confirmed the successful construction of an Mpro inhibitor screening system. Lanqin Oral Liquid (LQL) and Gardeniae fructus exhibited strong inhibitory effects on Mpro. Ten compounds were identified from G. fructus extracts; among them, deacetyl asperulosidic acid methyl ester (DAAME) and Gardoside were found to strongly bind to Mpro, with dissociation constants (KD) of 3.41 μM and 801 nM, respectively. The half-maximal inhibitory concentrations (IC50) of DAAME and Gardoside for Mpro were 27.46 and 13.7 μM, respectively. This study established a functional Mpro inhibitor screening system. Among the 10 components identified from G. fructus that bind to Mpro, DAAME and Gardoside displayed strong binding and inhibitory activity, indicating their potential as lead compounds for inhibiting SARS-CoV-2 viral replication.

Enhancing human activity recognition for the elderly and individuals with disabilities through optimized Internet-of-Things and artificial intelligence integration with advanced neural networks.

Elderly and individuals with disabilities can greatly benefit from human activity recognition (HAR) systems, which have recently advanced significantly due to the integration of the Internet of Things (IoT) and artificial intelligence (AI). The blending of IoT and AI methodologies into HAR systems has the potential to enable these populations to lead more autonomous and comfortable lives. HAR systems are equipped with various sensors, including motion capture sensors, microcontrollers, and transceivers, which supply data to assorted AI and machine learning (ML) algorithms for subsequent analyses. Despite the substantial advantages of this integration, current frameworks encounter significant challenges related to computational overhead, which arises from the complexity of AI and ML algorithms. This article introduces a novel ensemble of gated recurrent networks (GRN) and deep extreme feedforward neural networks (DEFNN), with hyperparameters optimized through the artificial water drop optimization (AWDO) algorithm. This framework leverages GRN for effective feature extraction, subsequently utilized by DEFNN for accurately classifying HAR data. Additionally, AWDO is employed within DEFNN to adjust hyperparameters, thereby mitigating computational overhead and enhancing detection efficiency. Extensive experiments were conducted to verify the proposed methodology using real-time datasets gathered from IoT testbeds, which employ NodeMCU units interfaced with Wi-Fi transceivers. The framework's efficiency was assessed using several metrics: accuracy at 99.5%, precision at 98%, recall at 97%, specificity at 98%, and F1-score of 98.2%. These results then were benchmarked against other contemporary deep learning (DL)-based HAR systems. The experimental outcomes indicate that our model achieves near-perfect accuracy, surpassing alternative learning-based HAR systems. Moreover, our model demonstrates reduced computational demands compared to preceding algorithms, suggesting that the proposed framework may offer superior efficacy and compatibility for deployment in HAR systems designed for elderly or individuals with disabilities.

State-of-the-art in human activity recognition based on inertial measurement unit sensors: survey and applications

Human activity recognition systems using wearable sensors is an important issue in pervasive computing, which applies to various domains related to healthcare, context aware and pervasive computing, sports, surveillance and monitoring, and the military. Three approaches can be considered for activity recognition: video sensor-based, physical sensor-based, and environmental sensor-based. This paper investigates the related work regarding the physical sensor-based approaches to motion processing. In this paper, a wide range of artificial intelligence models, from single classifications to methods based on deep learning, have been reviewed. The human activity detection accuracy of different methods, under natural and experimental conditions poses several challenges. These challenges cause problems regarding the accuracy and applicability of the proposed methods. This paper analyzes the methods, challenges, approaches, and future work. The goal of this paper is to establish a clear distinction in the field of motion detection using inertial sensors.

Human Activity Recognition System Based on Continuous Learning with Human Skeleton Information

Human Activity Recognition System Based on Continuous Learning with Human Skeleton Information

Kinematic matrix: One-shot human action recognition using kinematic data structure

One-shot action recognition, which refers to recognizing human-performed actions using only a single training example, holds significant promise in advancing video analysis, particularly in domains requiring rapid adaptation to new actions. However, existing algorithms for one-shot action recognition face multiple challenges, including high computational complexity, limited accuracy, and difficulties in generalization to unseen actions. To address these issues, we propose a novel kinematic-based skeleton representation that effectively reduces computational demands while enhancing recognition performance. This representation leverages skeleton locations, velocities, and accelerations to formulate the one-shot action recognition task as a metric learning problem, where a model projects kinematic data into an embedding space. In this space, actions are distinguished based on Euclidean distances, facilitating efficient nearest-neighbour searches among activity reference samples. Our approach not only reduces computational complexity but also achieves higher accuracy and better generalization compared to existing methods. Specifically, our model achieved a validation accuracy of 78.5%, outperforming state-of-the-art methods by 8.66% under comparable training conditions. These findings underscore the potential of our method for practical applications in real-time action recognition systems.

A Passenger Detection and Action Recognition System for Public Transport Vehicles

In the past few years, advances of autonomous driving technologies have made significant progress. One successful deployment with great potential is for the self-driving public transportation. Consequently, the safety of passengers under unmanned operations is an important issue for the future development. In this paper, we propose a vision system for the detection, counting and action recognition of passengers on board a bus. To have a complete coverage of the passenger area, a pair of cameras mounted on the ceiling is adopted for image acquisition. A new cross-camera CNN structure is proposed for passenger pose recognition and action classification. To deal with the double counting problem caused by the overlap of multiple viewpoints, a region prediction algorithm is developed for matching validation. The experiments carried out on a minibus have demonstrated the feasibility of our proposed techniques.

A Comparative Study of Sensor Quantity and Data Granularity in AAL Systems for Activity Recognition

Abstract As populations age, the demand for non-intrusive elderly care solutions increases, highlighting the need for efficient AAL systems. Current research predominantly focuses on wearable sensors at high data frequencies. There is a critical gap in research in understanding the effectiveness of model performance using ambient sensors operating at varied granularities and with varied numbers of sensors. This study addresses this gap by investigating how sensor quantity and data frequency affect an artificial intelligence model’s ability to detect various activities of daily living. The methodology employs a quantitative, experimental design to systematically assess the performance of artificial intelligence models across sensor quantity and data frequencies. This assessment will be conducted through a multi-study approach involving different populations to ensure robust and generalizable findings. Each model’s efficacy will be evaluated using 5-fold cross-validation and GridSearchCV for rigorous hyperparameter tuning, employing diverse data aggregation and imputations techniques to maintain comprehensive analysis integrity. The primary goal of this research is to determine the optimal data granularity and number of sensors that maximize AI models’ ability to detect daily living activities while minimizing resource demands, thereby enhancing the sustainability and scalability of AAL systems. This work aims to advance the field of ambient sensing in elderly care, offering significant implications for designing and implementing future AAL technologies and potentially improving the quality of life for the elderly population. Key messages • The study explores the optimal configuration of data granularity and sensor quantity to maximize AI model efficiency in detecting activities of daily living among older adults. • The study investigates the impact of data frequency and sensor count on AI model performance in detecting daily activities, aiming to optimize Ambient Assisted Living systems for the elderly.

Retraction Note: Rule-based multi-view human activity recognition system in real time using skeleton data from RGB-D sensor

Retraction Note: Rule-based multi-view human activity recognition system in real time using skeleton data from RGB-D sensor

Ensem-DeepHAR: Identification of human activity in smart environments using ensemble of deep learning methods and motion sensor data

Recognizing human activity plays a crucial role in many applications such as medical care services in smart healthcare environments. Inertial or motion sensors can measure physiognomies such as acceleration and angular velocity of body movement while performing the activities and we can use them to learn the models capable of activity recognition. Over the past decades, many state-of-the-art activity recognition systems have been developed but there is still room to improve. In this paper, we have proposed a novel approach to identify human activity from motion sensor data by employing an enormous analysis of sensor data. Based on data analysis, we yielded quality data by preprocessing using a preprocessing chain for human activity recognition (PC-HAR) which also includes the Synthetic Minority Over-sampling Technique to balance the data of the dataset. As a recognition model, we proposed an ensemble of three different deep learning algorithms, namely, modified DeepConvLSTM, modified InceptionTime, and modified ResNet which is named ‘Ensem-DeepHAR’. The outcome of the proposed model is carried out by stacking predictions from each of the mentioned models and then a Random Forest as a meta-model uses those predictions to recognize the final activity. We evaluated our method on both person-dependent and person-independent cases and achieved 99.31 %, 99.08 %, and 97.52 % accuracies for the former case and 97.95 %, 98.11 %, and 99.51 % accuracies for the latter case using three common benchmark datasets: WISDM_ar_v1.1, PAMAP2, and UCI-HAR respectively. The various performance metrics and measures of experimental results establish the supremacy of the proposed model over the state-of-the-arts.

Enhancing Fall Detection Accuracy: The Ground-Face Coordinate System for 3D Accelerometer Data

The global elderly population is on the rise, leading to increased physical, sensory, and cognitive changes that heighten the risk of falls. Consequently, fall detection (FD) has emerged as a significant concern, attracting considerable attention in recent years. Utilizing 3D accelerometer sensors for FD offers advantages such as cost-effectiveness and ease of implementation; however, traditional raw 3D accelerometer signals are inherently dependent on the device's orientation and placement within the device coordinate system. Misalignment between the device's axes and the direction of movement can lead to misinterpretation of acceleration signals, potentially causing misclassification of activities and resulting in false positives or missed falls. This study introduces a novel coordinate system called "ground-face," which is designed to be independent of the device's orientation and placement. In this system, the vertical axis is aligned perpendicularly to the Earth, while the device's x-axis is aligned with the individual's direction of movement. To assess the potential of the vertical component of ground-face referenced accelerometer signals for FD, it was compared with the commonly used acceleration magnitude signal. Detailed analysis was conducted using frequently preferred features in FD studies, and fall detection was performed with various classifiers. Comprehensive experiments demonstrated that the vertical component of the ground-face signal effectively characterizes falls, yielding approximately a 2% improvement in detection accuracy. Moreover, the proposed coordinate system is not limited to FD but can also be applied to human activity recognition (HAR) systems. By mitigating orientation-related discrepancies, it reduces the likelihood of misclassification and enhances the overall HAR capabilities.