Human Activity Recognition System Research Articles

Skeleton-Based Human Activity Recognition of Low-voltage Distribution Room Based on MQTT Protocol and Wireless Sensor Network

For the activity recognition of operators in low-voltage distribution rooms, a human activity recognition system based on the bidirectional gated recurrent unit model (BiGRU) is proposed to realize the skeleton information acquisition by the WiFi Channel State Information (CSI) sensor. The CSI sensor is used to collect the activity data and the Kinect v2 camera is used to capture the skeleton of the human body as annotations of the CSI data. The BiGRU model is employed to effectively extract the characteristics of the data. Aiming at the problem of poor remote communication, the MQTT connection can be established by using Message Queuing Telemetry Transport (MQTT) protocol combined with ESP8266 WiFi serial port communication module is proposed to improve the poor communication situation and carry out better data communication. The system achieves remote intelligent monitoring of the activities of the operator in the low-voltage distribution room.

Application of smartphone in recognition of human activities with machine learning

The aim of activity recognition is to determine the physical action being performed by one or more users based on a series of observations made during the user's actions in the relevant environment. Significant advancements in the field of human activity have resulted in the creation of novel ways for supporting elderly persons in doing their tasks independently. Using ambient computing, this type of service will be manageable. Many of services are provided by ambient technology, involving home automation tools, monitoring the behaviour of diseased individuals, and utility management. Numerous academics are focusing their efforts on computer software architectures, system infrastructure, and distributed applications utilising sensor devices. Aim of this project is to develop an algorithm that can perform human activity recognition (HAR) better than the existing state-of-the-art approach. Several tasks must be done to achieve this goal. To compete with an existing HAR system, this study will rely on secondary data from the cutting-edge experiment; no new data will be collected. The central experiment will be used to quantitatively identify the best classifier based on prediction accuracy. The current study entails monitoring and assessing existing literature in order to generate hypotheses that may be tested via experiment.

An Efficient System for Human activity recognition and monitoring for elderly people using Machine Learning

An Efficient System for Human activity recognition and monitoring for elderly people using Machine Learning

Human Activity Recognition from Wi-Fi CSI data using Principal Component-based Wavelet CNN

Human Activity Recognition (HAR) is an emerging technology with applications in the surveillance, security, and healthcare sectors. Noninvasive HAR systems based on Wi-Fi Channel State Information (CSI) signals can be developed leveraging the rapid growth of ubiquitous Wi-Fi technologies and the correlation between CSI dynamics and body motions. In this paper, we propose Principal Component-based Wavelet Convolutional Neural Network (or PCWCNN) – a novel approach for HAR that offers robustness and efficiency for practical real-time applications. Our proposed method incorporates two efficient preprocessing algorithms – the Principal Component Analysis (PCA) and the Discrete Wavelet Transform (DWT). We employ an adaptive activity segmentation algorithm that is accurate and computationally light. Additionally, we used Wavelet CNN – a deep convolutional network analogous to the well-studied ResNet and DenseNet networks, for activity classification. We evaluate the performance of our proposed PCWCNN algorithm on both line-of-sight data and non-line-of-sight data. We empirically show that our proposed PCWCNN algorithm performs very well on these real datasets, outperforming relevant state-of-the-art approaches.

A triboelectric gait sensor system for human activity recognition and user identification

Floor-based sensing systems to monitor human activities and identify users are essential for smart homes and intelligent buildings. A low-cost, easy-to-fabricate, and flexible gait sensor system based on triboelectric nanogenerator (TENG) is presented in this paper, which can transform gait movements, even in the low-frequency form, into electrical impulses without an external power source. To realize this, a TENG-based gait sensor unit with an optimized structure design is proposed to enhance the sensing sensitivity. A sensing insole module is formed by arranging the sensor units according to the foot pressure distribution. The sensor distribution is then explored and improved by comparative studies of gait recognition performance, which increases the recognition efficiency and the possible application in edge computing scenarios. Furthermore, a deep learning network is developed based on long short-term memory (LSTM) and residual units to extract deep features from multichannel time-series gait data to boost recognition performance. Experimental results demonstrate that the proposed gait sensor system can be utilized for human activity recognition and user identification with accuracies of 97.9 % and 99.4 %, respectively. Finally, a gait-sensing-based fitness exercise monitoring system is constructed that can estimate calorie expenditure and distinguish between standard and non-standard fitness activities with an accuracy of 97.2 %. This work can be extended to various application scenarios such as security surveillance, health monitoring, and intelligent control, which provides a new ubiquitous self-powered sensing solution for the Internet of Things (IoT).

IoT based Human Activity Recognition using Deep learning

Artificial intelligence and the Internet of things (IoT) are the fastest and latest growing technologies that can handle a huge amount of data in computing services. This paper presents a smart human activity recognition system based on IoT that can be used for surveillance purposes working as IoT-based armour. Pose estimation model viz. MoveNet has been employed to extract the anatomical key points from RGB video frames. Different subjects from different camera angles were employed to make the approach person-independent. Diverse Machine learning models such as Decision tree, support vector machines, XGboost, and random forest classifiers were employed using extracted keypoints for training the model for estimating human activity during posture estimation monitoring. SMS are sent to the designated person with the raising of buzzer alarm in case of anomalous behaviour detection.

Wearable Sensor-Based Human Activity Recognition System Employing Bi-LSTM Algorithm

AbstractHuman activity recognition (HAR) systems employing wearable sensors are a promising area of research for tracking human activity. Recently, wearable devices such as smartwatches and sensors have been developed for activity recognition and monitoring. These systems aim to obtain the subject’s state within his or her environment by exploiting heterogeneous sensors attached to the body. With the development of deep learning, new strategies have emerged to facilitate and solve the HAR problems. In this work, a deep multilayer bidirectional long-short memory (Bi-LSTM) architecture has been implemented to detect human activities. Instead of training a single model as in traditional LSTM methods, two models are presented in the Bi-LSTM scheme, one for learning the input data sequence and the other for learning the reverse sequence. Finally, a new novel postprocessing approach has been proposed based on windowing and voting in the last step to improve the average F1 score. Comprehensive investigations on the three publicly available datasets consisting of a different set of activities were used to evaluate the performance of the proposed framework. The empirical results of this paper on AReM, Mhealth and PAMAP2 datasets attained 95.46, 95.79 and 93.41% average F1 score, respectively. The results also revealed that selecting the window size and implementing the appropriate voting method has a significant effect on improving the average percentage of the F1 score.

RF-CM

Radio-Frequency (RF) based human activity recognition (HAR) enables many attractive applications such as smart home, health monitoring, and virtual reality (VR). Among multiple RF sensors, mmWave radar is emerging as a new trend due to its fine-grained sensing capability. However, laborious data collection and labeling processes are required when employing a radar-based sensing system in a new environment. To this end, we propose RF-CM, a general cross-modal human activity recognition framework. The key enabler is to leverage the knowledge learned from a massive WiFi dataset to build a radar-based HAR system with limited radar samples. It can significantly reduce the overhead of training data collection. In addition, RF-CM can work well regardless of the deployment setups of WiFi and mmWave radar, such as performing environments, users' characteristics, and device deployment. RF-CM achieves this by first capturing the activity-related variation patterns through data processing schemes. It then employs a convolution neural network-based feature extraction module to extract the high-dimensional features to be fed into the activity recognition module. Finally, RF-CM takes the generalization knowledge from WiFi networks as guide labels to supervise the training of the radar model, thus enabling a few-shot radar-based HAR system. We evaluate RF-CM by applying it to two HAR applications, fine-grained American sign language recognition (WiFi-cross-radar) and coarse-grained gesture recognition (WiFi-cross-RFID). The accuracy improvement of over 10% in both applications demonstrates the effectiveness of RF-CM. This cross-modal ability allows RF-CM to support more cross-modal applications.

Using a Hybrid Neural Network and a Regularized Extreme Learning Machine for Human Activity Recognition with Smartphone and Smartwatch.

Mobile health (mHealth) utilizes mobile devices, mobile communication techniques, and the Internet of Things (IoT) to improve not only traditional telemedicine and monitoring and alerting systems, but also fitness and medical information awareness in daily life. In the last decade, human activity recognition (HAR) has been extensively studied because of the strong correlation between people's activities and their physical and mental health. HAR can also be used to care for elderly people in their daily lives. This study proposes an HAR system for classifying 18 types of physical activity using data from sensors embedded in smartphones and smartwatches. The recognition process consists of two parts: feature extraction and HAR. To extract features, a hybrid structure consisting of a convolutional neural network (CNN) and a bidirectional gated recurrent unit GRU (BiGRU) was used. For activity recognition, a single-hidden-layer feedforward neural network (SLFN) with a regularized extreme machine learning (RELM) algorithm was used. The experimental results show an average precision of 98.3%, recall of 98.4%, an F1-score of 98.4%, and accuracy of 98.3%, which results are superior to those of existing schemes.

Transformer-based deep reverse attention network for multi-sensory human activity recognition

In today’s era, one of the important applications of Artificial Intelligence (AI) is Human Activity Recognition (HAR). It has a wide range of applicability in health monitoring for patients with chronic diseases, gaming consoles for gesture recognition, etc. Sensor-based HAR systems use signals collected over a period of time to label an activity. When we design an efficient sensor-based HAR system, a model requires learning an optimal association of spatial and temporal features. In this article, we propose a sensor-based HAR technique using the deep learning approach. We present a deep reverse transformer-based attention mechanism to guide the side residual features Unlike the conventional bottom-up approaches for feature fusion, we exploit a top-down feature fusion approach. The reverse attention is self-calibrated throughout the course of learning, which regularizes the attention modules and dynamically adjusts the learning rate. The overall framework outperforms several state-of-the-art methods and is shown to be statistically significant against these methods on five publicly available sensor-based HAR datasets, namely, MHEALTH, USC-HAD, WHARF, UTD-MHAD1, and UTD-MHAD2. Further, we conduct an ablation study to showcase the importance of each of the components of the proposed framework. Source code of this work is available at https://github.com/rishavpramanik/RevTransformerAttentionHAR.

Epileptic Patient Activity Recognition System Using Extreme Learning Machine Method

The Human Activity Recognition (HAR) system is the hottest research area in clinical research. The HAR plays a vital role in learning about a patient's abnormal activities; based upon this information, the patient's psychological state can be estimated. An epileptic seizure is a neurological disorder of the human brain and affects millions of people worldwide. If epilepsy is diagnosed correctly and in an early stage, then up to 70% of people can be seizure-free. There is a need for intelligent automatic HAR systems that help clinicians diagnose neurological disorders accurately. In this research, we proposed a Deep Learning (DL) model that enables the detection of epileptic seizures in an automated way, addressing a need in clinical research. To recognize epileptic seizures from brain activities, EEG is a raw but good source of information. In previous studies, many techniques used raw data from EEG to help recognize epileptic patient activities; however, the applied method of extracting features required much intensive expertise from clinical aspects such as radiology and clinical methods. The image data are also used to diagnose epileptic seizures, but applying Machine Learning (ML) methods could address the overfitting problem. In this research, we mainly focused on classifying epilepsy through physical epileptic activities instead of feature engineering and performed the detection of epileptic seizures in three steps. In the first step, we used the open-source numerical dataset of epilepsy of Bonn university from the UCI Machine Learning repository. In the second step, data were fed to the proposed ELM model for training in different training and testing ratios with a little bit of rescaling because the dataset was already pre-processed, normalized, and restructured. In the third step, epileptic and non-epileptic activity was recognized, and in this step, EEG signal feature extraction was automatically performed by a DL model named ELM; features were selected by a Feature Selection (FS) algorithm based on ELM and the final classification was performed using the ELM classifier. In our presented research, seven different ML algorithms were applied for the binary classification of epileptic activities, including K-Nearest Neighbor (KNN), Naïve Bayes (NB), Logistic Regression (LR), Stochastic Gradient Boosting Classifier (SGDC), Gradient Boosting Classifier (GB), Decision Trees (DT), and three deep learning models named Extreme Learning Machine (ELM), Long Short-Term Memory (LSTM), and Artificial Neural Network (ANN). After deep analysis, it is observed that the best results were obtained by our proposed DL model, Extreme Learning Machine (ELM), with an accuracy of 100% accuracy and a 0.99 AUC. Such high performance has not attained in previous research. The proposed model's performance was checked with other models in terms of performance parameters, namely confusion matrix, accuracy, precision, recall, F1-score, specificity, sensitivity, and the ROC curve.

Multi-View Human Action Recognition Using Skeleton Based-FineKNN with Extraneous Frame Scrapping Technique.

Human action recognition (HAR) is one of the most active research topics in the field of computer vision. Even though this area is well-researched, HAR algorithms such as 3D Convolution Neural Networks (CNN), Two-stream Networks, and CNN-LSTM (Long Short-Term Memory) suffer from highly complex models. These algorithms involve a huge number of weights adjustments during the training phase, and as a consequence, require high-end configuration machines for real-time HAR applications. Therefore, this paper presents an extraneous frame scrapping technique that employs 2D skeleton features with a Fine-KNN classifier-based HAR system to overcome the dimensionality problems.To illustrate the efficacy of our proposed method, two contemporary datasets i.e., Multi-Camera Action Dataset (MCAD) and INRIA Xmas Motion Acquisition Sequences (IXMAS) dataset was used in experiment. We used the OpenPose technique to extract the 2D information, The proposed method was compared with CNN-LSTM, and other State of the art methods. Results obtained confirm the potential of our technique. The proposed OpenPose-FineKNN with Extraneous Frame Scrapping Technique achieved an accuracy of 89.75% on MCAD dataset and 90.97% on IXMAS dataset better than existing technique.

WiTransformer: A Novel Robust Gesture Recognition Sensing Model with WiFi.

The past decade has demonstrated the potential of human activity recognition (HAR) with WiFi signals owing to non-invasiveness and ubiquity. Previous research has largely concentrated on enhancing precision through sophisticated models. However, the complexity of recognition tasks has been largely neglected. Thus, the performance of the HAR system is markedly diminished when tasked with increasing complexities, such as a larger classification number, the confusion of similar actions, and signal distortion To address this issue, we eliminated conventional convolutional and recurrent backbones and proposed WiTransformer, a novel tactic based on pure Transformers. Nevertheless, Transformer-like models are typically suited to large-scale datasets as pretraining models, according to the experience of the Vision Transformer. Therefore, we adopted the Body-coordinate Velocity Profile, a cross-domain WiFi signal feature derived from the channel state information, to reduce the threshold of the Transformers. Based on this, we propose two modified transformer architectures, united spatiotemporal Transformer (UST) and separated spatiotemporal Transformer (SST) to realize WiFi-based human gesture recognition models with task robustness. SST intuitively extracts spatial and temporal data features using two encoders, respectively. By contrast, UST can extract the same three-dimensional features with only a one-dimensional encoder, owing to its well-designed structure. We evaluated SST and UST on four designed task datasets (TDSs) with varying task complexities. The experimental results demonstrate that UST has achieved recognition accuracy of 86.16% on the most complex task dataset TDSs-22, outperforming the other popular backbones. Simultaneously, the accuracy decreases by at most 3.18% when the task complexity increases from TDSs-6 to TDSs-22, which is 0.14-0.2 times that of others. However, as predicted and analyzed, SST fails because of excessive lack of inductive bias and the limited scale of the training data.

Human Action Recognition: A Taxonomy-Based Survey, Updates, and Opportunities.

Human action recognition systems use data collected from a wide range of sensors to accurately identify and interpret human actions. One of the most challenging issues for computer vision is the automatic and precise identification of human activities. A significant increase in feature learning-based representations for action recognition has emerged in recent years, due to the widespread use of deep learning-based features. This study presents an in-depth analysis of human activity recognition that investigates recent developments in computer vision. Augmented reality, human-computer interaction, cybersecurity, home monitoring, and surveillance cameras are all examples of computer vision applications that often go in conjunction with human action detection. We give a taxonomy-based, rigorous study of human activity recognition techniques, discussing the best ways to acquire human action features, derived using RGB and depth data, as well as the latest research on deep learning and hand-crafted techniques. We also explain a generic architecture to recognize human actions in the real world and its current prominent research topic. At long last, we are able to offer some study analysis concepts and proposals for academics. In-depth researchers of human action recognition will find this review an effective tool.

Single Accelerometer to Recognize Human Activities Using Neural Networks.

Exoskeletons have decreased physical effort and increased comfort in activities of daily living (ADL) such as walking, squatting, and running. However, this assistance is often activity specific and does not accommodate a wide variety of different activities. To overcome this limitation and increase the scope of exoskeleton application, an automatic human activity recognition (HAR) system is necessary. We developed two deep-learning models for HAR using one-dimensional-convolutional neural network (CNN) and a hybrid model using CNNs and long-short term memory (LSTM). We trained both models using the data collected from a single three-axis accelerometer placed on the chest of ten subjects. We were able to classify five different activities, standing, walking on level ground, walking on an incline, running, and squatting, with an accuracy of 98.1% and 97.8%, respectively. A two subject real-time validation trial was also conducted to validate the real-time applicability of the system. The real-time accuracy was measured at 96.6% and 97.2% for the CNN and the hybrid model, respectively. The high classification accuracy in the test and real-time evaluation suggests that a single sensor could distinguish human activities using machine-learning-based models.

Hybrid classifier model with tuned weights for human activity recognition

A wide variety of uses, such as video interpretation and surveillance, human-robot interaction, healthcare, and sport analysis, among others, make this technology extremely useful, human activity recognition has received a lot of attention in recent decades. human activity recognition from video frames or still images is a challenging procedure because of factors including viewpoint, partial occlusion, lighting, background clutter, scale differences, and look. Numerous applications, including human-computer interfaces, robotics for the analysis of human behavior, and video surveillance systems all require the activity recognition system. This work introduces the human activity recognition system, which includes 3 stages: preprocessing, feature extraction, and classification. The input video (image frames) are subjected for preprocessing stage which is processed with median filtering and background subtraction. Several features, including the Improved Bag of Visual Words, the local texton XOR pattern, and the Spider Local Picture Feature (SLIF) based features, are extracted from the pre-processed image. The next step involves classifying data using a hybrid classifier that blends Bidirectional Gated Recurrent (Bi-GRU) and Long Short Term Memory (LSTM). To boost the effectiveness of the suggested system, the weights of the Long Short Term Memory (LSTM) and Bidirectional Gated Recurrent (Bi-GRU) are both ideally determined using the Improved Aquila Optimization with City Block Distance Evaluation (IACBD) method. Finally, the effectiveness of the suggested approach is evaluated in comparison to other traditional models using various performance metrics.

Data Integration Based Human Activity Recognition using Deep Learning Models

Regular monitoring of physical activities such as walking, jogging, sitting, and standing will help reduce the risk of many diseases like cardiovascular complications, obesity, and diabetes. Recently, much research showed that the effective development of Human Activity Recognition (HAR) will help in monitoring the physical activities of people and aid in human healthcare. In this concern, deep learning models with a novel automated hyperparameter generator are proposed and implemented to predict human activities such as walking, jogging, walking upstairs, walking downstairs, sitting, and standing more precisely and robustly. Conventional HAR systems are unable to manage real-time changes in the surrounding infrastructure. Improved HAR approaches overcome this constraint by integrating multiple sensing modalities. These multiple sensors can produce accurate information, leading to a better perception of activity recognition. The proposed approach uses sensor-level fusion to integrate gyroscope and accelerometer sensors. The analysis is carried out using the widely accepted benchmark UCI-HAR dataset. Based on several performance evaluation experiments, the classification accuracy of long short-term memory (LSTM), convolutional neural network (CNN), and deep neural network (DNN) classifiers is reported to be 96%, 92%, and 93%, respectively. Compared to state-of-the-art deep learning models, the proposed method gives better results.

CSI-Based Joint Location and Activity Monitoring for COVID-19 Quarantine Environments

The recent SARS-COV-2 virus, also known as COVID-19, badly affected the world’s healthcare system due to limited medical resources for a large number of infected human beings. Quarantine helps in breaking the spread of the virus for such communicable diseases. This work proposes a nonwearable/contactless system for human location and activity recognition using ubiquitous wireless signals. The proposed method utilizes the channel state information (CSI) of the wireless signals recorded through a low-cost device for estimating the location and activity of the person under quarantine. We propose to utilize a Siamese architecture with combined one-dimensional convolutional neural networks (1-D-CNNs) and bi-directional long short-term memory (Bi-LSTM) networks. The proposed method provides high accuracy for the joint task and is validated on two real-world testbeds, first, using the designed low-cost CSI recording hardware, and second, on a public dataset for joint activity and location estimation. The human activity recognition (HAR) results outperform state-of-the-art machine and deep learning methods, and localization results are comparable with the existing methods.

Developing a wearable human activity recognition (WHAR) system for an outdoor jacket

PurposeThe emergence of smart wearables using clothing as a technology platform is a significant milestone with considerable implications for industrial convergence, creating new value for fashion. This paper aimed to present a premeditated prototype to integrate a human activity recognition (HAR) system into outdoor clothing.Design/methodology/approachFor the development of wearable HAR (WHAR) clothing, this paper explored three subject areas: fashion design related to the structural feature of the clothing platform, electronics related to wearable circuits and modules design and graphic user interface design related to smartphone application development.FindingsFor WHAR functions in outdoor terrains, the coexistence of accelerometer–gyroscope sensing and distance-sensing could be practical to surpass the technological limitation of activity and posture recognition with gyro sensors highly depending on the changes of acceleration and angles.Research limitations/implicationsThrough the vital sign check and physical activity–change recognition function, this study's WHAR system allows users to check their health by themselves and avoid overwork. A quick rescue is possible manually and automatically in a dangerous situation by notifying others. Thus, it can help protect users' health and safety (life).Originality/valueThis study designed the modularization of HAR functions generally installed in indoor medical spaces. Through the approach, smart clothing–embracing WHAR systems optimized for health and safety care for outdoor environments was pursued to diversify expensive roles of clothing for technological applications.

Intermediate nanofibrous charge trapping layer-based wearable triboelectric self-powered sensor for human activity recognition and user identification

Significant research has been conducted to find practical methods to increase the triboelectric charge density of friction surfaces and improve the TENG output performance. In this study, a double-layer nanofibrous-TENG is newly proposed, consisting of MXene/P(VDF-TrFE) as a charge-generating layer and Siloxene/cobalt-nanoporous carbon/P(VDF-TrFE) as a charge-trapping layer, fabricated via a facile electrospinning process. The charge-generating layer generates abundant surface charges owing to the high electronegativity and electron affinity of MXene. Similarly, Siloxene as a filler in the charge-trapping layer improves the dielectric property of the layer, whereas hierarchically porous structure with a large surface area of cobalt nanoporous carbon provides more active sites for charge storage. After utilizing the charge-trapping layer, the current density and surface potential of the double-layer nanofibrous TENG is two-fold higher than the single-layer nanofibrous TENG. Furthermore, the TENG with Nylon 6/6 nanofiber as a positive friction layer, delivers a power density of 19 W m-2, which shows superior output performance compared to the state-of-the-art works. Finally, the fabricated device is attached to the shoe insole, and the triboelectric sensor data is analyzed using cutting-edge deep learning technology, which exhibited an accuracy of 99% in user identification and user activity recognition. Thus, this study investigates the possibilities of using the electrospun double-layer nanofibrous mat to boost the TENG output performance and explores its applications in artificial intelligence and human activity recognition systems.