Human Activity Recognition Research Research Articles

Multi-head CNN-based activity recognition and its application on chest-mounted sensor-belt

In recent years, a great deal of research has been done on the identification, monitoring, and classification of human activities. Human activity recognition (HAR) is a term commonly used to describe the automatic identification of physical activities. For activity recognition, there are primarily vision-based and sensor-based methods available. The computer vision-based method is generally effective in lab settings, but because of clutter, fluctuating light levels, and contrast, it may not perform well in real-world scenarios. Continuous monitoring and analysis of physiological signals obtained from heterogeneous sensors attached to an individual’s body is required to realise sensor-based HAR systems. Most of the previous research in human activity recognition (HAR) is biased along with feature engineering and pre-processing which requires a good amount of domain knowledge. Application-specific modelling and time-taking methods are involved in these approaches. In this work, the multi-head convolutional neural network-based human activity recognition framework is proposed where automatic feature extraction and classification are involved in the form of an end-to-end classification approach. Experiments of this approach are performed by taking raw wearable sensor data with few pre-processing steps and without the involvement of a handcrafted feature extraction technique. 99.23% and 93.55% accuracy are obtained on the WISDM and UCI-HAR datasets which denoted the much improvement in the assessment of HAR over other similar approaches. The model is also tested on locally collected data from a chest mounted belt with fabric sensors and an accuracy of 87.14% has been achieved on that data.

Empowering Participatory Research in Urban Health: Wearable Biometric and Environmental Sensors for Activity Recognition.

Participatory exposure research, which tracks behaviour and assesses exposure to stressors like air pollution, traditionally relies on time-activity diaries. This study introduces a novel approach, employing machine learning (ML) to empower laypersons in human activity recognition (HAR), aiming to reduce dependence on manual recording by leveraging data from wearable sensors. Recognising complex activities such as smoking and cooking presents unique challenges due to specific environmental conditions. In this research, we combined wearable environment/ambient and wrist-worn activity/biometric sensors for complex activity recognition in an urban stressor exposure study, measuring parameters like particulate matter concentrations, temperature, and humidity. Two groups, Group H (88 individuals) and Group M (18 individuals), wore the devices and manually logged their activities hourly and minutely, respectively. Prioritising accessibility and inclusivity, we selected three classification algorithms: k-nearest neighbours (IBk), decision trees (J48), and random forests (RF), based on: (1) proven efficacy in existing literature, (2) understandability and transparency for laypersons, (3) availability on user-friendly platforms like WEKA, and (4) efficiency on basic devices such as office laptops or smartphones. Accuracy improved with finer temporal resolution and detailed activity categories. However, when compared to other published human activity recognition research, our accuracy rates, particularly for less complex activities, were not as competitive. Misclassifications were higher for vague activities (resting, playing), while well-defined activities (smoking, cooking, running) had few errors. Including environmental sensor data increased accuracy for all activities, especially playing, smoking, and running. Future work should consider exploring other explainable algorithms available on diverse tools and platforms. Our findings underscore ML's potential in exposure studies, emphasising its adaptability and significance for laypersons while also highlighting areas for improvement.

Open Datasets in Human Activity Recognition Research—Issues and Challenges: A Review

Open Datasets in Human Activity Recognition Research—Issues and Challenges: A Review

ONTOLOGICAL, FULLY PROBABILISTIC KNOWLEDGE MODEL FOR HUMAN ACTIVITY RECOGNITION

Efficiency and scalability are obstacles that have not yet received a viable response from the human activity recognition research community. This paper proposes an activity recognition method. The knowledge model is in the form of ontology, the state-of-the-art in knowledge representation and reasoning. The ontology starts with probabilistic information about subjects’ low-level activities and location and then is populated with the assertion axioms learned from data or defined by the user. Unlike methods that choose only the most probable candidate from sensor readings, the proposed method keeps multiple candidates with the known degree of confidence for each one and involves them in decision making. Using this method, the system is more flexible to deal with unreliable data, readings from sensors, and the final recognition rate is improved. Besides, to resolve the scalability problem, a system is designed and implemented to do reasoning and storing in a relational database management system. Numerical evaluations and conceptual benchmarking prove the proposed system feasibility.

New machine learning approaches for real-life human activity recognition using smartphone sensor-based data

In recent years, mainly due to the application of smartphones in this area, research in human activity recognition (HAR) has shown a continuous and steady growth. Thanks to its wide range of sensors, its size, its ease of use, its low price and its applicability in many other fields, it is a highly attractive option for researchers. However, the vast majority of studies carried out so far focus on laboratory settings, outside of a real-life environment. In this work, unlike in other papers, progress was sought on the latter point. To do so, a dataset already published for this purpose was used. This dataset was collected using the sensors of the smartphones of different individuals in their daily life, with almost total freedom. To exploit these data, numerous experiments were carried out with various machine learning techniques and each of them with different hyperparameters. These experiments proved that, in this case, tree-based models, such as Random Forest, outperform the rest. The final result shows an enormous improvement in the accuracy of the best model found to date for this purpose, from 74.39% to 92.97%.

A Petri Net Architecture for Real-Time Human Activity Recognition in Work Systems

Real-time human-centered assistance in industrial processes depends on the individual history of the work person's activities in the work system and requires adequate methods for tracking the person's actions. Most research in human activity recognition is based on recognizing actions from video data using computer vision methods. Digital equipment, standardized machine data interfaces, and smart wearable devices extend the possibilities to describe the current state of the work system. Petri nets have already been applied to human activity recognition, however, without the requirement of detecting actions in real-time. This paper proposes a Petri net architecture that enables hierarchical description-based human activity recognition in industrial work processes. We present an extension, a Partitioned Colored Petri Net, based on the colored Petri net formalism that infers activities from state transitions of the work system in real-time. In a case study, we demonstrate the Petri net's application for an error-based learning system that visualizes error consequences in augmented reality using experimentable digital twins.

HIT HAR: Human Image Threshing Machine for Human Activity Recognition Using Deep Learning Models.

In recent days, research in human activity recognition (HAR) has played a significant role in healthcare systems. The accurate activity classification results from the HAR enhance the performance of the healthcare system with broad applications. HAR results are useful in monitoring a person's health, and the system predicts abnormal activities based on user movements. The HAR system's abnormal activity predictions provide better healthcare monitoring and reduce users' health issues. The conventional HAR systems use wearable sensors, such as inertial measurement unit (IMU) and stretch sensors for activity recognition. These approaches show remarkable performances to the user's basic activities such as sitting, standing, and walking. However, when the user performs complex activities, such as running, jumping, and lying, the sensor-based HAR systems have a higher degree of misclassification results due to the reading errors from sensors. These sensor errors reduce the overall performance of the HAR system with the worst classification results. Similarly, radiofrequency or vision-based HAR systems are not free from classification errors when used in real time. In this paper, we address some of the existing challenges of HAR systems by proposing a human image threshing (HIT) machine-based HAR system that uses an image dataset from a smartphone camera for activity recognition. The HIT machine effectively uses a mask region-based convolutional neural network (R-CNN) for human body detection, a facial image threshing machine (FIT) for image cropping and resizing, and a deep learning model for activity classification. We demonstrated the effectiveness of our proposed HIT machine-based HAR system through extensive experiments and results. The proposed HIT machine achieved 98.53% accuracy when the ResNet architecture was used as its deep learning model.

GSAS: Enhancing efficiency of human activity recognition using GRU based Sub-activity stitching

Recognition of human activities requires the design of highly efficient models for key-frame selection, image segmentation, feature extraction and selection, sub-activity classification, and sub-activity stitching. Due to the advent of effective frame selection and segmentation methods, most of the research in human activity recognition (HAR) is focused on feature extraction and its post-processive classification. Most of these models work on the principle of direct activity recognition, wherein the entire video sequence is scanned, and the final activity is recognized from it. In this approach has some lacunas, which include, limited accuracy due to classification of a large number of frames; limited scalability due to large delays for training and evaluation, and limited applicability due to the requirement of large datasets for model training and validation. To remove these drawbacks, this text proposes the design of a novel gated recurrent unit (GRU) based sub-activity stitching model (GSAS). The proposed model works in 2 phases, which include, sub-activity recognition using GRU, followed by sub-activity stitching using a 1D convolutional neural network (CNN). The former converts input frames into numerical sub-activity instances, while the latter processes these instances and converts them into final activity types. Due to the use of this 2-layered architecture, the proposed model can achieve a precision of 89%, a recall of 85%, and an accuracy of 90% which is higher than most of the recently proposed HAR models. The evaluation was done on untrimmed datasets, which makes the proposed model applicable to real-time video sequences. This text also recommends some transfer-learning approaches that can be applied to the proposed model to further improve its performance in terms of scalability and accuracy.

Activity recognition using multiplex limited penetrable visibility graph

To solve the dynamic problem of different activities in human activity recognition research, an activity recognition method based on a multiplex limited penetrable visibility graph is proposed. The 21 pressure values for each sampling are mapped to nodes in the first-layer network; then the average path length of nodes in the asynchronous periodic network is obtained, and the second-layer network used to explore different activities is built. Finally, the characteristic parameters and dynamic characteristics of different activities are explored and analyzed. The experimental results demonstrate that through the joint distribution of the average clustering coefficient and the maximum degree parameter of the node, the discrimination problems of different postures can be better realized, and it has good adaptability. It provides a new approach to gait recognition research that can be used in medical clinical diagnosis, rehabilitation training, and public health.

Time-Series Deep-Learning Classifier for Human Activity Recognition Based On Smartphone Built-in Sensors

Human Activity Recognition (HAR) is gaining more interest in recent years due to its growing role in many human-related sectors such as the health sector especially with elderly people and motion restricted patients. In recent years, there has been great progress in identifying human activity using various machine learning approaches. However, traditional methods of feature extraction are the most challenging in the feature selection process. Deep learning is a promising approach in the human activity recognition research area and has overcome the feature selection problem. However, several challenges are still open to research issues such as classification performance. This paper describes how to identify specific types of human physical activities using the accelerator and gyroscope data generated by the smartphone user. A deep convolutional neural network architecture has been proposed to perform HAR efficiently and effectively the system has been trained and tested over a dataset generated with the aid of 50 volunteers with four activities (walking, running, walking up-down stairs finally sitting-standing on the chair) events in real-world conditions. We chose four classes, each of which performs well, get to know our range of activities achieving 99% for validation and 99.8% for testing overall accuracy

Human action recognition approaches with video datasets—A survey

Human Activity Recognition has recently attracted considerable attention. This has been triggered by the rapid development of advance technologies and learning methods. Human action recognition can be actively used in a number of application domains which may positively influence various aspects of the daily life. These include, (1) preventing dangerous activities and detection of crimes such as theft, murder, and property damage, and (2) predicting pedestrian activities in traffic, among others. To better serve these applications and the like, it is essential to highlight the various aspects related to the existing methods so that their actual users could realize and identify the good performing methods that work fast and are capable of recognizing the correct activities with high accuracy. The latter scope is covered in this survey which summarizes and analyzes the methods that perform learning and analysis processes on video datasets to grasp a new perspective on human action recognition. The survey also covers the major datasets commonly used in human activity recognition research. Accordingly, this survey could be recognized as a valuable source for researchers and practitioners.

Octave Mix: Data Augmentation Using Frequency Decomposition for Activity Recognition

Research in human activity recognition (HAR) requires a huge amount of data, but it is not easy to collect such measured sensor data. Besides, there is no much application of data augmentation (DA) in HAR. This study proposes Octave Mix as a novel synthetic-style DA method for sensor-based HAR. The proposed method uses frequency decomposition to intersect low- and high-frequency waveforms. In addition, we propose a DA ensemble method and a training algorithm to ensure robustness to the original sensor data while applicable to various feature representations. To evaluate the Octave Mix method’s effectiveness, we conduct experiments using four different benchmark datasets of sensor-based HAR and achieve high estimation accuracy in our results. Furthermore, we demonstrate that ensembling two DA strategies: Octave Mix with rotation and mixup with rotation, achieves higher accuracy.

Analysis of Deep Neural Networks for Human Activity Recognition in Videos—A Systematic Literature Review

From the past few decades, Human activity recognition (HAR) is one of the vital research areas in computer vision in which much research is ongoing. The researcher’s focus is shifting towards this area due to its vast range of real-life applications to assist in daily living. Therefore, it is necessary to validate its performance on standard benchmark datasets and state-of-the-art systems before applying it in real-life applications. The primary objective of this Systematic Literature Review (SLR) is to collect existing research on video-based human activity recognition, summarize, and analyze the state-of-the-art deep learning architectures regarding various methodologies, challenges, and issues. The top five scientific databases (such as ACM, IEEE, ScienceDirect, SpringerLink, and Taylor & Francis) are accessed to accompany this systematic study by summarizing 70 different research articles on human activity recognition after critical review. Human activity recognition in videos is a challenging problem due to its diverse and complex nature. For accurate video classification, extraction of both spatial and temporal features from video sequences is essential. Therefore, this SLR focuses on reviewing the recent advancements in stratified self-deriving feature-based deep learning architectures. Furthermore, it explores various deep learning techniques available for HAR, challenges researchers to face to build a robust model, and state-of-the-art datasets used for evaluation. This SLR intends to provide a baseline for video-based human activity recognition research while emphasizing several challenges regarding human activity recognition accuracy in video sequences using deep neural architectures.

Advances in human action recognition: an updated survey

Research in human activity recognition (HAR) has seen tremendous growth and continuously receiving attention from both the Computer Vision and the Image Processing communities. Due to the existence of numerous publications in this field, undoubtedly, there have been a number of review papers on this subject that categorise these techniques. Many of the recent works have started to tackle more challenging problems and these proposed techniques are addressing more realistic real-world scenarios. Conspicuously, an updated survey that covers these methods is timely due. To simplify the categorisation, this study takes a two-layer hierarchical approach. At the top level, the categorisation is based on the basic process flow of HAR, i.e. input data-type, features-type, descriptor-type, and classifier-type. At the second layer, each of these components is further subcategorised based on the diversity of the proposed methods. Finally, a remark on the coming popularity of deep learning approach in this field is also given.

Compressive Representation for Device-Free Activity Recognition with Passive RFID Signal Strength

Understanding and recognizing human activities is a fundamental research topic for a wide range of important applications such as fall detection and remote health monitoring and intervention. Despite active research in human activity recognition over the past years, existing approaches based on computer vision or wearable sensor technologies present several significant issues such as privacy (e.g., using video camera to monitor the elderly at home) and practicality (e.g., not possible for an older person with dementia to remember wearing devices). In this paper, we present a low-cost, unobtrusive, and robust system that supports independent living of older people. The system interprets what a person is doing by deciphering signal fluctuations using radio-frequency identification (RFID) technology and machine learning algorithms. To deal with noisy, streaming, and unstable RFID signals, we develop a compressive sensing, dictionary-based approach that can learn a set of compact and informative dictionaries of activities using an unsupervised subspace decomposition. In particular, we devise a number of approaches to explore the properties of sparse coefficients of the learned dictionaries for fully utilizing the embodied discriminative information on the activity recognition task. Our approach achieves efficient and robust activity recognition via a more compact and robust representation of activities. Extensive experiments conducted in a real-life residential environment demonstrate that our proposed system offers a good overall performance and shows the promising practical potential to underpin the applications for the independent living of the elderly.

Analysis and Effect of Feature Selection with Generic Classifier over Activity Recognition Dataset for Daily Life Activity Recognition

years, the convergence in sensing environments and pervasive computing has opened a scope for human activity recognition research. In this paper, we form populations from smartphone accelerometer and gyroscope sensor training- based data for Human Activity Recognition (HAR). In the same vein, we described in this work an Activity Recognition database, assembled from the recorded activities of 24 subjects doing Activities of Daily Living (ADL). The paper then aggregated this time- series data into features and subsets of features were selected using two filters based, classifier- independent feature selection methods. We used 10-fold cross-validation strategy to validate the experimentation. Evaluation of the variation of generic decision tree classifiers showed that the feature subsets bring forth acceptable performances than classification with the entire feature set resulting in productive computer overhead in the reduced feature subset. Therefore, classifier-independence feature set should be useful for developing and improving HAR systems across and within populations.

Automatic video surveillance using statistical analysis of temporal posture sequences

PurposeThe problem of automatic recognition of human activity is one of the most important and challenging areas of research in computer vision because of the wide range of possible applications, for example surveillance, advanced human‐computer interactions, monitoring. This paper presents statistical computer vision approaches to automatically recognize different human activities.Design/methodology/approachThe human activity recognition process has three steps: firstly human blobs are segmented by motion analysis; then the human body posture is estimated and, finally a temporal model of the detected posture series is generated by discrete hidden Markov models to identify the activity.FindingsThe system was tested on image sequences acquired in a real archaeological site while some people simulated both legal and illegal actions. Four kinds of activity were automatically classified with a high percentage of correct detections.Research limitations/implicationsThe proposed approach provides efficient solutions to some of the most common problems in the human activity recognition research field: high detailed image requirement, sequence alignment and intensive user interaction in the training phase. The main constraint of this framework is that the posture estimation approach is not completely view independent.Practical implicationsResults of time performance tests were very encouraging for the use of the proposed method in real time surveillance applications.Originality/valueThe proposed framework can work using low cost cameras with large view focal lenses. It does not need any a priori knowledge of the scene and no intensive user interaction is required in the early training phase.