Human Activity Identification Research Articles

Exploiting relation of video segments for temporal action detection

With the development of urban intelligence, intelligent identification of human activities has become imminent. Temporal Action Detection (TAD) is designed to identify real human social activities, which is a challenging task in video understanding. The current methods mainly use global features for boundary matching or predefine all possible proposals, while ignoring the interference of background and the causal relevance of the temporal action, resulting in the generation of more redundant proposals and the decline of detection accuracy. To fill this gap, we propose a novel Dilated Convolution Locate and Action Relevant Score model called DCAR. Specifically, DCAR includes a Dilated Location Network (DL-Net) and Action Relevance Calculation (ARC) block. For the DL-Net, we design a Boundary Feature Enhancement (BFE) block, which enhances the boundary feature of actions and fuses the similar features of different channels by pooling and channel squeeze to reduce the interference of the background. Meanwhile, we also design multiple dilated convolutional structures to aggregate long contextual information in time point/interval after boundary enhancement. For the ARC block, we use the hyperbolic space distance and cross attention to calculate the causal correlation of action proposals classification, which removes the misclassification of action proposals. We conduct extensive experiments on Thumos14 and ActivityNet-1.3, which shows our method significantly improves the performance and achieves state-of-the-art results. On Thumos14, at tIoU 0.3, 0.4 and 0.5, it achieves mAP of 63.2%, 57.0% and 48.5%, respectively; On ActivityNet-1.3 it reaches 9.42% at tIoU 0.95.

Multi-head CNN-based activity recognition and its application on chest-mounted sensor-belt

In recent years, a great deal of research has been done on the identification, monitoring, and classification of human activities. Human activity recognition (HAR) is a term commonly used to describe the automatic identification of physical activities. For activity recognition, there are primarily vision-based and sensor-based methods available. The computer vision-based method is generally effective in lab settings, but because of clutter, fluctuating light levels, and contrast, it may not perform well in real-world scenarios. Continuous monitoring and analysis of physiological signals obtained from heterogeneous sensors attached to an individual’s body is required to realise sensor-based HAR systems. Most of the previous research in human activity recognition (HAR) is biased along with feature engineering and pre-processing which requires a good amount of domain knowledge. Application-specific modelling and time-taking methods are involved in these approaches. In this work, the multi-head convolutional neural network-based human activity recognition framework is proposed where automatic feature extraction and classification are involved in the form of an end-to-end classification approach. Experiments of this approach are performed by taking raw wearable sensor data with few pre-processing steps and without the involvement of a handcrafted feature extraction technique. 99.23% and 93.55% accuracy are obtained on the WISDM and UCI-HAR datasets which denoted the much improvement in the assessment of HAR over other similar approaches. The model is also tested on locally collected data from a chest mounted belt with fabric sensors and an accuracy of 87.14% has been achieved on that data.

Emerging Trends in Sports and Artificial Intelligence: A Scientometric Analysis in Citespace

This study endeavors to utilize CiteSpace software to construct and examine co-citation networks of references within the domains of sports and artificial intelligence (AI), gathered from the Web of Science. The aim is to delineate the progression of AI applications in sports and to pinpoint current research focal points. Analyzing a corpus of 2,493 papers sourced from the Web of Science via CiteSpace, the findings reveal a year-on-year escalation in the application of AI within sports over the last twelve years (2010 to 2021). Machine learning has found extensive application in sports, notably in the analysis of athlete behavior, prediction of match outcomes, and physiological monitoring, among other areas. The evolution of sports and AI has witnessed two significant fluctuations within the past twelve years, specifically in 2011 and 2014. The modularity shift in 2011 indicated an uptick in the analysis of tasks and the identification of human activities. The year 2014 was marked as a pivotal moment with the advent of visualization utilization in team sports and wearable technologies. The study underscores the presence of numerous unexplored intellectual avenues within the sports and AI domain, which warrant further exploration in future research endeavors. Key milestones spanning from 2010 to 2021 are elucidated through visual analysis.

Contactless human activity and sport action identification based on UHF RFID tag wall and AI

The existing approaches to activity assisted living(AAL) are complex, expensive, and intrusive, which reduce theirpracticality and end-user acceptance. However, advancements inconcomitant technology, such as artificial intelligence and wirelesscommunications, can be leveraged to improvise advanced AALsystems that can reduce healthcare costs and hospitalisations. Such systems can identify, monitor, and localize hazardousactivities to respond swiftly to unforeseen events. To addressthese issues, we present a Transparent RFID Tag Wall (TRTWall) that utilizes a passive ultra-high frequency (UHF) arrayof radio frequency identification (RFID) tags and deep learningbasedpredictive analytics for contactless human physical activitymonitoring. Specifically, we consider five different activities thatinclude sitting, standing, walking (in both directions), and noactivity. Our experimental results demonstrate that TRT-Wallcan accurately differentiate among five distinct activities withan average accuracy is 95.6%. This suggests that our proposedcontactless AAL system possesses the significant potential toenhance elderly patient-assisted living.

Semi-supervised ensemble learning for human activity recognition in casas Kyoto dataset

—The automatic identification of human physical activities, commonly referred to as Human Activity Recognition (HAR), has garnered significant interest and application across various sectors, including entertainment, sports, and notably health. Within the realm of health, a myriad of applications exists, contingent upon the nature of experimentation, the activities under scrutiny, and the methodology employed for data and information acquisition. This diversity opens doors to multifaceted applications, including support for the well-being and safeguarding of elderly individuals afflicted with neurodegenerative diseases, especially in the context of smart homes. Within the existing literature, a multitude of datasets from both indoor and outdoor environments have surfaced, significantly contributing to the activity identification processes. One prominent dataset, the CASAS project developed by Washington State University (WSU) University, encompasses experiments conducted in indoor settings. This dataset facilitates the identification of a range of activities, such as cleaning, cooking, eating, washing hands, and even making phone calls. This article introduces a model founded on the principles of Semi-supervised Ensemble Learning, enabling the harnessing of the potential inherent in distance-based clustering analysis. This technique aids in the identification of distinct clusters, each encapsulating unique activity characteristics. These clusters serve as pivotal inputs for the subsequent classification process, which leverages supervised techniques. The outcomes of this approach exhibit great promise, as evidenced by the quality metrics' analysis, showcasing favorable results compared to the existing state-of-the-art methods. This integrated framework not only contributes to the field of HAR but also holds immense potential for enhancing the capabilities of smart homes and related applications.

A Review Work: Human Action Recognition in Video Surveillance Using Deep Learning Techniques

Despite being extensively used in numerous uses, precise and effective human activity identification continues to be an interesting research issue in the area of vision for computers. Currently, a lot of investigation is being done on themes like pedestrian activity recognition and ways to recognize people's movements employing depth data, 3D skeletal data, still picture data, or strategies that utilize spatiotemporal interest points. This study aims to investigate and evaluate DL approaches for detecting human activity in video. The focus has been on multiple structures for detecting human activities that use DL as their primary strategy. Based on the application, including identifying faces, emotion identification, action identification, and anomaly identification, the human occurrence forecasts are divided into four different subcategories. The literature has been carried several research based on these recognitions for predicting human behavior and activity for video surveillance applications. The state of the art of four different applications' DL techniques is contrasted. This paper also presents the application areas, scientific issues, and potential goals in the field of DL-based human behavior and activity recognition/detection.

TCN-attention-HAR: human activity recognition based on attention mechanism time convolutional network

Wearable sensors are widely used in medical applications and human–computer interaction because of their portability and powerful privacy. Human activity identification based on sensor data plays a vital role in these fields. Therefore, it is important to improve the recognition performance of different types of actions. Aiming at the problems of insufficient time-varying feature extraction and gradient explosion caused by too many network layers, a time convolution network recognition model with attention mechanism (TCN-Attention-HAR) was proposed. The model effectively recognizes and emphasizes the key feature information. The ability of extracting temporal features from TCN (temporal convolution network) is improved by using the appropriate size of the receiver domain. In addition, attention mechanisms are used to assign higher weights to important information, enabling models to learn and identify human activities more effectively. The performance of the Open Data Set (WISDM, PAMAP2 and USC-HAD) is improved by 1.13%, 1.83% and 0.51%, respectively, compared with other advanced models, these results clearly show that the network model presented in this paper has excellent recognition performance. In the knowledge distillation experiment, the parameters of student model are only about 0.1% of those of teacher model, and the accuracy of the model has been greatly improved, and in the WISDM data set, compared with the teacher's model, the accuracy is 0.14% higher.

Attention-enhanced gated recurrent unit for action recognition in tennis.

Human Action Recognition (HAR) is an essential topic in computer vision and artificial intelligence, focused on the automatic identification and categorization of human actions or activities from video sequences or sensor data. The goal of HAR is to teach machines to comprehend and interpret human movements, gestures, and behaviors, allowing for a wide range of applications in areas such as surveillance, healthcare, sports analysis, and human-computer interaction. HAR systems utilize a variety of techniques, including deep learning, motion analysis, and feature extraction, to capture and analyze the spatiotemporal characteristics of human actions. These systems have the capacity to distinguish between various actions, whether they are simple actions like walking and waving or more complex activities such as playing a musical instrument or performing sports maneuvers. HAR continues to be an active area of research and development, with the potential to enhance numerous real-world applications by providing machines with the ability to understand and respond to human actions effectively. In our study, we developed a HAR system to recognize actions in tennis using an attention-based gated recurrent unit (GRU), a prevalent recurrent neural network. The combination of GRU architecture and attention mechanism showed a significant improvement in prediction power compared to two other deep learning models. Our models were trained on the THETIS dataset, one of the standard medium-sized datasets for fine-grained tennis actions. The effectiveness of the proposed model was confirmed by three different types of image encoders: InceptionV3, DenseNet, and EfficientNetB5. The models developed with InceptionV3, DenseNet, and EfficientNetB5 achieved average ROC-AUC values of 0.97, 0.98, and 0.81, respectively. While, the models obtained average PR-AUC values of 0.84, 0.87, and 0.49 for InceptionV3, DenseNet, and EfficientNetB5 features, respectively. The experimental results confirmed the applicability of our proposed method in recognizing action in tennis and may be applied to other HAR problems.

IoT-assisted Human Activity Recognition Using Bat Optimization Algorithm with Ensemble Voting Classifier for Disabled Persons

Internet of Things (IoT)-based human action recognition (HAR) has made a significant contribution to scientific studies. Furthermore, hand gesture recognition is a subsection of HAR, and plays a vital role in interacting with deaf people. It is the automatic detection of the actions of one or many subjects using a series of observations. Convolutional neural network structures are often utilized for finding human activities. With this intention, this study presents a new bat optimization algorithm with an ensemble voting classifier for human activity recognition (BOA-EVCHAR) technique to help disabled persons in the IoT environment. The BOA-EVCHAR technique makes use of the ensemble classification concept to recognize human activities proficiently in the IoT environment. In the presented BOA-EVCHAR approach, data preprocessing is generally achieved at the beginning level. For the identification and classification of human activities, an ensemble of two classifiers namely long short-term memory (LSTM) and deep belief network (DBN) models is utilized. Finally, the BOA is used to optimally select the hyperparameter values of the LSTM and DBN models. To elicit the enhanced performances of the BOA-EVCHAR technique, a series of experimentation analyses were performed. The extensive results of the BOA-EVCHAR technique show a superior value of 99.31% on the HAR process.

Human Activity Recognition on Smartphones using Innovative Logistic Regression and Comparing Accuracy of Naive Bayes Algorithm

The objective of this study is to compare the Naive Bayes algorithm with Innovative Logistic Regression in order to enhance human activity identification for sitting and walking. To predict human activity, Naive Bayes and Innovative Logistic Regression are used with different training and testing splits. From each group, ten sets of samples are selected, yielding a total of twenty samples. About 80% of the data from an independent sample T test were utilized in the Gpower test (g power setup parameters: α = 0.05 and power = 0.80, β = 0.2). Compared to Naive Bayes (90.7210%), Innovative Logistic Regression (95.5680%) has higher accuracy, with a statistical significance value of P = 0.003 (p < 0.05). When compared to Naive Bayes, Innovative Logistic Regression has higher accuracy.

Elderly and visually impaired indoor activity monitoring based on Wi-Fi and Deep Hybrid convolutional neural network

A drop in physical activity and a deterioration in the capacity to undertake daily life activities are both connected with ageing and have negative effects on physical and mental health. An Elderly and Visually Impaired Human Activity Monitoring (EV-HAM) system that keeps tabs on a person’s routine and steps in if a change in behaviour or a crisis might greatly help an elderly person or a visually impaired. These individuals may find greater freedom with the help of an EVHAM system. As the backbone of human-centric applications like actively supported living and in-home monitoring for the elderly and visually impaired, an EVHAM system is essential. Big data-driven product design is flourishing in this age of 5G and the IoT. Recent advancements in processing power and software architectures have also contributed to the emergence and development of artificial intelligence (AI). In this context, the digital twin has emerged as a state-of-the-art technology that bridges the gap between the real and virtual worlds by evaluating data from several sensors using artificial intelligence algorithms. Although promising findings have been reported by Wi-Fi-based human activity identification techniques so far, their effectiveness is vulnerable to environmental variations. Using the environment-independent fingerprints generated from the Wi-Fi channel state information (CSI), we introduce Wi-Sense. This human activity identification system employs a Deep Hybrid convolutional neural network (DHCNN). The proposed system begins by collecting the CSI with a regular Wi-Fi Network Interface Controller. Wi-Sense uses the CSI ratio technique to lessen the effect of noise and the phase offset. The t- Distributed Stochastic Neighbor Embedding (t-SNE) is used to eliminate unnecessary data further. The data dimension is decreased, and the negative effects on the environment are eliminated in this process. The resulting spectrogram of the processed data exposes the activity’s micro-Doppler fingerprints as a function of both time and location. These spectrograms are put to use in the training of a DHCNN. Based on our findings, EVHAM can accurately identify these actions 99% of the time.

Enhancing temple surveillance through human activity recognition: A novel dataset and YOLOv4-ConvLSTM approach

Automated identification of human activities remains a complex endeavor, particularly in unique settings like temple environments. This study focuses on employing machine learning and deep learning techniques to analyze human activities for intelligent temple surveillance. However, due to the scarcity of standardized datasets tailored for temple surveillance, there is a need for specialized data. In response, this research introduces a pioneering dataset featuring Eight distinct classes of human activities, predominantly centered on hand gestures and body postures. To identify the most effective solution for Human Activity Recognition (HAR), a comprehensive ablation study is conducted, involving a variety of conventional machine learning and deep learning models. By integrating YOLOv4’s robust object detection capabilities with ConvLSTM’s ability to model both spatial and temporal dependencies in spatio-temporal data, the approach becomes capable of recognizing and understanding human activities in sequences of images or video frames. Notably, the proposed YOLOv4-ConvLSTM approach emerges as the optimal choice, showcasing a remarkable accuracy of 93.68%. This outcome underscores the suitability of the outlined methodology for diverse HAR applications in temple environments.

Bringing Online Egocentric Action Recognition Into the Wild

To enable a safe and effective human-robot cooperation, it is crucial to develop models for the identification of human activities. Egocentric vision seems to be a viable solution to solve this problem, and therefore many works provide deep learning solutions to infer human actions from first person videos. However, although very promising, most of these do not consider the major challenges that comes with a realistic deployment, such as the portability of the model, the need for real-time inference, and the robustness with respect to the novel domains (i.e., new spaces, users, tasks). With this letter, we set the boundaries that egocentric vision models should consider for realistic applications, defining a novel setting of egocentric action recognition in the wild, which encourages researchers to develop novel, applications-aware solutions. We also present a new model-agnostic technique that enables the rapid repurposing of existing architectures in this new context, demonstrating the feasibility to deploy a model on a tiny device (Jetson Nano) and to perform the task directly on the edge with very low energy consumption (2.4 W on average at 50 fps).

Identifying human failure events (HFEs) for external hazard probabilistic risk assessment

In recent years, several advancements in nuclear power plant (NPP) probabilistic risk assessment (PRA) have been driven by increased understanding of external hazards, plant response, and uncertainties. However, major sources of uncertainty associated with external hazard PRA remain. One important source is how risk-significant human actions that are carried out to enable plant response and recovery from natural hazards cause the close coupling of physical impacts on plants and overall plant risk during these hazard events. This makes human reliability and human-plant interactions important elements to consider in resolving PRA gaps in external hazards. One of the challenges in considering human response in external hazard probabilistic risk assessment (XHPRA) is that most existing human reliability analysis (HRA) models were not developed for assessing actions outside the control room (termed ex-control room actions) and hazard response. To support this new scope, HRA models will need to be developed or modified to support identification of human activities, causal factors, and uncertainties inherent in external hazard response, thereby providing insights regarding event timing and physical event conditions as they relate to human performance. In this study, there are two main objectives: (1) evaluate the applicability of an existing cognitive-based HRA method, Phoenix, to ex-control room actions, and (2) identify sources of uncertainty to be characterized or reduced in order to make this method suitable for XHPRA. The first step of such work is performed by assessing the suitability of existing HRA methods to support identifying human failure events (HFEs) for human response to flooding hazards. These HFEs are human actions or inactions that are involved in human responses to flooding hazards and could contribute to the loss of a critical function for the plant in the scenario being examined. In this work, decomposition analyses using the cognitive-based Phoenix HRA model are used to identify HFEs. The Phoenix method was found to be suitable for analyzing ex-control room actions as well as identifying specific HFEs and underlying crew failure modes (CFMs). However, the method's suitability for use in ex-control room actions would benefit from expanding the available CFMs to accommodate a larger variety of physical and communication tasks.

Recognizing human activities using light-weight and effective machine learning methodologies

Background Human activity recognition is a dynamic and challenging task. It is a large field of research and development. It involves predicting the movement of a person from the raw sensor data using a machine learning model. To accurately detect human activities for e-health systems, several research attempts have been carried out using data mining and machine learning techniques, but there still is room to improve the performance. To this aim, human activities such as walking, standing, laying, sitting, walking upstairs, walking downstairs are predicted using prominent machine learning models. The aim of human activity recognition is examining actions from photos or video clips. This serves as the driving force behind human activity identification systems' aim to accurately classify input data into the relevant activity category. Methods Six machine learning techniques, including decision tree, random forest, linear regression, Naïve bayes, k-nearest neighbour, and neural networks algorithms, were used for human activity recognition. Results The performance of decision tree, random forest, linear regression, Naïve bayes, k-nearest neighbor, and neural network algorithms was assessed with a human activity recognition dataset. From the results, the random forest classifier and neural network gave good results, whereas the Naïve bayes result was not satisfying. Conclusions We classified the SITTING, STANDING, LAYING, WALKING, WALKING_DOWNSTAIRS, WALKING_UPSTAIRS activities with machine learning techniques with 98% of accuracy

Human Action Recognition: A Taxonomy-Based Survey, Updates, and Opportunities.

Human action recognition systems use data collected from a wide range of sensors to accurately identify and interpret human actions. One of the most challenging issues for computer vision is the automatic and precise identification of human activities. A significant increase in feature learning-based representations for action recognition has emerged in recent years, due to the widespread use of deep learning-based features. This study presents an in-depth analysis of human activity recognition that investigates recent developments in computer vision. Augmented reality, human-computer interaction, cybersecurity, home monitoring, and surveillance cameras are all examples of computer vision applications that often go in conjunction with human action detection. We give a taxonomy-based, rigorous study of human activity recognition techniques, discussing the best ways to acquire human action features, derived using RGB and depth data, as well as the latest research on deep learning and hand-crafted techniques. We also explain a generic architecture to recognize human actions in the real world and its current prominent research topic. At long last, we are able to offer some study analysis concepts and proposals for academics. In-depth researchers of human action recognition will find this review an effective tool.

Human Activity Identification and Suspicious Behaviour Detection System

Recognizing human action has been one of the biggest challenges in computer vision for the past two decades. Recently, it has become feasible to extract precise and cost-effective skeleton information. Our proposed system utilizes a cut-based framework to identify human actions using skeleton data. By using a single stationary camera as input, this system can recognize various continuous human activities in real-time, including raising or waving one or more hands, sitting down, and bending over. The recognition process is based on machine learning. Firstly, a dataset with the human body's coordinates is created. Then, a training model is developed using Logistic Regression and an outcome to be achieved. Finally, the model is utilized to identify human activities such as sitting, running, and waking up, as well as recognizing suspicious behaviour.

A new method for human activity identification using convolutional neural networks

Body fitness monitoring applications are using mobile sensors to identify human activities. Human activity identification is a challenging task because of the wide availability of human activities. This paper proposes a novel technique that extracts the discriminative dimensions for human activity identification. Particularly, a novel technique with convolutional neural networks (CNN) is used for catching dependency. A deep convolutional neural network (DCNN) consists of two different types of layers, convolutional and pooling are used. The depth of each filter increases from left to right in the network. Three activities like walking, running, remaining still are collected from smart mobile sensors. The axis like x, y, and z information was transferred with column vector magnitude information and utilised for studying or training CNN. Experimental results show that CNN-based method achieves 93.67% accuracy than the baseline random forest approach's 89.20%.

Fire Hawk Optimizer with Deep Learning Enabled Human Activity Recognition

Human-Computer Interaction (HCI) is a sub-area within computer science focused on the study of the communication between people (users) and computers and the evaluation, implementation, and design of user interfaces for computer systems. HCI has accomplished effective incorporation of the human factors and software engineering of computing systems through the methods and concepts of cognitive science. Usability is an aspect of HCI dedicated to guaranteeing that human–computer communication is, amongst other things, efficient, effective, and sustaining for the user. Simultaneously, Human activity recognition (HAR) aim is to identify actions from a sequence of observations on the activities of subjects and the environmental conditions. The vision-based HAR study is the basis of several applications involving health care, HCI, and video surveillance. This article develops a Fire Hawk Optimizer with Deep Learning Enabled Activity Recognition (FHODL-AR) on HCI driven usability. In the presented FHODL-AR technique, the input images are investigated for the identification of different human activities. For feature extraction, a modified SqueezeNet model is introduced by the inclusion of few bypass connections to the SqueezeNet among Fire modules. Besides, the FHO algorithm is utilized as a hyperparameter optimization algorithm, which in turn boosts the classification performance. To detect and categorize different kinds of activities, probabilistic neural network (PNN) classifier is applied. The experimental validation of the FHODL-AR technique is tested using benchmark datasets, and the outcomes reported the improvements of the FHODL-AR technique over other recent approaches.

A new method for human activity identification using convolutional neural networks

A new method for human activity identification using convolutional neural networks