Human Activity Recognition Framework Research Articles

Continuous Multi-View Human Action Recognition

Human action recognition which recognizes human actions in a video is a fundamental task in computer vision field. Although multiple existing methods with single-view or multi-view have been presented for human action recognition, these recognition approaches cannot be extended into new action recognition or action classification tasks, as well as discover underlying correlations among different views. To tackle the above problem, this paper proposes a new lifelong multi-view subspace learning framework for continuous human action recognition, which could exploit the complementary information amongst different views from a lifelong learning perspective. More specifically, a set of view-specific libraries is established to gradually store the useful information within multiple views. As a new action recognition task comes, we decompose the model parameters into a set of embedded parameters over view-specific libraries. A latent representation subspace is constructed via encouraging it to be close to different view-specific libraries, which can leverage the high-order correlations among different views and further avoid partial information for action recognition task. Meanwhile, we propose to employ an alternating direction strategy to optimize our proposed method. Empirical studies on real-world multi-view action recognition datasets have shown that our proposed framework attains the superior recognition performance and saves the computational time when continually learning new action recognition tasks.

Wearable multi-sensor data fusion approach for human activity recognition using machine learning algorithms

Wearable sensor based human activity recognition (HAR) has a broad range of applications in healthcare, fitness, smart home, and surveillance. In spite of the substantial amount of computational research on HAR, the open challenges in multi-sensor based activity recognition including complex temporal data, identifying discriminative feature vectors from multi-modal data and dimensionality reduction require considerable research attention. Hence, in this paper, using multi-resolution time-frequency analysis of received signal strength (RSS) between the wearable sensors, we present a supervised machine learning (ML) based activity recognition framework. The multi-sensor activity data acquired using the wireless sensor network (WSN) nodes and inertial-sensor embedded in a smartphone are decomposed using the discrete wavelet transform (DWT) and the empirical mode decomposition (EMD) techniques for extracting the prominent feature vector. Using the discriminative statistical features from DWT along with the entropy features from EMD, the four ML classifier models such as support vector machine (SVM), K-nearest neighbour (KNN), ensemble classifier (EC), and decision tree (DT) are trained to classify various human activities. The efficacy of the proposed design is assessed on a publicly available dataset from UCI. The experimental results assessed using the confusion matrix and parallel coordinate plot (PCP) substantiate that the proposed ML based HAR framework can achieve a maximum classification accuracy of 99.63% and is superior to several state-of-the-art ML techniques.

An efficient human activity recognition model based on deep learning approaches

Human Activity Recognition (HAR) has gained traction in recent years in diverse areas such as observation, entertainment, teaching and healthcare, using wearable and smartphone sensors. Such environments and systems necessitate and subsume activity recognition, aimed at recognizing the actions, characteristics, and goals of one or more individuals from a temporal series of observations streamed from one or more sensors. Different developed models for HAR have been explained in literature. Deep learning systems and algorithms were shown to perform highly in HAR in recent years, but these algorithms need lots of computerization to be deployed efficiently in applications. This paper presents a HAR lightweight, low computing capacity, deep learning model, which is ideal for use in real-time applications. The generic HAR framework for smartphone sensor data is proposed, based on Long Short-Term Memory (LSTM) networks for time-series domains and standard Convolutional Neural Network (CNN) used for classification. The findings demonstrate that many of the deployed deep learning and machine learning techniques are surpassed by the proposed model. TRANSLATE with x English Arabic Hebrew Polish Bulgarian Hindi Portuguese Catalan Hmong Daw Romanian Chinese Simplified Hungarian Russian Chinese Traditional Indonesian Slovak Czech Italian Slovenian Danish Japanese Spanish Dutch Klingon Swedish English Korean Thai Estonian Latvian Turkish Finnish Lithuanian Ukrainian French Malay Urdu German Maltese Vietnamese Greek Norwegian Welsh Haitian Creole Persian // TRANSLATE with COPY THE URL BELOW Back EMBED THE SNIPPET BELOW IN YOUR SITE Enable collaborative features and customize widget: Bing Webmaster Portal Back // ORIGINAL: "; langMenu.appendChild(origLangDiv); LanguageMenu.Init('LanguageMenu', LanguageMenu_keys, LanguageMenu_values, LanguageMenu_callback, LanguageMenu_popupid); window["LanguageMenu"] = LanguageMenu; clearInterval(intervalId); } }, 1); // ]]>

A human activity recognition framework for grossly labeled smartphone sensing data through combining genetic algorithm with multiple instance multiple label learning

A human activity recognition framework for grossly labeled smartphone sensing data through combining genetic algorithm with multiple instance multiple label learning

Privacy preserving human activity recognition framework using an optimized prediction algorithm

Human activity recognition, in computer vision research, is the area of growing interest as it has plethora of real-world applications. Inferring actions from one or more persons captured through a live video has its immense utility in the contemporary era. Same time, protecting privacy of humans is to be given paramount importance. Many researchers contributed towards this end leading to privacy preserving action recognition systems. However, having an optimized model that can withstand any adversary models that strives to disclose privacy information. To address this problem, we proposed an algorithm known optimized prediction algorithm for privacy preserving activity recognition (OPA-PPAR) based on deep neural networks. It anonymizes video content to have adaptive privacy model that defeats attacks from adversaries. The privacy model enhances the privacy of humans while permitting highly accurate approach towards action recognition. The algorithm is implemented to realize privacy preserving human activity recognition framework (PPHARF). The visual recognition of human actions is made using an underlying adversarial learning process where the anonymization is optimized to have an adaptive privacy model. A dataset named human metabolome database (HMDB51) is used for empirical study. Our experiments with using Python data science platform reveal that the OPA-PPAR outperforms existing methods.

Hybrid handcrafted and learned feature framework for human action recognition

Recognising human actions in video is a challenging task in real-world. Dense trajectory (DT) offers accurate recording of motions over time that is rich in dynamic information. However, DT models lack the mechanism to distinguish dominant motions from secondary ones over separable frequency bands and directions. By contrast, deep learning-based methods are promising over the challenge though still suffering from limited capacity in handling complex temporal information, not mentioning huge datasets needed to guide the training. To take the advantage of semantical meaningful and “handcrafted” video features through feature engineering, this study integrates the discrete wavelet transform (DWT) technique into the DT model for gaining more descriptive human action features. Through exploring the pre-trained dual-stream CNN-RNN models, learned features can be integrated with the handcrafted ones to satisfy stringent analytical requirements within the spatial-temporal domain. This hybrid feature framework generates efficient Fisher Vectors through a novel Bag of Temporal Features scheme and is capable of encoding video events whilst speeding up action recognition for real-world applications. Evaluation of the design has shown superior recognition performance over existing benchmark systems. It has also demonstrated promising applicability and extensibility for solving challenging real-world human action recognition problems.

Hybrid deep learning framework for human activity recognition

The aim of the recognition in the human activity is to recognize the actions of the individuals using a set of observations and their environmental conditions. Since last two decades, the research on this Human Activity Recognition (HAR) has captured the attention of several computer science communities because of the strength to provide support to different applications and the connection to different fields of study such as, human-computer interaction, healthcare, monitoring, entertainment and education. There are many existing methods like deep learning which have been used to develop to recognize the different activities of the human, but couldn’t identify the sudden change of the activities in the human. This paper presents a method using the deep learning methods which can recognize the specific identities and identify a change from one activity to another for the applications of the healthcare. In this method, a deep convolutional neural network is built using which the features are extracted for the collection of the data from the sensors. After which the Gated Recurrent Unit (GRU) captures the long-tern dependency between the different actions which helps to improve the identification rate of the HAR. From the CNN and GRU, a model of wearable sensor can be proposed which can identify the changes of the activities and can accurately recognize these activities. Experiment have been conducted using open-source University of California (UCI) HAR dataset which composed of six different activity such as lying, standing, sitting, walking downstairs, walking upstairs and walking. The CNN-based model achieves a detection accuracy of 95.99% whereas the CNN-GRU model achieves a detection accuracy of 96.79% which is better than most existing HAR methods.

Pose-Guided Graph Convolutional Networks for Skeleton-Based Action Recognition

Graph convolutional networks (GCN), which can model the human body skeletons as spatial and temporal graphs, have shown remarkable potential in skeleton-based action recognition. However, in the existing GCN-based methods, graph-structured representation of the human skeleton makes it difficult to be fused with other modalities, especially in the early stages. This may limit their scalability and performance in action recognition tasks. In addition, the pose information, which naturally contains informative and discriminative clues for action recognition, is rarely explored together with skeleton data in existing methods. In this work, we proposed pose-guided GCN (PG-GCN), a multi-modal framework for high-performance human action recognition. In particular, a multi-stream network is constructed to simultaneously explore the robust features from both the pose and skeleton data, while a dynamic attention module is designed for early-stage feature fusion. The core idea of this module is to utilize a trainable graph to aggregate features from the skeleton stream with that of the pose stream, which leads to a network with more robust feature representation ability. Extensive experiments show that the proposed PG-GCN can achieve state-of-the-art performance on the NTU RGB+D 60 and NTU RGB+D 120 datasets.

A Hybrid Duo-Deep Learning and Best Features Based Framework for燗ction燫ecognition

Human Action Recognition (HAR) is a current research topic in the field of computer vision that is based on an important application known as video surveillance. Researchers in computer vision have introduced various intelligent methods based on deep learning and machine learning, but they still face many challenges such as similarity in various actions and redundant features. We proposed a framework for accurate human action recognition (HAR) based on deep learning and an improved features optimization algorithm in this paper. From deep learning feature extraction to feature classification, the proposed framework includes several critical steps. Before training fine-tuned deep learning models – MobileNet-V2 and Darknet53 – the original video frames are normalized. For feature extraction, pre-trained deep models are used, which are fused using the canonical correlation approach. Following that, an improved particle swarm optimization (IPSO)-based algorithm is used to select the best features. Following that, the selected features were used to classify actions using various classifiers. The experimental process was performed on six publicly available datasets such as KTH, UT-Interaction, UCF Sports, Hollywood, IXMAS, and UCF YouTube, which attained an accuracy of 98.3%, 98.9%, 99.8%, 99.6%, 98.6%, and 100%, respectively. In comparison with existing techniques, it is observed that the proposed framework achieved improved accuracy.

An Efficient Human Instance-Guided Framework for Video Action Recognition.

In recent years, human action recognition has been studied by many computer vision researchers. Recent studies have attempted to use two-stream networks using appearance and motion features, but most of these approaches focused on clip-level video action recognition. In contrast to traditional methods which generally used entire images, we propose a new human instance-level video action recognition framework. In this framework, we represent the instance-level features using human boxes and keypoints, and our action region features are used as the inputs of the temporal action head network, which makes our framework more discriminative. We also propose novel temporal action head networks consisting of various modules, which reflect various temporal dynamics well. In the experiment, the proposed models achieve comparable performance with the state-of-the-art approaches on two challenging datasets. Furthermore, we evaluate the proposed features and networks to verify the effectiveness of them. Finally, we analyze the confusion matrix and visualize the recognized actions at human instance level when there are several people.

A multi-task learning based hybrid prediction algorithm for privacy preserving human activity recognition framework

There is ever increasing need to use computer vision devices to capture videos as part of many real-world applications. However, invading privacy of people is the cause of concern. There is need for protecting privacy of people while videos are used purposefully based on objective functions. One such use case is human activity recognition without disclosing human identity. In this paper, we proposed a multi-task learning based hybrid prediction algorithm (MTL-HPA) towards realising privacy preserving human activity recognition framework (PPHARF). It serves the purpose by recognizing human activities from videos while preserving identity of humans present in the multimedia object. Face of any person in the video is anonymized to preserve privacy while the actions of the person are exposed to get them extracted. Without losing utility of human activity recognition, anonymization is achieved. Humans and face detection methods file to reveal identity of the persons in video. We experimentally confirm with joint-annotated human motion data base (JHMDB) and daily action localization in YouTube (DALY) datasets that the framework recognises human activities and ensures non-disclosure of privacy information. Our approach is better than many traditional anonymization techniques such as noise adding, blurring, and masking.

DCNN based human activity recognition framework with depth vision guiding

The smartphone-based human activity recognition (HAR) provides abundant healthcare guidance for telemedicine and clinic treatment. The continually increasing daily activities cause many difficulties for recognition and labeling. Although multimodal data fusion and artificial intelligence (AI) techniques can solve these problems, big data collection and labeling are still heavy. This paper proposes a remarkable depth data-guided framework based on smartphones for complex HAR and automatic labeling. The hardware platform is utilized to collect information of depth vision from the Microsoft Kinect camera and Inertial Measurement Unit (IMU) signals from the smartphone simultaneously. This framework consists of five clustering layers and deep learning (DL) based classification model to identify 12 complex daily activities. The results show that the hierarchical k-medoids (Hk-medoids) algorithm obtains the labels with high accuracy (93.89%). Furthermore, the performance evaluation of the DCNN model is evaluated better by comparing it with other machine learning (ML) and DL methods.

A Lightweight Framework for Human Activity Recognition on Wearable Devices

Human Activity Recognition (HAR) is the automatic detection and understanding of human motion behavior based on data extracted from video camera, ambient sensors or wearable sensors which particularly has recently attracted increased attention from both researchers and industry. However, for running practical HAR systems on wearable devices, there are some requirements such as design and development of small, lightweight, powerful, and low-cost smart sensors. In this context, data must be continuously collected, and edge computing is a viable solution, which is an energy-efficient technique, offering real-time response and privacy requirements for HAR applications. Rather than sending data to the cloud, edge computing is a local process that minimizes the data transmission time and responds with low latency. Recently, HAR system designers have adopted deep learning techniques inspired by their outstanding performance in many application areas and achieved relevant gain in activity recognition performance, however, these techniques were not demonstrated suitable for running on resource constrained devices. Thus, designing energy-efficient deep learning models is critical for realizing efficient HAR for mobile applications. In this work, we present a lightweight framework for the deployment of low-power but accurate HAR systems for these devices. We also implement and run the system in a microcontroller and analyze computational cost and energy consumption, and how different system configurations and deep learning model complexity influence on that.

Simulation for a Mems-Based CTRNN Ultra-Low Power Implementation of Human Activity Recognition.

This paper presents an energy-efficient classification framework that performs human activity recognition (HAR). Typically, HAR classification tasks require a computational platform that includes a processor and memory along with sensors and their interfaces, all of which consume significant power. The presented framework employs microelectromechanical systems (MEMS) based Continuous Time Recurrent Neural Network (CTRNN) to perform HAR tasks very efficiently. In a real physical implementation, we show that the MEMS-CTRNN nodes can perform computing while consuming power on a nano-watts scale compared to the micro-watts state-of-the-art hardware. We also confirm that this huge power reduction doesn't come at the expense of reduced performance by evaluating its accuracy to classify the highly cited human activity recognition dataset (HAPT). Our simulation results show that the HAR framework that consists of a training module, and a network of MEMS-based CTRNN nodes, provides HAR classification accuracy for the HAPT that is comparable to traditional CTRNN and other Recurrent Neural Network (RNN) implantations. For example, we show that the MEMS-based CTRNN model average accuracy for the worst-case scenario of not using pre-processing techniques, such as quantization, to classify 5 different activities is 77.94% compared to 78.48% using the traditional CTRNN.

A Deep Learning-Based Framework for Human Activity Recognition in Smart Homes

Human behavior modeling in smart environments is a growing research area treating several challenges related to ubiquitous computing, pattern recognition, and ambient assisted living. Thanks to recent progress in sensing devices, it is now possible to design computational models able of accurate detection of residents’ activities and daily routines. For this goal, we introduce in this paper a deep learning-based framework for activity recognition in smart homes. This framework proposes a detailed methodology for data preprocessing, feature mining, and deep learning techniques application. The novel framework was designed to ensure a deep exploration of the feature space since three main approaches are tested, namely, the all-features approach, the selection approach, and the reduction approach. Besides, the framework proposes the evaluation and the comparison of several well-chosen deep learning techniques such as autoencoder, recurrent neural networks (RNN), and some of their derivatives models. Concretely, the framework was applied on the “Orange4Home” dataset which represents a recent dataset specially designed for smart homes research. Our main findings show that the best approach for efficient classification is the selection approach. Furthermore, our overall results outperformed baseline models based on random forest classifiers and the principal component analysis technique, especially the results of our RNN-based model for the all-features approach and the results of our autoencoder-based model for the feature reduction approach.

A transfer learning-based efficient spatiotemporal human action recognition framework for long and overlapping action classes

A transfer learning-based efficient spatiotemporal human action recognition framework for long and overlapping action classes

A smartphone sensors-based personalized human activity recognition system for sustainable smart cities

According to the Sustainable Development Agenda 2030 of the World Health Organization, maintaining physical activities have multiple societal privileges for healthier cities and societies. The amalgamation of the Internet of Things (IoT) and pervasive smartphones has become of paramount importance to produce a significant breakthrough in various domains of smart cities, including healthcare, fitness, skill assessment, and personal assistants, to support independent living. The IoT-supported devices capacitate, embedded with sensors, enabled numerous context-aware applications to recognize physical activities. There are some activity recognition applications; however, they are still deficient in recognizing activities accurately. In this paper, a novel framework for human activity recognition (HAR) is proposed using raw readings from a combination of fused smartphone sensors: accelerometer, gyroscope, magnetometer, and Google Fit activity tracking module. The proposed framework applies deep recurrent neural network (DRNN) to an extensive training dataset. The latter consists of five activity classes from 12 individuals using a deep recurrent neural network (DRNN). An extensive training dataset is used consisting of five activity classes from a group of 12 individuals. The designed android application (runs in the background) collects data from the smartphone's embedded sensors fused with the Google Fit API to validate the results proposed framework. The proposed framework shows promising results in recognizing human activities compared to other similar studies and achieves an accuracy of 99.43% for activity recognition using DRNN.

RF-Based Human Activity Recognition Using Signal Adapted Convolutional Neural Network

Human activity recognition (HAR) plays a critical role in a wide range of real-world applications, and it is traditionally achieved via wearable sensing. Recently, to avoid the burden and discomfort caused by wearable devices, device-free approaches exploiting radio-frequency (RF) signals arise as a promising alternative for HAR. Most of the latest device-free approaches require training a large deep neural network model in either time or frequency domain, entailing extensive storage to contain the model and intensive computations to infer human activities. Consequently, even with some major advances on device-free HAR, current device-free approaches are still far from practical in real-world scenarios where the computation and storage resources possessed by, for example, edge devices, are limited. To overcome these weaknesses, we introduce HAR-SAnet which is a novel RF-based HAR framework. It adopts an original signal adapted convolutional neural network architecture: instead of feeding the handcraft features of RF signals into a classifier, HAR-SAnet fuses them adaptively from both time and frequency domains to design an end-to-end neural network model. We apply point-wise grouped convolution and depth-wise separable convolutions to confine the model scale and to speed up the inference execution time. The experiment results show that the recognition accuracy of HAR-SAnet substantially outperforms the state-of-the-art algorithms and systems.

An adversarial framework for open-set human action recognition using skeleton data

An adversarial framework for open-set human action recognition using skeleton data

Low-Cost and Device-Free Human Activity Recognition Based on Hierarchical Learning Model.

Human activity recognition (HAR) has been a vital human–computer interaction service in smart homes. It is still a challenging task due to the diversity and similarity of human actions. In this paper, a novel hierarchical deep learning-based methodology equipped with low-cost sensors is proposed for high-accuracy device-free human activity recognition. ESP8266, as the sensing hardware, was utilized to deploy the WiFi sensor network and collect multi-dimensional received signal strength indicator (RSSI) records. The proposed learning model presents a coarse-to-fine hierarchical classification framework with two-level perception modules. In the coarse-level stage, twelve statistical features of time–frequency domains were extracted from the RSSI measurements filtered by a butterworth low-pass filter, and a support vector machine (SVM) model was employed to quickly recognize the basic human activities by classifying the signal statistical features. In the fine-level stage, the gated recurrent unit (GRU), a representative type of recurrent neural network (RNN), was applied to address issues of the confused recognition of similar activities. The GRU model can realize automatic multi-level feature extraction from the RSSI measurements and accurately discriminate the similar activities. The experimental results show that the proposed approach achieved recognition accuracies of 96.45% and 94.59% for six types of activities in two different environments and performed better compared the traditional pattern-based methods. The proposed hierarchical learning method provides a low-cost sensor-based HAR framework to enhance the recognition accuracy and modeling efficiency.