An unsupervised open-set recognition method for user-independent human activity recognition

Video-based human action recognition is one of the most important and challenging areas of research in the field of computer vision. Human action recognition has found many pragmatic applications in video surveillance, human-computer interaction, entertainment, autonomous driving, etc. Owing to the recent development of deep learning methods for human action recognition, the performance of action recognition has significantly enhanced for challenging datasets. Deep learning techniques are mainly used for recognizing actions in images and videos comprising of Euclidean data. A recent development in deep learning methods is the extension of these techniques to non-Euclidean data or graph data with many nodes and edges. Human body skeleton resembles a graph, therefore, the graph convolutional network (GCN) is applicable to the non-Euclidean body skeleton. In the past few years, GCN has emerged as an important tool for skeleton-based action recognition. Therefore, we conduct a survey using GCN methods for action recognition. Herein, we present a comprehensive overview of recent GCN techniques for action recognition, propose a taxonomy for the categorization of GCN techniques for action recognition, carry out a detailed study of the benchmark datasets, enlist relevant resources and open-source codes, and finally provide an outline for future research directions and trends. To the best of authors' knowledge, this is the first survey for action recognition using GCN techniques.

This paper proposes a probabilistic, time efficient, data-driven method for human low and medium level activity recognition and indoor tracking. The obtained results can be applied to a probabilistic reasoner for high level activity recognition. The proposed method is tested on Opportunity, a dataset consisting of daily morning activities in a highly sensor-rich environment. The main objective of this research is to suggest and apply methods suitable for batch processing of big data. In this case, performance in terms of CPU time and efficiency in storage usage are the top priorities. We applied fast signal processing methods to compute proper features from different collections of sensor signals. The relevant collections of features are selected and fed into a classifier to obtain results in the form of probability for each instance belonging to available classes. Additionally, the most probable locations of each subject in the room are calculated by processing noisy data from location tags on the subjects' body. Afterwards, the proposed probabilistic data smoothing method is applied to further increase accuracy. To evaluate the methods, the most probable recognitions are benchmarked against the results of the Opportunity Challenge competitions as well as provided results by the Opportunity group. We also implemented a couple of well-known methods on the current dataset and compared them with ours. Moreover, the performance of different sensors assemblies is investigated. Our proposed method could obtain very close results in terms of accuracy while it is more optimal in terms of number of features and required time.

The article considers the problem of undefined complex classes activity recognition. The human activity recognition relevance in the field of production and services is presented. The classification of physical activity types is described. The possibilities of using methods based on data and knowledge for human activity recognition are considered. An analysis of zero-shot learning is given. A functional model of the method for undefined complex human activity recognition is proposed. The datasets are described. The HH101 and HH105 datasets obtained from CASAS smart homes were used to conduct experimental studies. An experimental study of the developed method for human activity recognition is carried out.

Human action recognition has recently started to find its way into applications in different applications. Accordingly, human action recognition methods are becoming increasingly important in our daily life. They are used for different purposes such as automation, security, surveillance, health, smart home systems, and customer behaviour prediction, among others. Though have more systems with methods provides a rich pool of choices, it is important to well understand the performance of these systems and their success rates in recognizing the right activities in order to decide on the most appropriate system for the current application domain. This survey tackles this issue by analyzing and commenting on the available human action recognition systems and methods.

In this paper, we present a method of human action recognition that uses spatial-temporal local and global motion descriptions from the image sequence with selected variability. The local motions use the dense optical flow velocity of the image sequence. The principal components are extracted from the silhouette image sequence of an action and are regarded as global motions. In order to address the variability, several parameters, such as anthropometry of the person, phase, camera observations (zoom, tilt, and rotation of the human body), and variations in view are proposed. We use support vector machine for learning and recognizing the actions. We successfully recognize some daily life human actions in the indoor and outdoor environment and our proposed method of human action recognition is robust and efficient.

Human Activity Recognition (HAR) refers to recognizing a human's ongoing actions through sensor data. At present, one of the main problems faced by Human Activity Recognition is that different subjects, devices and wearing positions can cause inconsistent sensor data distribution. When a classification model trained using some labeled dataset is used to classify a new unlabeled data with different distributions, there will be a significant performance loss. However, it is difficult to annotate manually sensor data for new subjects. Prior works applying unsupervised domain adaptation methods to solve this problem only used a single source domain. However, in practice, it is common to have multiple labeled source domains. Inspired by a work in the field of computer vision, we propose an unsupervised domain adaptation method for human activity recognition using multiple source domains. Experimental results on a commonly used public HAR dataset show that our model can effectively alleviate the performance loss caused by inconsistent distributions. Moreover, compared with the single-source domain adaptation, the multi-source domain adaptation method can improve the accuracy further.

An enhanced method for human action recognition

In this letter, a simple data augmentation method for micro-Doppler radar-based human activity recognition (HAR) is proposed. The proposed augmentation method can improve the performance of a neural network with insufficient training samples. It is applied directly to the spectrograms of the human activity radar data. The augmentation strategy consists of three operations: 1) time shift, 2) frequency disturbance, and 3) frequency shift. Without destroying this kinematic information in the spectrograms, the three operations are used to change the three attributes, i.e., dynamic-static state, instantaneous speed, and overall speed, of human motion spectrograms. The experimental results show that the proposed augmentation method can significantly improve the recognition accuracy of different classic deep models used in radar-based HAR. Moreover, we performed another experiment that utilizes the different groups of volunteers' data for training and testing. The results reveal that the generalization ability of the network can be significantly improved by the proposed augmentation method.

Motion representation is a challenging task in human action recognition. To represent motion, most traditional methods usually require certain intermediate processing steps such as actor segmentation, body tracking, and interest point detection, which make these methods sensitive to errors caused by these processing steps. In this paper, motivated by the successful recovery of low-rank matrix using robust principal component analysis (RPCA), we present a novel motion representation method for action recognition by extracting refined low-rank features of RPCA. Compared with the traditional methods, our method does not require the intermediate processing steps mentioned above. Unfortunately, with traditional λ, RPCA is incapable of extracting the discriminative information of motion in action videos, thus we first conduct extensive experiments to determine a feasible parameter λ suitable for action recognition. Then, we perform RPCA with this λ to obtain the low-rank images including the discriminative information of motion. To represent characteristic of the obtained low-rank images, we define two descriptors [i.e., edge distribution histogram (EDH) and accumulated edge distribution histogram (AEDH)] to refine the low-rank images. Finally, a support vector machine is trained to classify human actions represented by EDH or AEDH features. The efficacy of the proposed method is verified on three public datasets, and experimental results have shown the promising results of our method for human action recognition.

Action and activity representation and recognition are very demanding research area in computer vision and man-machine interaction. Though plenty of researches have been done in this arena, the field is still immature. Over the last decades, extensive research methodologies have been developed on human activity analysis and recognition for various applications. This paper overviews various recent methods for human activity recognition with analysis. We attempt to sum up the various methods related to human motion representation and recognition. We make an effort to categorize the recent methods from the best in the business, and finally figure out the short-comings and challenges to dig out in future to develop robust action recognition approaches. This work exclusively endeavors to encompass the researches related only to human action recognition mainly from 2001 till-to-date with critical assessment of the methods. We also present our work along with to solve some of the shortcomings. It will widely benefit the researchers to understand and compare the related advancements in this area.

Human action recognition has been widely used in various fields of computer vision, pattern recognition, and human–computer interaction and has attracted substantial attention. Combining deep learning and depth information, this paper proposed a method of human action recognition based on improved convolutional neural networks (CNN). First, we use the depth motion maps to extract the depth sequence features and obtain three projected maps corresponding to front, side, and the top views. On this basis, an improved CNN is constructed to realize the recognition of human action, which uses three-dimensional (3-D) input and two-dimensional process identification to speed up the computation and reduce the complexity of recognition process. We evaluate our approach on two public 3-D action datasets: MSR Action3D dataset and UT-Kinect dataset, and our private CTP Action3D dataset built using Kinect to collect data. The experimental results show that the proposed methods of human action recognition achieve higher average recognition rate of 91.3% on MSR Action3D dataset, 97.98% on UT-Kinect dataset, and the average recognition rate is 93.8% on our CTP Action3D dataset. Furthermore, the trained model on one depth video sequence dataset can be easily generalized to different datasets without changing network parameters.

Unsupervised learning for human activity recognition using smartphone sensors

Human action recognition is a way of retrieving videos emerged from Content Based Video Retrieval (CBVR).It is a growing area of research in the field of computer vision nowadays. Human action recognition has gained popularity because of its wide applicability in automatic retrieval of videos of particular action using visual features. The most common stages for action recognition includes: object and human segmentation, feature extraction, activity detection and classification. This paper describes the application and challenges of human action recognition. Features and limitations of various methods for human action recognition are discussed. This paper introduces survey on different types of actions like single person action recognition, two person or person-object interaction and multiple people action recognition.

Purpose: The objective of this research article is to methodically combine the existing literature on Human Activity Recognition (HAR) and provide an understanding of the present state of the HAR literature. Additionally, the article aims to suggest an appropriate HAR system that can be used for detecting real-time activities such as suspicious behavior, surveillance, and healthcare. Objective: This review study intends to delve into the current state of human activity detection and recognition methods, while also pointing towards promising avenues for further research and development in the field, particularly with regards to complex and multi-task human activity recognition across different domains. Design/Methodology/Approach: A systematic literature review methodology was adopted by collecting and analyzing the required literature available from international and national journals, conferences, databases and other resources searched through the Google Scholar and other search engines. Findings/Result: The systematic review of literature uncovered the various approaches of Human activity detection and recognition. Even though the prevailing literature reports the investigations of several aspects of Human activity detection and recognition, there is still room for exploring the role of this technology in various domains to enhance its robustness in detecting and recognizing of multiple human actions from preloaded CCTV cameras, which can aid in detecting abnormal and suspicious activities and ultimately reduce aberrant human actions in society. Originality/Value: This paper follows a systematic approach to examine the factors that impact the detection and recognition of Human activity and suggests a concept map. The study undertaken supplements the expanding literature on knowledge sharing highlighting its significance. Paper Type: Review Paper.

Human-robot collaborative assembly (HRCA) has become a vital technology in the current context of intelligent manufacturing. To ensure the efficiency and safety of the HRCA process, robots must rapidly and accurately recognize human assembly actions. However, due to the complexity and variability of the human state, it is challenging to accurately recognize such actions. Furthermore, with the lack of a large-scale assembly action dataset, the model only constructed from the data obtained in a single assembly scenario demonstrates limited robustness when applied to other situations. To achieve rapid and cost-effective action recognition, this paper proposes a method for human action recognition based on skeleton data and transfer learning. First, we screen the action samples which are similar to assembly actions from the NTU-RGB+D dataset to build the source dataset and reduce the dimension of its skeleton data. Afterwards, the Long Short-Term Memory (LSTM) network is used for learning universal features from the source dataset. Second, we use Microsoft Kinect to collect skeleton data of human assembly actions as the initial target dataset and use the sliding time window method to expand its size. After aligning the data of two datasets, the gradient freezing strategy is adopted during the transfer learning process to transfer the features learned from the source dataset into the recognition of HRCA actions. Third, the transfer model is validated through a small-scale reducer assembly task. The experimental results demonstrate that the method proposed can achieve assembly actions recognition rapidly and cost-effectively while ensuring a certain level of accuracy.