Real-time Human Activity Recognition Research Articles

Learning dictionaries of sparse codes of 3D movements of body joints for real-time human activity understanding.

Real-time human activity recognition is essential for human-robot interactions for assisted healthy independent living. Most previous work in this area is performed on traditional two-dimensional (2D) videos and both global and local methods have been used. Since 2D videos are sensitive to changes of lighting condition, view angle, and scale, researchers begun to explore applications of 3D information in human activity understanding in recently years. Unfortunately, features that work well on 2D videos usually don't perform well on 3D videos and there is no consensus on what 3D features should be used. Here we propose a model of human activity recognition based on 3D movements of body joints. Our method has three steps, learning dictionaries of sparse codes of 3D movements of joints, sparse coding, and classification. In the first step, space-time volumes of 3D movements of body joints are obtained via dense sampling and independent component analysis is then performed to construct a dictionary of sparse codes for each activity. In the second step, the space-time volumes are projected to the dictionaries and a set of sparse histograms of the projection coefficients are constructed as feature representations of the activities. Finally, the sparse histograms are used as inputs to a support vector machine to recognize human activities. We tested this model on three databases of human activities and found that it outperforms the state-of-the-art algorithms. Thus, this model can be used for real-time human activity recognition in many applications.

Handling real-world context awareness, uncertainty and vagueness in real-time human activity tracking and recognition with a fuzzy ontology-based hybrid method.

Human activity recognition is a key task in ambient intelligence applications to achieve proper ambient assisted living. There has been remarkable progress in this domain, but some challenges still remain to obtain robust methods. Our goal in this work is to provide a system that allows the modeling and recognition of a set of complex activities in real life scenarios involving interaction with the environment. The proposed framework is a hybrid model that comprises two main modules: a low level sub-activity recognizer, based on data-driven methods, and a high-level activity recognizer, implemented with a fuzzy ontology to include the semantic interpretation of actions performed by users. The fuzzy ontology is fed by the sub-activities recognized by the low level data-driven component and provides fuzzy ontological reasoning to recognize both the activities and their influence in the environment with semantics. An additional benefit of the approach is the ability to handle vagueness and uncertainty in the knowledge-based module, which substantially outperforms the treatment of incomplete and/or imprecise data with respect to classic crisp ontologies. We validate these advantages with the public CAD-120 dataset (Cornell Activity Dataset), achieving an accuracy of 90.1% and 91.07% for low-level and high-level activities, respectively. This entails an improvement over fully data-driven or ontology-based approaches.

Real-Time Human Action Recognition System Using Depth Map Sequences

This work presents a real-time human action recognition system that uses depth map sequence as input. The system contains the segmentation of human, the action modeling based on 3D shape context and the action graph algorithm. We effectively solve the problem of segmenting human from complex and cluttered scenes by combing a novel quadtree split-and-merge method and the codebook background modeling algorithm. We aims at recognizing actions that are used in games and interactions, especially complex actions that contain foot motion and body heave. By expanding the shape context descriptor into 3D space, we obtain translation and scale invariant features and get rid of normalization error, which is a common problem of real-life applications. Experiments in various scenarios demonstrate the high speed and excellent performance of our procedure.

Real-time human action recognition based on depth motion maps

This paper presents a human action recognition method by using depth motion maps (DMMs). Each depth frame in a depth video sequence is projected onto three orthogonal Cartesian planes. Under each projection view, the absolute difference between two consecutive projected maps is accumulated through an entire depth video sequence forming a DMM. An l2-regularized collaborative representation classifier with a distance-weighted Tikhonov matrix is then employed for action recognition. The developed method is shown to be computationally efficient allowing it to run in real-time. The recognition results applied to the Microsoft Research Action3D dataset indicate superior performance of our method over the existing methods.

3D flow estimation for human action recognition from colored point clouds

Abstract Motion perception and classification are key elements exploited by humans for recognizing actions. The same principles can serve as a basis for building cognitive architectures which can recognize human actions, thus enhancing challenging applications such as human robot interaction, visual surveillance, content-based video analysis and motion capture. In this paper, we propose an autonomous system for real-time human action recognition based on 3D motion flow estimation. We exploit colored point cloud data acquired with a Microsoft Kinect and we summarize the motion information by means of a 3D grid-based descriptor. Finally, temporal sequences of descriptors are classified with the Nearest Neighbor technique. We also present a newly created public dataset for RGB-D human action recognition which contains 15 actions performed by 12 different people. Our overall system is tested on this dataset and on the dataset used in Ballin, Munaro, and Menegatti (2012) , showing the effectiveness of the proposed approach in recognizing about 90% of the actions.

Recognizing Human Activities User-independently on Smartphones Based on Accelerometer Data

Real-time human activity recognition on a mobile phone is presented in this article. Unlike in most other studies, not only the data were collected using the accelerometers of a smartphone, but also models were implemented to the phone and the whole classification process (preprocessing, feature extraction and classification) was done on the device. The system is trained using phone orientation independent features to recognize five everyday activities: walking, running, cycling, driving a car and sitting/standing while the phone is in the pocket of the subject's trousers. Two classifiers were compared, knn (k nearest neighbors) and QDA (quadratic discriminant analysis). The models for real-time activity recognition were trained offline using a data set collected from eight subjects and these offline results were compared to real-time recognition rates, which are obtained by implementing models to mobile activity recognition application which currently supports two operating systems: Symbian^3 and Android. The results show that the presented method is light and, therefore, suitable for be used in real-time recognition. In addition, the recognition rates on the smartphones were encouraging, in fact, the recognition accuracies obtained are approximately as high as offline recognition rates. Also, the results show that the method presented is not an operating system dependent.

A Fast Statistical Approach for Human Activity Recognition

An essential part of any activity recognition system claiming be truly real-time is the ability to perform feature extraction in real-time. We present, in this paper, a quite simple and computationally tractable approach for real-time human activity recognition that is based on simple statistical features. These features are simple and relatively small, accordingly they are easy and fast to be calculated, and further form a relatively low-dimensional feature space in which classification can be carried out robustly. On the Weizmann publicly benchmark dataset, promising results (i.e. 97.8%) have been achieved, showing the effectiveness of the proposed approach compared to the-state-of-the-art. Furthermore, the approach is quite fast and thus can provide timing guarantees to real-time applications.

A lattice-based neuro-computing methodology for real-time human action recognition

This work describes a computational approach for a typical machine-vision application, that of human action recognition from video streams. We present a method that has the following advantages: (a) no human intervention in pre-processing stages, (b) a reduced feature set, (c) modularity of the recognition system and (d) control of the model’s complexity in acceptable for real-time operation levels. Representation of each video frame and feature extraction procedure are formulated in the lattice theory context. The recognition system consists of two components: an ensemble of neural network predictors which correspond to the training video sequences and one classifier, based on the PREMONN approach, capable of deciding at each time instant which known video source has potentially generated a new sequence of frames. Extensive experimental study on three well known benchmarks validates the flexibility and robustness of the proposed approach.

Real-time human action recognition on an embedded, reconfigurable video processing architecture

In recent years, automatic human action recognition has been widely researched within the computer vision and image processing communities. Here we propose a real-time, embedded vision solution for human action recognition, implemented on an FPGA-based ubiquitous device. There are three main contributions in this paper. Firstly, we have developed a fast human action recognition system with simple motion features and a linear support vector machine classifier. The method has been tested on a large, public human action dataset and achieved competitive performance for the temporal template class of approaches, which include “Motion History Image” based techniques. Secondly, we have developed a reconfigurable, FPGA based video processing architecture. One advantage of this architecture is that the system processing performance can be reconfigured for a particular application, with the addition of new or replicated processing cores. Finally, we have successfully implemented a human action recognition system on this reconfigurable architecture. With a small number of human actions (hand gestures), this stand-alone system is operating reliably at 12 frames/s, with an 80% average recognition rate using limited training data. This type of system has applications in security systems, man–machine communications and intelligent environments.