Motion History Image Research Articles

Multi‐view human action recognition using 2D motion templates based on MHIs and their HOG description

In this study, a new multi‐view human action recognition approach is proposed by exploiting low‐dimensional motion information of actions. Before feature extraction, pre‐processing steps are performed to remove noise from silhouettes, incurred due to imperfect, but realistic segmentation. Two‐dimensional motion templates based on motion history image (MHI) are computed for each view/action video. Histograms of oriented gradients (HOGs) are used as an efficient description of the MHIs which are classified using nearest neighbor (NN) classifier. As compared with existing approaches, the proposed method has three advantages: (i) does not require a fixed number of cameras setup during training and testing stages hence missing camera‐views can be tolerated, (ii) requires less memory and bandwidth requirements and hence (iii) is computationally efficient which makes it suitable for real‐time action recognition. As far as the authors know, this is the first report of results on the MuHAVi‐uncut dataset having a large number of action categories and a large set of camera‐views with noisy silhouettes which can be used by future workers as a baseline to improve on. Experimentation results on multi‐view with this dataset gives a high‐accuracy rate of 95.4% using leave‐one‐sequence‐out cross‐validation technique and compares well to similar state‐of‐the‐art approaches.

Human Action Recognition Using Motion History Image Based Temporal Segmentation

A human action recognition system based on image depth is proposed in this paper. Depth information features are not easily disturbed by noise; and due to this characteristic, the system can quickly extract foreground targets. Moreover, the target data, namely, depth and two-dimensional (2D) data, are projected to three orthogonal planes. In this manner, the action featured in the depth motion along the optical axis can clearly describe the trajectory. Based on the change of motion energy and the angle variations of motion orientations, the temporal segmentation (TS) method automatically segments the complex action into several simple movements. Three-dimensional (3D) data is further applied to acquire the three-viewpoint (3V) motion history trajectory, whereby a target’s motion is described through the motion history images (MHIs) from the 3Vs. The weightings corresponding to the gradients of the MHIs are included for determining the viewpoint that bests describe the target’s motion. In terms of feature extraction, the application of multi-resolution motion history histograms can effectively reduce the computational load and achieve a high recognition rate. Experimental results demonstrate that the proposed method can effectively solve the self-occlusion problem.

Gesture recognition of traffic police based on static and dynamic descriptor fusion

We present a method to recognize gestures made by Chinese traffic police based on the static and dynamic descriptor fusion for driver assistance systems and intelligent vehicles. Gesture recognition is made possible by combining the extracted static and dynamic features. First, the point cloud data of human upper body in each frame of input video is obtained to estimate the static descriptor with 2.5D gesture model. Then, the dynamic descriptor is estimated by computing the motion history image of the input RGB video sequence. Finally, the above two descriptors are fused and the mean structural similarity index is used to recognize the gestures made by Chinese traffic police. A comparative study and qualitative evaluation are proposed with other gesture recognition methods, which demonstrate that better recognition results can be obtained using the proposed method on a number of video sequences.

Fall detection in dusky environment

Accidental fall is the most prominent factor that causes the accidental death of elder people due to their slow body reaction. Automatic fall detection technology integrated in a health-care system can assist human monitoring the occurrence of fall, especially in dusky environments. In this paper, a novel fall detection system focusing mainly on dusky environments is proposed. In dusky environments, the silhouette images of human bodies extracted from conventional CCD cameras are usually imperfect due to the abrupt change of illumination. Thus, our work adopts a thermal imager to detect human bodies. The proposed approach adopts a coarse-to-fine strategy. Firstly, the downward optical flow features are extracted from the thermal images to identify fall-like actions in the coarse stage. The horizontal projection of motion history images (MHI) extracted from fall-like actions are then designed to verify the incident by the proposed nearest neighbor feature line embedding (NNFLE) in the fine stage. Experimental results demonstrate that the proposed method can distinguish the fall incidents with high accuracy even in dusky environments and overlapping situations.

Real-time monitoring of behavioural parameters related to psychological stress.

We have developed a system, allowing real-time monitoring of human gestures, which can be used for the automatic recognition of behavioural correlates of psychological stress. The system is based on a low-cost camera (Microsoft Kinect), which provides video recordings capturing the subject's upper body activity. Motion History Images (MHIs) are calculated in real-time from these recordings. Appropriate algorithms are thereafter applied over the MHIs, enabling the real-time calculation of activity-related behavioural parameters. The system's efficiency in real-time calculation of behavioural parameters has been tested in a pilot trial, involving monitoring of behavioural parameters during the induction of mental stress. Results showed that our prototype is capable to effectively calculate simultaneously eight different behavioural parameters in real-time. Statistical analysis indicated significant correlations between five of these parameters and self-reported stress. The preliminary findings suggest that our approach could potentially prove useful within systems targeting automatic stress detection, through unobtrusive monitoring of subjects.

Human action classification using adaptive key frame interval for feature extraction

Human action classification based on the adaptive key frame interval (AKFI) feature extraction is presented. Since human movement periods are different, the action intervals that contain the intensive and compact motion information are considered in this work. We specify AKFI by analyzing an amount of motion through time. The key frame is defined to be the local minimum interframe motion, which is computed by using frame differencing between consecutive frames. Once key frames are detected, the features within a segmented period are encoded by adaptive motion history image and key pose history image. The action representation consists of the local orientation histogram of the features during AKFI. The experimental results on Weizmann dataset, KTH dataset, and UT Interaction dataset demonstrate that the features can effectively classify action and can classify irregular cases of walking compared to other well-known algorithms.

A Fusion Face Recognition Approach Based on 7-Layer Deep Learning Neural Network

This paper presents a method for recognizing human faces with facial expression. In the proposed approach, a motion history image (MHI) is employed to get the features in an expressive face. The face can be seen as a kind of physiological characteristic of a human and the expressions are behavioral characteristics. We fused the 2D images of a face and MHIs which were generated from the same face’s image sequences with expression. Then the fusion features were used to feed a 7-layer deep learning neural network. The previous 6 layers of the whole network can be seen as an autoencoder network which can reduce the dimension of the fusion features. The last layer of the network can be seen as a softmax regression; we used it to get the identification decision. Experimental results demonstrated that our proposed method performs favorably against several state-of-the-art methods.

Action Recognition Based on Sub-action Motion History Image and Static History Image

In this paper, we propose a robust and effective framework to largely improve the performance of human action recognition using depth maps. The key contribution is the proposition of the Sub-action Motion History Image (SMHI) and Static History Image (SHI) in a depth sequence. We evenly subdivide the normalized motion energy into a set of segments which corresponding frame indices are used to partition a video into different sub-actions segments. The Local Binary Patterns (LBP) descriptor is then computed from the SMHI and SHI for the representation of an action. We evaluate the proposed framework on MSR Action3D dataset. Experimental results indicate that the proposed approach outperforms the most of the art methods and demonstrate the effectiveness of the proposed approaches.

차량검출 GMM 2.0을 적용한 도로 위의 차량 검출 시스템 구축

This paper presents a vehicle detection system using the video as a input image what has moving of vehicles.. Input image has constraints. it has to get fixed view and downward view obliquely from top of the road. Road detection is required to use only the road area in the input image. In introduction, we suggest the experiment result and the critical point of motion history image extraction method, SIFT(Scale_Invariant Feature Transform) algorithm and histogram analysis to detect vehicles. To solve these problem, we propose using applied Gaussian Mixture Model(GMM) that is the Vehicle Detection GMM(VDGMM). In addition, we optimize VDGMM to detect vehicles more and named VDGMM 2.0. In result of experiment, each precision, recall and F1 rate is 9%, 53%, 15% for GMM without road detection and 85%, 77%, 80% for VDGMM2.0 with road detection.

Multiview human activity recognition system based on spatiotemporal template for video surveillance system

An efficient view invariant framework for the recognition of human activities from an input video sequence is presented. The proposed framework is composed of three consecutive modules: (i) detect and locate people by background subtraction, (ii) view invariant spatiotemporal template creation for different activities, (iii) and finally, template matching is performed for view invariant activity recognition. The foreground objects present in a scene are extracted using change detection and background modeling. The view invariant templates are constructed using the motion history images and object shape information for different human activities in a video sequence. For matching the spatiotemporal templates for various activities, the moment invariants and Mahalanobis distance are used. The proposed approach is tested successfully on our own viewpoint dataset, KTH action recognition dataset, i3DPost multiview dataset, MSR viewpoint action dataset, VideoWeb multiview dataset, and WVU multiview human action recognition dataset. From the experimental results and analysis over the chosen datasets, it is observed that the proposed framework is robust, flexible, and efficient with respect to multiple views activity recognition, scale, and phase variations.

이종 알고리즘을 융합한 다중 이동객체 검출

Object tracking algorithms can reduce computational cost by avoiding computation over the whole image through the selection of region of interests based on object detection. So, accurate object detection is an important task for object tracking. The background subtraction algorithm has been widely used in moving object detection using a stationary camera. However, it has the problem of object detection error due to incorrect background modeling, whereas the method of background modeling has been improved by many researches. This paper proposes a new moving object detection algorithm to overcome the drawback of the conventional background subtraction algorithm by combining the background subtraction algorithm with the motion history image algorithm that is usually used in gesture detection. Although the proposed algorithm demands more processing time because of time taken for combining two algorithms, it meet the real-time processing requirement. Moreover, experimental results show that it has higher accuracy compared with the previous two algorithms.

Pedestrian activity classification using patterns of motion and histogram of oriented gradient

This paper presents a view-based spatio-temporal template formation scheme for pedestrian activity recognition and classification. It emphasizes on detailed classification of the entire wide range of pedestrian actions in the context of road crossing and their recognition from a viewpoint of a static car in the traffic signal. For this purpose, we develop a new pedestrian action dataset containing 160 videos of eight actions with twenty subjects per action. First, action sequences are converted into Motion History Image where silhouette sequence is condensed into gray scale images, while dominant motion information is preserved. Then, Histogram of Oriented Gradient of the Motion History Image is calculated from local appearance and shape within the image for feature extraction. Finally, Support Vector Machine is employed for pedestrian action classification. The demonstrated method successfully classified 91 % of the pedestrian action. To compare the method with other methods, we also implement some established methods with the developed dataset.

Computing a rodent’s diary

Rodent monitoring in biomedical laboratories is a time consuming and tedious task. Several automatic solutions that rely on different types of sensors have been proposed. Computer vision provides a significantly more universal and less intrusive solution. In this article we propose a new method to detect and classify three behaviors in rodents: exploring, rearing, and static. The method uses motion history images and a multiple classifier system to detect the three behaviors under typical laboratory conditions. It is independent of the color of the rodent and of the background. The method performs equally well on short and long video sequences, achieving a success rate of 87 %.

Emergency situation monitoring service using context motion tracking of chronic disease patients

Nowadays, great attention is paid to studies on fusion health and medical care combined of IT and BT for chronic disease patients due to westernization of dietary life, increase in stress, decrease in physical activities, and others. In reality, full recovery of chronic disease is difficult to achieve as its cause is diverse and complex. Therefore, the necessity for continuous management is proposed rather than approach to treat the disease. It is urgent to come up with the countermeasure since the lengthening of life expectancy in the aging society brings about the increase in chronic diseases and such increased medical expense becomes a big burden in socioeconomic activities. Companies are promoting test-projects in association with health management together with nationwide health management business for chronic diseases. In this study, we proposed the emergency situation monitoring service using context motion tracking for chronic disease patients. Proposed service diagnoses current status of patient based on contextual information collected and it provides information necessary for chronic disease management by analyzing life habits. Bio status recognition can provide proper service through the extraction of contextual data relevant to chronic disease patients. The context motion tracking provides emergency situation monitoring service accordingly with alert and symptom level in case of detecting symptoms through measured results and analysis. Semantic inference engine for context awareness conducts active and intelligent analysis on health condition and life patterns. Since it can properly correspond to extraordinary circumstances, it provides necessary service environment for emergency situation or symptoms. The cameras, speakers, and sensors are installed accordingly with the structure of indoor living space of user and the contextual information is transmitted from them. Considering the user convenience, motion history image is used for the motion recognition and continuous tracking from video. The system detects the patterns of expertise based on life pattern and psychological state through life log based motion detection and provides the service accordingly. It provides health related information and emergency situation monitoring service to user at anytime and anywhere and it is easy to use with simple handling. As a result, this system has the advantage of being able to detect emergency situations realistically and intuitively.

Temporal analysis for fast motion detection in a crowd

We present a fast motion detection technique in a crowd as an abnormal motion based on optical flow and a motion history image (MHI). Since a camera view is usually not in perpendicular with motion direction, the velocity of motion is not uniform spatially. Instead of object detection directly from an image, we separate an image into several blocks. In this paper, we propose a novel method to analyze a motion using MHI representation, called a shift space and a shift histogram. Together with a velocity histogram, the method can detect fast motion in a crowd, realizing local abnormal event detection. The performance of the proposed method is experimentally illustrated and evaluated.

On the Development Robust and Fast Algorithm of Action and Identity Recognition

Human action recognition and surveillance applications are playing a key important in the present days and took an increasing interest in modern. Since most previous methods strictly limited to action classification in different scenarios and not take attention to human identity that makes an action at the same time. We present a novel and fast algorithm to recognize action and identity in a single framework. We assumed one person makes one action in a video. To identify and training the owner of the video to the classifier, we proposed the watermark embedded as 2-D wavelet transform as binary image, which is contains identity information in the training video. We used these wavelet coefficients as identity descriptors. To represent feature motion representation, we used motion energy image (MEI) and motion history image (MHI) as temporal template of the human actions and Zernike moments to extract shape features of the action from MEI and MHI. In this research, a set of Zernike moment based feature vectors is proposed for human action recognition, which is capture the global properties of an object rather than the local ones. We have composed two different feature vectors by evaluating the variance values of lower order Zernike moments in the four-dimensional Zernike moment space with encouraging experimental results. It has discriminative information that is suitable for classification, especially on related actions, such as running and jogging, that is most previous researches fail to classify them even human vision HVS. Nearest neighbor classifier is used for action and identity categorization. The result of these experiments suggests that this method has a high recognition rate in both action and identity accuracy on KTH data sets.

Using Discrete Cosine Transform Based Features for Human Action Recognition

Recognizing human action in complex video sequences has always been challenging for researchers due to articulated movements, occlusion, background clutter, and illumination variation. Human action recognition has wide range of applications in surveillance, human computer interaction, video indexing and video annotation. In this paper, a discrete cosine transform based features have been exploited for action recognition. First, motion history image is computed for a sequence of images and then blocked-based truncated discrete cosine transform is computed for motion history image. Finally, K-Nearest Neighbor (K-NN) classifier is used for classification. This technique exhibits promising results for KTH and Weizmann dataset. Moreover, the proposed model appears to be computationally efficient and immune to illumination variations; however, this model is prone to viewpoint variations

Robust Abnormal Event Recognition via Motion and Shape Analysis at ATM Installations

Automated teller machines (ATM) are widely being used to carry out banking transactions and are becoming one of the necessities of everyday life. ATMs facilitate withdrawal, deposit, and transfer of money from one account to another round the clock. However, this convenience is marred by criminal activities like money snatching and attack on customers, which are increasingly affecting the security of bank customers. In this paper, we propose a video based framework that efficiently identifies abnormal activities happening at the ATM installations and generates an alarm during any untoward incidence. The proposed approach makes use of motion history image (MHI) and Hu moments to extract relevant features from video. Principle component analysis has been used to reduce the dimensionality of features and classification has been carried out by using support vector machine. Analysis has been carried out on different video sequences by varying the window size of MHI. The proposed framework is able to distinguish the normal and abnormal activities like money snatching, harm to the customer by virtue of fight, or attack on the customer with an average accuracy of 95.73%.

Research on Three-dimensional Motion History Image Model and Extreme Learning Machine for Human Body Movement Trajectory Recognition

Based on the traditional machine vision recognition technology and traditional artificial neural networks about body movement trajectory, this paper finds out the shortcomings of the traditional recognition technology. By combining the invariant moments of the three-dimensional motion history image (computed as the eigenvector of body movements) and the extreme learning machine (constructed as the classification artificial neural network of body movements), the paper applies the method to the machine vision of the body movement trajectory. In detail, the paper gives a detailed introduction about the algorithm and realization scheme of the body movement trajectory recognition based on the three-dimensional motion history image and the extreme learning machine. Finally, by comparing with the results of the recognition experiments, it attempts to verify that the method of body movement trajectory recognition technology based on the three-dimensional motion history image and extreme learning machine has a more accurate recognition rate and better robustness.

실시간 영상에서 물체의 색/모양 정보를 이용한 움직임 검출 알고리즘 구현

Motion detection according to the movement and the change area detection method according to the background difference and the motion history image for use in a motion estimation technique using a real-time image, the motion detection method according to the optical flow, the back-projection of the histogram of the object to track for motion tracking At the heart of MeanShift center point of the object and the object to track, while used, the size, and the like due to the motion tracking algorithm CamShift, Kalman filter to track with direction. In this paper, we implemented the motion detection algorithm based on color and shape information of the object and verify.