Recovering 3-D Human Body Postures from Depth Maps and Its Application in Human Activity Recognition

Abstract

Listen

Translate article

We present an approach of how to recover 3-D human body postures from depth maps captured by a stereo camera and an application of this approach to recognize human activities with the joint angles derived from the recovered body postures. With a pair of images captured with a stereo camera, first a depth map is computed to get the 3-D information (i.e., 3-D data) of a human subject. Separately the human body is modeled in 3-D with a set of connected ellipsoids and their joints: the joint is parameterized with the kinematic angles. Then the 3-D body model and 3-D data are co-registered with our devised algorithm that works in two steps: the first step assigns the labels of body parts to each point of the 3-D data; the second step computes the kinematic angles to fit the 3-D human model to the labeled 3-D data. The co-registration algorithm is iterated until it converges to a stable 3-D body model that matches the 3-D human posture reflected in the 3-D data. We present our demonstrative results of recovering body postures in full 3-D from continuous video frames of various activities with an error of about 60-140 in the estimated kinematic angles. Our technique requires neither markers attached to the human subject nor multiple cameras: it only requires a single stereo camera. As an application of our body posture recovery technique in 3-D, we present how various human activities can be recognized with the body joint angles derived from the recovered body postures. The features of body joints angles are utilized over the conventional binary body silhouettes and Hidden Markov Models are utilized to model and recognize various human activities. Our experimental results show the presented techniques outperform the conventional human activity recognition techniques.