Human pose estimation from corrupted silhouettes using a sub-manifold voting strategy in latent variable space

Abstract

In this paper, a learning-based framework is proposed for human pose estimation in complicated environments. Human silhouettes extracted from input images are always incomplete and corrupted due to shadows, occlusions, motion blur, or foreground/background color similarity. Given a corrupted body silhouette, our goal is to infer the corresponding pose structure robustly, and to reconstruct the input silhouette as well. The basic assumption of our method is that the body pose (and configuration) can be indicated by some parts (components) of the silhouette given a training data set. Based on this assumption, a robust statistical method is applied to gather the information from uncorrupted components, and to ignore the effects from the outliers. In this method, Gaussian Process is used to learn the low-dimensional manifold of visual input data, and to create the sub-manifold corresponding to each component of the silhouette. Different from traditional methods, the likelihood probability is computed by means of a sub-manifold voting strategy based on the learned sub-manifolds. By fusing the likelihood and the prior of human poses, the proposed learning-based framework can specify the location of the input human pose in the latent space. The intrinsic pose and configuration can then be deduced from this location, or be refined after outlier rejection. Experiments show that our approach has a great ability to estimate human poses from corrupted silhouettes with small computational burden. Therefore, it can be applied for tracking initialization, 3D pose estimation, 2D configuration reconstruction in occluded, shadowed and noisy environments.