Robust Sparse and Dense Nonrigid Structure From Motion

Abstract

Modeling deformable three-dimensional (3-D) shape from a video sequence is a fundamental task in computer vision. Nonrigid structure from motion (NRSfM) refers to the problem of recovering the 3-D shape and pose of an object, deforming over time from a monocular video sequence. Presently, dense NRSfM is a research problem of great interest in academia and the industry due to the large demand for 3-D data in various contexts. We provide a robust system for the sparse and dense NRSfM. The strength of our approach comes with the ability to deal with the trajectories corrupted with outliers that serve as an input to NRSfM. To tackle this problem, the input trajectories are processed with a density-based spatial clustering approach that is combined with a RANSAC technique for the outlier' s detection. Processing the trajectories with this process enhances the trajectories by removing the unwanted outliers. Also, extending the work from sparse to dense NRS fM substantially increases the difficulty of the optimization problem. Thus, the proposed system also provides asymptotic improvements to the current optimization approaches by providing an efficient and a novel supervised Gauss-Newton method. Extensive experiments have demonstrated that the proposed method outperforms most of the existing NRSfM methods. The results show that the proposed method reconstructs largely deforming objects accurately and efficiently.