Monocular 3D tracking of deformable surfaces using sequential second order cone programming

Abstract

Reconstructing structures of deformable objects from monocular image sequences is important for applications like visual servoing and augmented reality. In this paper, we propose a method to recover 3D shapes of deformable surfaces using sequential second order cone programming (SOCP). The key of our approach is to represent the surface as a triangulated mesh and introduce two sets of constraints, one for model-to-image keypoint correspondences which are SOCP constraints, another for retaining the original lengths of the mesh edges which are non-convex constraints. In the process of tracking, the surface structure is iteratively updated by solving sequential SOCP feasibility problems in which the non-convex constraints are replaced by a set of convex constraints over a local convex region. The shape constraints used in our approach is more generic than previous methods, that enables us to reliably recover surface shapes with smooth, sharp and other complex deformations. The capability and efficiency of our approach are evaluated quantitatively with synthetic image sequences and qualitatively with real image sequences.