High Dimensional Space Model for Dense Monocular Surface Recovery

Abstract

Dense surface reconstruction from monocular image sequences — known as Non-Rigid Structure from Motion (NRSfM) — is a highly ill-posed inverse problem. The objective of NRSfM is to learn 3D shapes from 2D point tracks in an unsupervised manner. While existing methods rely on low-rank models, we propose the concept of High Dimensional Space Model (HDSM). In HDSM, time-varying geometry is encoded by a high-dimensional static structure projected into different metric subspaces. To express non-rigid deformations, instead of directly modelling in the 3D space, we gradually increase space dimensionality as the complexity of the scene increases. HDSM allows for a compact representation with deformation localisation and can be interpreted as a generalisation of the previously proposed models for NRSfM. Relying on HDSM, we develop an algorithm for dense monocular surface recovery. Experiments show that the proposed method achieves high accuracy while allowing for the fine-grained control.