A Perspective on Non-Isometric Shape-from-Template

Abstract

Shape-from-Template (SfT) uses an object's shape template and a deformation law to achieve single-image reconstruction. SfT is a fundamental tool to retexture or augment a deformable object in a monocular video. It has matured for isometric deformations, but the non-isometric case is yet largely open. This is because modeling is generally more complicated and the constraints certainly weaker. Existing algorithms use, for instance, linear elasticity, require one to provide boundary conditions represented by known deformed shape parts and need nonconvex optimization. We use a very simple and generic model to show that non-isometric SfT has a unique solution up to scale under strong perspective imaging and mild deformation curvature. Our model uses a novel type of homography interpretation that we call Perspective-Projection-Affine-Embedding. It may use boundary conditions if available and can be estimated with Linear Least Squares optimization. We provide experimental results on synthetic and real data.