A Differential–Algebraic Projective Framework for the Deformable Single-View Geometry of the 1D Perspective Camera

Abstract

Single-View Geometry (SVG) studies the world-to-image mapping or warp, which is the relationship that exists between a body’s model and its image. For a rigid body observed by a projective camera, the warp is described by the usual camera matrix and its properties. However, it is clear that for a body whose deformation state changes between the body’s model and its image, the ‘simple,’ globally parameterized warp described solely by the camera matrix, breaks down. Existing work has exploited deformation to reconstruct the deformed body from its image, but did not establish the properties of the deformable warp. Studying these properties is part of deformable SVG and forms a recent research topic. Because deformations may take place anywhere in the object’s body, and because they may be uncorrelated, the warp is local in nature. Using a differential framework is thus an obvious choice. We propose a differential–algebraic projective framework based on modeling the body’s surface by a locally rational projective embedding and on the 1D projective camera. We show that this leads, via the study of univariate rational functions, to differential invariants that the warp must satisfy. It may seem surprising, given the generic hypothesis made on the observed body, hardly stronger than mere local smoothness, that constraints can still be found. Our framework generalizes the Schwarzian derivative, the first-order projective differential invariant, which holds under the assumption that the body’s shape is locally linear. Our invariants may be used to construct regularizers to be used in warp estimation. We report experimental results of two types on simulated and real data. The first type shows that the proposed invariants hold well for an independently estimated warp. The second type shows that the proposed regularizers improve warp estimation from point correspondences compared to the classical derivative-penalizing regularizers.