Correspondence labelling for wide-timeframe free-form surface matching

Abstract

This paper addresses the problem of estimating dense correspondence between arbitrary frames from captured sequences of shape and appearance for surfaces undergoing free-form deformation. Previous techniques require either a prior model, limiting the range of surface deformations, or frame-to-frame surface tracking which suffers from stabilisation problems over complete motion sequences and does not provide correspondence between sequences. The primary contribution of this paper is the introduction of a system for wide-timeframe surface matching without the requirement for a prior model or tracking. Deformation- invariant surface matching is formulated as a locally isometric mapping at a discrete set of surface points. A set of feature descriptors are presented that are invariant to isometric deformations and a novel MAP-MRF framework is presented to label sparse-to-dense surface correspondence, preserving the relative distribution of surface features while allowing for changes in surface topology. Performance is evaluated on challenging data from a moving person with loose clothing. Ground-truth feature correspondences are manually marked and the recall-accuracy characteristic is quantified in matching. Results demonstrate an improved performance compared to non-rigid point-pattern matching using robust matching and graph-matching using relaxation labelling, with successful matching achieved across wide variations in human body pose and surface topology.