Tracking of object deformations in color and depth video: deformation models and applications

Abstract

ABSTRACT The research on deformation tracking based on color image data has continuously gained a wide interest in thelast 15 years. In addition, using depth sensors such as the Microsoft Kinect , allows to mitigate the ambiguityproblems that arise when trying to solve the deformation tracking tasks on color images only, by adding depthinformation. However, the fusion of color and depth data is not straight forward, and the deformation trackingtask is still ill-posed due to the lack of a general deformation model. The problem is usually circumventedby providing special deformation functions for the task at hand, e.g., skeleton-based for reconstructing peopleor triangle-based for tracking planar surfaces. In this article we summarize the Analysis by Synthesis (AbS)approach for deformation tracking in depth and color video and show some successful applications of specializeddeformation functions. To overcome the issues with NURBS based deformation tracking we propose a newgeodesic RBF-based deformation model, which can adapt to any surface topology and shape, while keepingthe number of deformation parameters low. Example deformations for objects of di erent topologies are given,showing the versatility and eciency of the proposed model.Keywords: AbS, Deformation, Tracking, RBF, Geodesic