3D pose and shape estimation with deformable models in lifelike scenes

Abstract

In real world scenes a large amount of objects can occur, that have to be recognized and localized by humanoids. In this paper we propose a new approach to initialize three-dimensional pose and shape parameters for a large variety of objects, which is applicable to single images. In order to feed the recognition system with a priori knowledge, three-dimensional models are used and in order to cope with shape variations, deformable variants of these models are built. Thereby, three-dimensional object descriptions, which incorporate shape parameters, are provided to the recognition system. By discretizing relevant shape and pose parameters, synthetic model views are created. A new method for the selection of the best fitting model view is proposed, which is realized, by applying a chain of filters on large sets of relevant model views. The pose parameters, that are associated with the selected model view, are enhanced in precision by means of a model-based parameter optimization technique. Overall, the approach allows to cope with strongly variable object shapes by combining the benefits of appearance-based and deformable model-based approaches. We present experimental results, proving the high variability of the proposed method and its robustness against partial occlusions. Furthermore, the method was applied to a real world scene, where the estimated pose and shape parameters were used to initialize a tracking application.