3D object registration in image sequences

Abstract

In this paper, we address the issue of pose estimation for 3D polyhedral objects in 2D color image sequences. Knowing the 3D object model defined as a mesh or as a set of segments with adjency properties, the generic approach proposed here involves a robust 2D segment extraction method. This method involves two consecutive frames in the sequence, and is based on three steps: (1) color segmentation, (2) color gradient map computation based on the minimum vector dispersion detector, (3) edge modeling simulating the specific shape and magnitude of the gradient, and local active segment-based 2D matching. The procedure of updating the edge model makes it possible to easily take into account partially occluded objects. Introducing an active matching reduces drastically confusion and ambiguity cases. The 3D/2D matching of the model is achieved by minimizing the distance between the projected segments of the 3D model and the segments extracted in the 2D image. Results on simulated and real images are presented and discussed, the accuracy of the pose estimation depending mainly on the accuracy of the 3D object model.