3D triangular mesh matching through a sequence of registered 2D and 3D images

Abstract

VR systems were traditionally used for tasks relying on high-quality graphics rendering where virtual environments were entirely made of user-defined objects. We are foreseeing that serious breakthroughs will emerge, where 'augmented reality' environments will be created in a more dynamic fashion, using 2D and 3D data from computer vision sensors. Thus, virtual environments will not be made exclusively of user-defined objects, but also from real data. This data, after proper modeling to provide some behaviour and abstraction levels, will then be used to feed high-performance imaging systems. This paper deals with: 1) local surface modeling of the 3D data visible from each 2D viewpoint using a modified marching cubes algorithm; and 2) region matching of the local models (from neighboring views in the sequence) using color region clustering information from the 2D snapshots. Local models and the region matching information are used to build complete global models from a sequence of images.