Generation, visualization, and editing of 3D video

Abstract

3D video is the ultimate image medium recording dynamic visual events in the real world as is. Recorded object behaviors can be observed from any viewpoint, because 3D video records the object's full 3D shape, motion, and precise surface properties (i.e. color and texture). In our last paper, we presented a method of reconstructing dynamic 3D object shape from multi-view video images, by which a temporal series of 3D voxel representations of the object behavior can be obtained in real-time. In this paper, following an overview of the real-time 3D shape reconstruction method, we present: 1) an algorithm of generating texture on the 3D object surface from the multi-view video images, and 2) an editing system for visualizing 3D video with an omnidirectional background image using versatile 3D camera works. This paper mainly discusses how we can generate high fidelity object images from arbitrary viewpoints based on the 3D object shape of limited accuracy. We propose a novel texture mapping algorithm which maps textures onto the 3D object surface depending on a viewpoint. Experimental results demonstrate its effectiveness in generating high fidelity object images from arbitrary viewpoints.