3D Shape Reconstruction from Multi-view Video Data

Abstract

3D shape reconstruction has been a major research topic in computer vision and a variety of different algorithms have been developed. This chapter first defines a computational model for 3D video production, followed by an overview of Shape from X methods to investigate practically useful visual cues for 3D shape reconstruction from multi-view video data. Then, we categorize 3D shape reconstruction methods for 3D video production into three types and analyze their computational characteristics: (1) frame-wise 3D shape reconstruction, (2) simultaneous 3D shape and motion estimation, and (3) 3D shape sequence estimation. The computational characteristic analysis leads us to the three essential design factors for 3D shape reconstruction algorithms: (1) photo-consistency, (2) visibility evaluation, and (3) shape representation with corresponding computational model for optimization. Based on these design factors, we implemented several practical algorithms for frame-wise 3D shape reconstruction and simultaneous 3D shape and motion reconstruction. Their performance is evaluated quantitatively with synthesized data as well as qualitatively with real world multi-view video data of MAIKO dance and Yoga performance.