Real-time 3D shape reconstruction, dynamic 3D mesh deformation, and high fidelity visualization for 3D video

Abstract

3D video [IEEE Multimedia (1997) 18] is the ultimate image media recording dynamic visual events in the real world as is; it records time varying 3D object shape with high fidelity surface properties (i.e., color and texture). Its applications cover wide varieties of personal and social human activities: entertainment (e.g., 3D game and 3D TV), education (e.g., 3D animal picture books), sports (e.g., sport performance analysis), medicine (e.g., 3D surgery monitoring), culture (e.g., 3D archive of traditional dances), and so on. In this paper, we propose: (1) a PC cluster system for real-time reconstruction of dynamic 3D object action from multi-view video images, (2) a deformable 3D mesh model for reconstructing the accurate dynamic 3D object shape, and (3) an algorithm of rendering natural-looking texture on the 3D object surface from the multi-view video images. Experimental results with quantitative performance evaluations demonstrate the effectiveness of these methods in generating high fidelity 3D video from multi-view video images.