Temporal-Dense Dynamic 3-D Reconstruction With Low Frame Rate Cameras

Abstract

Temporal-dense 3-D reconstruction for dynamic scenes is a challenging and important research topic in signal processing. Although dynamic scenes can be captured by multiple high frame rate cameras, high price, and large storage are still problematic for practical applications. To address this problem, we propose a new method for temporal-densely capturing and reconstructing dynamic scenes with low frame rate cameras, which consists of spatio-temporal sampling, spatio-temporal interpolation, and spatio-temporal fusion. In spatio-temporal fusion, dual-tree discrete wavelet transform and shape context are employed to compute positional constraints that drive a Poisson image editing framework to obtain unsampled images and hence realistic time-varying shapes. With this method, not only shapes but also textures are recovered. This method can be extended to temporal-denser reconstruction by simply adding more cameras or using a few higher frame rate cameras. Experimental results show that temporal-dense dynamic 3-D reconstruction can be achieved with low frame rate cameras by our proposed method.