Variational Fusion of Time-of-Flight and Stereo Data for Depth Estimation Using Edge-Selective Joint Filtering

Abstract

In this paper, we propose variational fusion of time-of-flight (TOF) and stereo data for depth estimation using edge-selective joint filtering (ESJF). ESJF is able to adaptively select edges for depth upsampling from the TOF depth map, stereo matching-based disparity map, and stereo images. We adopt ESJF to produce high-resolution (HR) depth maps with accurate edge information from low-resolution ones captured by the TOF camera. First, we measure confidences of TOF and stereo data based on a Gaussian function to be used as fusion weights. Then, we upsample the TOF depth map using ESJF and extract vertical and horizontal discontinuity maps from it. Finally, we perform variational fusion of TOF and stereo depth data guided by the discontinuity maps. Experimental results show that the proposed method successfully produces HR depth maps and outperforms the state of the art in preserving edges and removing noise.