Depth image restoration based on bimodal joint sequential filling

Abstract

In this paper, we propose a depth image restoration algorithm based on the bimodal joint sequential filling. Basically, the types, causes, and characteristics of invalid points in the depth image are thoroughly analyzed. Besides, using the off-line calibration, accurate registration between the depth and the color image is effectively achieved. Finally, filling priority estimation based on the bimodal joint conditional entropy and the depth value prediction depended on the hypothesis testing are put forward and integrated to realize the depth image restoration under the framework of sequential filling. In the experiments, a comparison between different settings is carried out under the framework of sequential filling. Then the quantitative and qualitative comparisons between other competing methods and ours are implemented. The results demonstrate that our algorithm can restore various types of invalid points in the depth images, and it has significant improvements over several popular competing algorithms in terms of both robustness and preciseness.