Color Correction and Depth-Based Hierarchical Hole Filling in Free Viewpoint Generation

Abstract

In this paper, a high-quality depth-image-based-rendering-based free viewpoint generation model is proposed, which consists of three parts. First, a boundary-aware 3-D warping is introduced for projecting the side views from original view plane to target virtual view plane. Specifically, a high-pass filter is used to remove the ghost contours along object boundaries. Second, a color correction-based image blending (CC-IB) procedure is put forward for synthesizing the two warped side views. During the proposed CC-IB, occluded regions are color-corrected and reconstructed in the first place, on the basis of an equivalent substitution thought and a polynomial fitting method, then color-continuous virtual view can be produced by blending the un-occluded regions and the reconstructed disocclusions. Finally, a depth guided hierarchical hole-filling method is presented for filling the remaining holes in the blended virtual view. With the assistance of depth, background pixels are preferentially selected for filling the holes. Experimental results show that the proposed model can produce high-quality virtual views with comfortable visual perceptions. Comparisons with other state-of-the-art models demonstrate that the proposed method can achieve good performance in both visual quality and quantitative evaluation.