Phase-based frame rate up-conversion for depth video

Abstract

Due to the limits of frame rate sampling like time-of-flight and transmission bandwidth constraints of depth video, depth frame rate up-conversion is imperative. Most researches interpolate the depth frames using the motion information of the texture images, which require well-rectified images. They usually find dense correspondences to synthesize an in-between frame. Estimating the correspondences is sensitive to occlusion, disocclusion, and color or luminance changes. This paper proposes a phase-based method to interpolate depth frames without explicitly finding correspondences. Combining top-down phase difference preservation and bottom-up phase difference correction, varying amounts of motions are well modeled in a complex-steerable pyramid decomposition. The pixel amplitude is also shifted and interpolated based on the phase information. Finally, a depth value correction is employed to correct the inaccurate depth value. Experimental results show that the depth video interpolated by our proposed approach provides better subjective and objective qualities.