Enhancement of Depth Maps With Alpha Channel Estimation for 3-D Video

Abstract

Depth images are widely used for 3-D scene generation. In depth image acquisition, accurate estimation of the depths of object boundaries, which has a critical impact on the visual quality of the generated 3-D scene, is very difficult, especially in the case of objects with a hairy region. We aimed to generate a dynamic 3-D scene without serious degradation in visual quality by developing solutions for the problems that occur in depth images obtained using an active depth sensor. A novel alpha channel estimation algorithm is proposed for seamless composition along with a depth map improvement method for hairy objects. By utilizing additional depth or infrared (IR) information, the existing matting algorithm can be improved significantly. We further enhanced the alpha estimation method in the temporal domain. The depth map was enhanced by filtering depth values along spatiotemporal neighborhoods based on information provided by the color and alpha images. The proposed method was examined mainly using a time-of-flight (TOF) camera, and Kinect is used too. The experimental results demonstrated that the proposed method can generate a 3-D scene with a greater degree of naturalness as compared to other methods.