Efficient Depth Enhancement Using a Combination of Color and Depth Information.

Kyungjae Lee,Yuseok Ban,Sangyoun Lee

doi:10.3390/s17071544

Abstract

Studies on depth images containing three-dimensional information have been performed for many practical applications. However, the depth images acquired from depth sensors have inherent problems, such as missing values and noisy boundaries. These problems significantly affect the performance of applications that use a depth image as their input. This paper describes a depth enhancement algorithm based on a combination of color and depth information. To fill depth holes and recover object shapes, asynchronous cellular automata with neighborhood distance maps are used. Image segmentation and a weighted linear combination of spatial filtering algorithms are applied to extract object regions and fill disocclusion in the object regions. Experimental results on both real-world and public datasets show that the proposed method enhances the quality of the depth image with low computational complexity, outperforming conventional methods on a number of metrics. Furthermore, to verify the performance of the proposed method, we present stereoscopic images generated by the enhanced depth image to illustrate the improvement in quality.

Highlights

IntroductionWith the development of low-cost commercial RGB-D sensors such as Kinect and PrimeSense, computer vision technologies utilizing depth images or color and depth images have been used to develop many vision applications such as object tracking [1,2], pose estimation [3,4,5] for human-computer interaction (HCI), The practical use of depth information is recognized as a key technology for many three-dimensional multimedia applications
RGB-D sensors are used to identify color and depth simultaneously in real time
Extensive multimedia research based on depth information has been conducted, such as depth image-based rendering (DIBR) [12,13], free-viewpoint television (FTV) [14,15], augmented reality (AR) [16], virtual reality (VR) [17] and mixed reality (MR) [18]

Summary

Introduction

With the development of low-cost commercial RGB-D sensors such as Kinect and PrimeSense, computer vision technologies utilizing depth images or color and depth images have been used to develop many vision applications such as object tracking [1,2], pose estimation [3,4,5] for human-computer interaction (HCI), The practical use of depth information is recognized as a key technology for many three-dimensional multimedia applications. Depth sensors that rely on infrared laser light with a speckle pattern (e.g., the Kinect sensor) suffer from missing or inaccurate depth information. These problems are caused by the incorrect matching of infrared patterns and a positional difference between the internal infrared sensors.

Objectives

Discussion

Conclusion