A super-resolution method of combined color image with depth map based on deep learning

Abstract

The task of a qualitative increase in image resolution is one of the most current issues of digital image processing. Super-resolution (SR) methods are the group of signal-processing algorithms, which allow producing a high-resolution (HR) image from single or multiple low-resolution (LR) images of the same scene. Convolutional neural network (CNN) has been widely applied to color image and depth map super-resolution problem, where a high-resolution depth map can be restored from a LR depth map with the guidance of an additional HR or LR color image of the same scene. Proposed method is based on the algorithm, that HR depth map is reconstructed by joint LR depth map and corresponding LR intensity image. The Joint double branch network (JDBNet) is formed with a multi-scale upsampling conception for solving image super-resolution problems. Such approach can considerably enhance the condition of the recovered HR depth images. Low-resolution intensity image and low-resolution depth map of the same scene are input data for training networks. The output data of the system is a high-resolution depth map. The performance of represented methods was evaluated by Root Mean Square Error (RMSE).