A novel 2D to 3D video conversion system based on a machine learning approach

Abstract

There has been recently a significant increase in the number of available 3D displays and players. Nevertheless, the amount of 3D content has not increased in the same magnitude, creating a gap between 3D offer and demand. To reduce this difference, many algorithms have appeared that perform 2D-to-3D image and video conversion. These algorithms usually require several images from the same scene to perform the conversion. In this paper, an automatic algorithm for estimating the 3D structure of a scene from a single color image is proposed. It is based on the key assumption that color images with similar structure will likely present similar depth structures. The conversion algorithm is split into an offline and an online module to be easily deployable into consumer devices, such as smartphones or TVs. The offline module pre-processes a color+depth image database to speed up the subsequent depth estimation. The online module infers a depth prior from a color query image using the previous database as training data. Then, it is refined through a segmentation-guided filtering. The conversion algorithm has been evaluated in three publicly available databases, and compared with several state-of-theart algorithms to prove its efficiency.