Semi-Automatic 2D-to-3D Conversion Using Low-Rank Matrix Recovery

Hongxing Yuan

doi:10.3991/ijoe.v14i01.7838

Abstract

Semi-automatic 2D-to-3D conversion is a promising solution to 3D stereoscopic content creation. However, the depth continuous transition between user marked neighboring regions will be lost when user scribbles are sparse. To help solve this problem, a piecewise-continuity regularized low-rank matrix recovery method is developed. Our approach is based on the fact that a depth-map can be decomposed into a low-rank matrix and an outlier term matrix. First, an initial dense depth-map is interpolated from the user scribbles using matting Laplacian scheme under the assumption that depth-map is piecewise-continuous. Second, a piecewise-continuity constrained low-rank recovery model is developed to remove outliers which are introduced by the interpolation. Experimental comparisons with existing algorithms show that our method demonstrates significant advantage over depth continuous transition between neighboring regions.

Highlights

With the development of 3D display technology, increasingly kinds of 3D electronic products, such as television, mobile phone, projector, are appearing in the ordinary people's life [1]
We report some experimental results which compare our approach with Random Walks (RW) [10], Graph Cuts (GC) [12], hybrid Graph Cuts and RaniJOE ‒ Vol 14, No 1, 2018
Panel (a) illustrates the input color image, panel (b) shows the user marked scribbles overlaid on the original image, panel (c) is the depth-map generated by GC [12], panel (d) is the result done by random walks (RW) [10], panel (e) is the depth from HGR [12], panel (f) shows depth generated by our approach

Summary

Introduction

With the development of 3D display technology, increasingly kinds of 3D electronic products, such as television, mobile phone, projector, are appearing in the ordinary people's life [1]. Most videos and images are still in 2D. It is urgent need for 2Dto3D conversion which can generate 3D content from existing 2D images/videos. The main challenge of 2Dto3D conversion is how to retrieve the depth information from 2D images/videos which lost in the capture process. Existing 2Dto3D conversion methods can be generally divided into two categories: automatic and semiautomatic ones. The automatic conversion methods rely on different kinds of depth cues to generate depth-maps. Since the relationships between these cues and depth are nonlinear, current automatic methods usually make some global assumptions about the scene. Once the assumptions do not hold, depth errors will appear. The accuracy of depth-maps generated by the automatic methods still can't meet the 3D display demand

Methods

Results

Discussion

Conclusion