Unconstrained 2D to Stereoscopic 3D Image and Video Conversion Using Semi-Automatic Energy Minimization Techniques

Abstract

We present a method for semi-automatically converting unconstrained 2D images and video content into stereoscopic 3D. The user is presented with the image to convert, and brushes user-defined depth strokes in certain areas. These correspond to a rough estimate of the scene depths within these points. After, the rest of the depths are solved using this information, producing a depth map to create stereoscopic 3D content. For video, the user chooses several keyframes for brushing, and the depths for the entire video are found in a volumetric basis. Additionally for video, the user has the option of minimizing effort by employing a robust tracking algorithm, where the first frame only needs to be labeled. After, the labels are propagated throughout the entire video, ultimately increasing accuracy with more frames labeled. Our work combines the merits of two energy minimization techniques: Graph Cuts and Random Walks. The former respects boundaries, but does not have suitable depth diffusion, making the scene look like “cardboard cutouts”. The latter has good depth diffusion, but object boundaries are blurred. Therefore, combining the merits of both will lead to a higher quality result. Current efforts rely on automatic or manual conversion by rotoscopers. The former prohibits user intervention, while the latter is time consuming, prohibiting use in smaller studios. Semi-automatic is a compromise to allow for more faster and accurate conversion, decreasing the time for studios to release 3D content. The results shown in this paper generate good quality stereoscopic depth maps with minimal effort required.