Anchoring Graph Cuts Towards Accurate Depth Estimation in Integral Images

Abstract

Integral imaging is a three-dimensional (3D) imaging technique that allows the displaying of full color images with continuous parallax. Its commercial potential has been increased, due to its ability of presenting to the viewers smooth 3D images, with full parallax, in a wide viewing zone. Being able to extract the inherent 3D information from the planar integral images and produce 3D reconstructions, offers advantages in various applications of immersive entertainment and communications. On this scope, this paper addresses the problem of accurate depth estimation in integral images. The proposed method, relying on the assumption that a pixel is the projection of a 3D imaging ray, aims to specify the first intersection of each pixel's projection ray with the 3D scene in order to assign to it the corresponding depth value. This task is formulated as an energy optimization problem and the graph cuts approach is utilized to solve it. The energy term is twofold; its first part aims to restrict the desired solution to be close to the observed data, i.e., the integral image, while the second one enforces regional smoothness in the depth estimation. This combination offers an accurate and spatially smooth scene structure. The novelty of the paper lies on the framework's formulation as one single optimization procedure and on the way that this optimization is constrained by a set of reliably estimated 3D surface points, called the “anchor points”. Anchoring the optimization results in enhanced depth estimation accuracy, while decreasing the optimization processing burden. The proposed algorithm is evaluated in both synthetic and real integral images consisting of complicated object scenes. A comparison against other state-of-the-art algorithms proves the superiority of the proposed method in terms of depth estimation accuracy.