XSlit cameras for free navigation with depth image-based rendering

Abstract

In free navigation applications, any viewpoint to a three-dimensional scene can be synthesized through Depth Image-Based Rendering (DIBR). In this paper we show that XSlit cameras achieve up to 3 dB PSNR gain over conventional pinhole camera arrays when synthesizing virtual views to the scene with DIBR for small camera displacements. XSlit cameras are a type of general linear cameras where the light rays pass through two non-intersecting slits instead of a single point (the optical center of conventional cameras), resulting in a different epipolar geometry and projection equations. Instead of synthesizing virtual views with DIBR from a set of conventional pinhole cameras, e.g. a stereo camera pair, a single XSlit camera exploits the distance between its slits and their relative rotation to obtain disparity, out of which DIBR virtual views can be synthesized. We first present a theoretical study to present the parameters of XSlit cameras, different from pinhole cameras, at the same time making sure that the XSlit camera is physically implementable. We then validate the study with DIBR achieved on synthetic content using perfect depth maps obtained from an in-house modified version of Blender’s engine Cycles to simulate XSlit cameras. The virtual view synthesis uses an adapted version of the Reference View Synthesis software used in MPEG, the worldwide standardization committee for media compression. Our experiments show that for the same overall space covered by the studied camera architectures, XSlit cameras often obtain better DIBR view synthesis results, with up to 3 dB PSNR gain.