Depth of Field Image Sequences: 3D Cuing of High Efficiency

Abstract

We study the ensemble of depth of field (DOF) images pertaining to continuously varying focal distance but with the position, angle and aperture of the camera fixed, called the DOF image sequence. It is shown that all member images of the ensemble can be approximated with good precision as a linear combination of few basis images. By exploiting the above newly discovered sparsity structure of DOF images, we develop a new coding scheme for DOF image sequences. The encoder works as a DOF image modeler; reciprocally the decoder acts as an ultra fast DOF image renderer. This coding scheme enables real-time generation of DOF videos that achieve realistic 3D perceptions via combined use of motion parallax and depth of field. The proposed new technique outperforms the image-based DOF rendering in image quality while having a lower complexity. The same sparsity of DOF images also inspires our design of a new computational display system that can offer multiview DOF video presentations to different users all on a common screen. Experimental results show practical values of this research in multiuser VR applications, in both aspects of content generation and presentation.