Dense view interpolation of 4D light fields for real-time augmented reality applications

Abstract

Dense multi-view 3D displays appropriately combined with camera arrays may provide consistently augmented reality, where we enjoy a simple solution of occlusion caused by visual obstructions in the real world as if they were transparent for many users at the same time. In this paper, toward such multi-view systems electrically transmitting light fields beyond visual obstructions, we propose a novel method for real-time dense interpolation of entire 4D light fields acquired by sparse viewpoints. First, we discuss appearances of our proposed multiview system, while examining how dense 4D light fields should be. Then, we propose a novel method of real-time 4D light field interpolation to develop an inexpensive system by reducing the number of image sensors to acquire 4D light fields. By introducing simple depth estimation from equally-spaced multiview images without severe increase of computational cost, the novel method significantly improves reconstruction quality of the previously proposed one, that can only work for almost constant depths. Finally, experimental results show that our practical implementation on a GPGPU achieves real-time interpolation of entire 4D light fields while obtaining sufficient quality of reconstructed images. It is even comparable to that of various conventional methods for view interpolation, which depend on high computational cost not suitable for our real-time applications.