Efficient and Iterative Training for High-Performance Light Field Synthesis

Abstract

Light field displays are considered a promising technique for future augmented reality (AR) displays since they provide a fully natural visual experience by reproducing light rays in space. However, the acquisition of a high-resolution light field is a great challenge. In this paper, we propose a high-performance, efficient, and iterative training framework that helps synthesize a light field from a pair of stereo images. Our iterative training framework combines the advantage of different input image disparity measures and performs favorably against state-of-the-art algorithms for light field synthesis from extremely sparse (only one, two, or four) views on real and synthetic light field datasets. Our model structure consists of a convolutional neural network (CNN) that enforces a left-right consistency constraint on the light fields synthesized from left and right stereo views, a stage that merges light fields synthesized from left and right stereo views with a novel alpha blending technique, and a final refinement network using a unique 3D convolution operation. Our method also speeds up the process of light field synthesis, realizing real-time display of light fields for AR.