3D reconstruction based on light field information

Abstract

As an important branch of computational photography, light field photography combines the hardware design of optical system with key algorithm of signal processing quite well. Unlike traditional photography which can only record light ray's two-dimensional position, light field photography system can record four-dimensional position and direction. Therefore, much more image information can be obtained from light field photography. With the development of 3D display technology, light field based autofocus and 3D display technology is becoming more and more popular. In this paper, a light field based new 3D reconstruction algorithm for buildings and office environment is proposed by applying Wavelet Transform and SVM (Support Vector Machine) model to obtain the image focusing quality assessment, along with the Mean Shift Algorithm and Random Field Model to get the depth map of the scene. Firstly, light field image is captured by using a light field camera. Secondly, we use frequency domain digital refocus algorithm to manipulate light field image and obtain several serialized refocused images with different focus. Thirdly, wavelet features are extracted from each refocused image, and then an image focusing quality assessment is conducted by using RBF (Radial Basis Function) kernel based SVM model. Finally, we use Mean Shift algorithm to realize color clustering of the original light field image, and then build MRF (Markov Random Field) Model with color nodes. By iterating the likelihood depth result obtained from real scenario depth calibrations according to image focusing quality assessment, finally the depth map of the scene is reconstructed. Experiments are conducted to prove the feasibility of the proposed 3D reconstructed algorithm based on light field. And the experimental results on real datasets demonstrate good performance of this algorithm.