Separation of foreground and background from light field using gradient information.

Abstract

Studies of computer vision or machine vision applications using a light field camera have been increasing in recent years. However, the abilities that the light field camera has are not fully used in these applications. In this paper, we propose a method for direct separation of foreground and background that uses the gradient information and can be used in various applications such as pre-processing. From an optical phenomenon whereby the bundles of rays from the background are flipped, we derive that the disparity sign of the background in the captured three-dimensional scene has the opposite disparity sign of the foreground. Using the majority-weighted voting algorithm based on the gradient information with the Lambertian assumption and the gradient constraint, the foreground and background can be separated at each pixel. In regard to pre-processing, the proposed method can be used for various applications such as occlusion and saliency detection, disparity estimation, and so on. Experimental results with the EPFL light field dataset and Stanford Lytro light field dataset show that the proposed method achieves better performance in terms of the occlusion detection, and thus can be effectively used in pre-processing for saliency detection and disparity estimation.