Depth map estimation with 4D light fields using confocal stereo

Abstract

We present a scene depth map generation method based on light field cameras. From the plenoptic function, the angular information about each image point under different sizes of aperture is extracted, which could be used for confocal stereo. Considering confocal constancy and gradient constancy, we take into account two constraints: (1) When a pixel is in focus, its relative intensities across aperture should match the variation predicted by the relative exitance of the lens; and (2) When a pixel is in focus, the gradient of the pixel should equal to that of the corresponding pixel in reference image. Based on these two constraints, we develop data term which measures the probability of each pixel in each depth. Considering the textureless area, we also develop smoothness term which helps to determine the depth of textureless area by its neighboring texture area. Finally, the depth map is estimated via multi-label optimization and weighted median filtering.