Occlusion-aware light field depth estimation with view attention

Abstract

Depth estimation for light field images is crucial in light field applications such as image-based rendering and refocusing. Previous learning-based methods com bining neural network with cost volume can achieve accurate depth estimation but fail in regions with occlusion. In this paper, a two-stage attention-based occlusion-aware light field depth estimation network is proposed. In the initial depth estimation stage, the sub-aperture images are divided into four groups based on different view directions and four initial cost volumes are constructed using the feature maps from each group to aggregate initial depth maps. Then in the refined depth estimation stage, the four aggregated volumes from the initial stage are fused into one based on the view attention, where features of views with less occlusion are highly weighted to provide more effective information. Experiment results demonstrate that the proposed method can accomplish robust and accurate depth estimation in the presence of occlusion, which ranks the first place on 4D light field benchmark in terms of most accuracy metrics.