Enhanced Light Field Reconstruction by Combining Disparity and Texture Information in PSVs via Disparity-Guided Fusion

Abstract

Dense Light Fields (LFs) can be reconstructed based on a few sparse views. One popular pipeline is the explicit-depth-based pipeline, which first estimates disparity maps using the existing sparse views and then produces the object texture of the target views by warping the relevant information from the given sparse views. However, due to errors in disparity estimation, such a solution often loses object details. Such problems, particularly in regions of small disparity, can be sorted out by estimating the texture without disparity estimation using the so-called implicit-depth-based pipeline. This work integrates both pipelines. It will adapt to the disparity status in the local region and automatically adjust the LFs produced by both pipelines through disparity-guided fusion. In addition, plane-sweep volumes (PSVs) are exploited for handling the given sparse view information, which can reliably decouple the disparity and texture information into different dimensions. In this way, it can best supply the relevant information to the explicit-depth-based and implicit-depth-based pipelines, respectively. The proposed solution shows superiority in various LF reconstruction tasks in our experiments.