COLF‐GAN: Learning to axial super‐resolve focal stacks

Abstract

In order to generate a denser focal stack, a cooperative generative adversarial network is proposed to learn the refocusing ability from light field imaging. To keep the axial continuity, the proposed framework is designed to learn features of a focal stack in both axial directions. Different from the classic generative adversarial network, our generative module consists of a forward prediction sub-network and a backward prediction sub-network, taking the forward-and-backward focal stacks as their inputs, respectively. The bi-directional predictions are then fused by a weighting process, which is guided by an adversarial module. The proposed network is trained on light field focal stacks conducted via digital refocusing. Without loss of the refocus continuity, one can axial super-resolve a focal stack by using the trained model. The effectiveness of the proposed algorithm on different types of focal stacks produced by both light fields and traditional camera shootings is validated. The experimental results indicate that the refocus variation of a focal stack can be well learned and predicted without a complete light field. Therefore, the proposed algorithm outperforms the traditional digital refocusing in terms of run-time.