A Hybrid Approach to Wide Baseline View Synthesis with Convolutional Neural Networks

Abstract

Convolutional Neural Networks (CNN) have been recently employed for implementing complete end-to-end view synthesis architectures, from reference view warping to target view blending while dealing with occlusions as well. However, the convolutional sizes filters must increase with the distance between reference views, making all-convolutional approaches prohibitively complex for wide baseline setups. In this work we propose a hybrid approach to view synthesis where we first warp the reference views resolving the occlusions, and then we train a simpler convolutional architecture for blending the preprocessed views. By warping the reference views, we reduce the equivalent distance between reference views, allowing the use of smaller convolutional filters and thus lower network complexity. We experimentally show that our method performs favorably against both traditional and convolutional synthesis methods while retaining lower complexity with respect to the latter.