Point-based rendering enhancement via deep learning

Abstract

Current state-of-the-art point rendering techniques such as splat rendering generally require very high-resolution point clouds in order to create high-quality photo realistic renderings. These can be very time consuming to acquire and oftentimes also require high-end expensive scanners. This paper proposes a novel deep learning-based approach that can generate high-resolution photo realistic point renderings from low-resolution point clouds. More specifically, we propose to use co-registered high-quality photographs as the ground truth data to train the deep neural network for point-based rendering. The proposed method can generate high-quality point rendering images very efficiently and can be used for interactive navigation of large-scale 3D scenes as well as image-based localization. Extensive quantitative evaluations on both synthetic and real datasets show that the proposed method outperforms state-of-the-art methods.