Fast radiance field reconstruction from sparse inputs

Abstract

Neural Radiance Field (NeRF) has emerged as a powerful method in data-driven 3D reconstruction because of its simplicity and state-of-the-art performance. However, NeRF requires densely captured calibrated images and lengthy training and rendering time to realize high-resolution reconstruction. Thus, we propose a fast radiance field reconstruction method from a sparse set of images with silhouettes. Our approach integrates NeRF with Shape from Silhouette, a traditional 3D reconstruction method that uses silhouette information to fit the shape of an object. To combine NeRF’s implicit representation with Shape from Silhouette’s explicit representation, we propose a novel explicit–implicit radiance field representation consisting of voxel grids with confidence and feature embedding for geometry and a multilayer perceptron network to decode view-dependent color emission for appearance. We propose to make the reconstructed geometry compact by taking advantage of silhouette images, which can avoid the majority of artifacts in sparse input scenarios and speed up training and rendering. We also apply voxel dilating and pruning to refine the geometry prediction. In addition, we impose a total variation regularization on our model to encourage a smooth radiance field. Experiments on the DTU and the NeRF-Synthetic datasets show that our algorithm surpasses the existing baselines in terms of efficiency and accuracy.