TVNeRF: Improving few-view neural volume rendering with total variation maximization

Abstract

Neural Radiance Fields (NeRF) has profoundly impacted few-shot novel view synthesis and 3D reconstruction with its innovative rendering techniques and high-quality output. However, synthesizing novel views from sparse inputs remains challenging for NeRF. With limited view inputs, NeRF tends to overfit the available views, leading to reconstruction discrepancies and artifacts. To address this issue, we propose TVNeRF, which introduces two complementary regularization methods. By maximizing the total variation of unseen rays, we control the density distribution of ray sampled points to prevent excessive sampling in non-critical areas, thereby reducing artifacts and floaters. Concurrently, minimizing the accumulated opacity of unseen rays allows for the effective selection of important rays and sampled points without additional computational overhead, while also mitigating potential over-regularization effects. These combined strategies not only prevent overfitting but also achieve effects similar to the Least Absolute Shrinkage and Selection Operator (LASSO) regularization. Moreover, TVNeRF achieves these improvements without the need for pre-trained models or prior knowledge. By utilizing these straightforward yet effective regularization techniques, TVNeRF can deliver results comparable to state-of-the-art models while maintaining computational efficiency. This approach highlights that substantial improvements in NeRF performance can be realized through targeted regularization methods, effectively addressing the challenges posed by sparse view synthesis.