Clear-Plenoxels: Floaters free radiance fields without neural networks

Abstract

Listen

Translate article

Directly optimizing radiance fields with explicit 3D primitives, such as voxels and 3D Gaussians, demonstrates impressive effectiveness in both training and rendering. However, optimizing explicit 3D structures from limited training views using differentiable volumetric rendering often results in ghosting artifacts (floaters), i.e., it suffers from early-stage errors in unseen space and remains as evident ghosting artifacts as gradient becomes very sparse spatially with optimization procedure processes. Thus, to address this problem, this paper proposes Clear-Plenoxels, which involves three simple yet non-trivial components: 1) a Visual Hull-based initialization strategy for maximizing the utilization of object resolution and effectively rejecting false updates during early-stage optimization by learning rate assignment per voxel grid; 2) an effective penalty function on local grid discrepancy for ensuring consistent rendering at different view directions, which can remove unnecessary voxels; 3) a mask guided transmittance supervision for each training ray, significantly guaranteeing depth prediction precision when the object's surface color is close to the background. Experiments on a public dataset demonstrate that our method successfully overcomes ghosting artifacts by directly optimizing explicit primitive strategy. The proposed method achieves approximately 1 dB improvement in voxel grid-based methods and matches, if not surpasses, state-of-the-art quality regarding novel view rendering. The code and visualization videos are publicly available at https://github.com/nortonBryan/Clear-plenoxels.