DirectVoxGO++: Grid-based fast object reconstruction using radiance fields

Abstract

In recent years, the computer vision and computer graphics communities have seen the emergence of new classes of models for representing objects based on Neural Radiance Fields (NeRFs). These techniques use ideas based on traditional methods from computer graphics, for example, radiance fields, and combine them with implicit neural representations and neural image-based rendering (IBR). However, one significant drawback of this class of techniques was that they were too slow, taking in the range of hours in high-end GPUs. Due to these limitations, new techniques have been created for the fast reconstruction of scenes, such as Direct Voxel Grid Optimization (DirectVoxGO). Alongside this limitation, initially, NeRFs had limited compositing and modeling capabilities. For example, a simple task such as separating the background from the foreground could not be modeled with NeRFs until the emergence of new techniques such as NeRF++. Our method extends DirectVoxGO to allow the handling of unbounded scenes inspired by some ideas from NeRF++ and also adapts it to incorporate elements from a neural hashing approach employed by other works. Our technique improved photorealism compared with DirectVoxGO and Plenoxels on a subset of the Light Field Dataset on average in at least 2%, 8%, and 8% for Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index Measure (SSIM), and Learned Perceptual Image Patch Similarity (LPIPS) metrics, respectively, while also being an order of magnitude faster than NeRF++. Also, we demonstrate that, for the evaluated scenes, our technique has comparable training time and memory consumption than previous works. Code is available in https://github.com/danperazzo/dvgoplusplus.