Dense viewpoint encoding of 3D light fields based on neural graphics primitives

Abstract

The enormous rendering time is a challenge for generating the Projector Image Array (PIA) using parallax images. Based on Neural Radiance Fields (NeRF), a novel method for directly rendering sub-images of PIA is proposed to accelerate light field encoding. The method is a unique combination of 3D light field coding and recent instant Neural Graphics Primitives (NGP) technique, showing how to obtain high-quality PIA quickly. The NGP is used to accelerate the training and rendering of the NeRF, and to improve the quality of the rendered parallax images by utilizing a multi-resolution hash grid container. The light ray of the traditional light field coding method is viewed as the key object linking the display system and the implicit radiance field. By encoding the light field of the actual display system as input to the NGP network, the proposed method can directly obtain the sub-PIA displayed in a single projector. For a projector with a resolution of 720×1280, the sub-PIA rendering time is only 25ms. The PIA is then synthesized from all the sub-PIAs that have been rendered. Numerical simulation results show that this method runs faster than the traditional method with high-quality displays. Light field display experiments are conducted to demonstrate the method's feasibility.