Towards multi-view consistency in neural ray fields using parametric medial surfaces

Abstract

Deep learning methods are revolutionizing the solutions to visual computing problems, such as shape retrieval and generative shape modeling, but require novel shape representations that are both fast and differentiable. Neural ray fields and their improved rendering performance are promising in this regard, but struggle with a reduced fidelity and multi-view consistency when compared to the more studied coordinate-based methods which, however, are slower in training and evaluation. We propose PMARF, an improved ray field which explicitly models the skeleton of the target shape as a set of (0-thickness) parametric medial surfaces. This formulation reduces by construction the degrees-of-freedom available in the reconstruction domain, improving multi-view consistency even from sparse training views. This in turn improves fidelity while facilitating a reduction in the network size.