Abstract
Novel view synthesis is a long-standing problem. Despite the rapid development of neural radiance field (nerf), in terms of rendering dynamic human body, NeRF still cannot achieve a good trade-off in precision and efficiency. In this paper, we aim at synthesizing a free-viewpoint video of an arbitrary human performers in an efficient way, only requiring a sparse number of camera views as inputs and skirting per-case fine-tuning. Recently, several works have addressed this problem by learning person-specific neural radiance fields (NeRF) to capture the appearance of a particular human. In parallel, some work proposed to use pixel-aligned features to generalize radiance fields to arbitrary new scenes and objects. Adopting these generalization approchs to human achieve reasonable rendering result. However, due to the difficulties of modeling the complex appearance of human and the dynamic sense, it is challenging to train nerf well in an efficient way. We find that the slow convergence of the human body reconstruction model is largely due to the nerf representation. In this work, we introduce a voxel grid based representation for human view synthesis, termed Voxel Grid Performer(VGP). Specifically, a sparse voxel grid is designed to represent the density and color in every space voxel, which enable better performance and less computation than conventional nerf optimization. We perform extensive experiments on both seen human performer and unseen human performer, demonstrating that our approach surpasses nerf-based methods on a wide variety of metrics. Code and data will be made available at https://github.com/fanzhongyi/vgp.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.