Performance of NeRF and its optimized models on generating images from 2D to 3D of animation or game character

Abstract

Neural Radiance Fields (NeRF) has recently emerged as a promising approach for single-image 3D scene reconstruction. Despite this, most studies on NeRF have focused on real-world data or items with established camera projection connections, rather than snapshots of video game characters that may not have a distinct camera projection connection. Although some research has been conducted on the use of animatable mannequins, the majority of these studies have used photographs of real people as their data source. To date, there has been limited research into the use of NeRF for animation or game characters. In this paper, we compare and evaluate a range of NeRF models on a dataset of animated game characters. The models considered include original NeRF, Block-NeRF, Mip-NeRF, and Pixel-NeRF, each of which has its own design choices and trade-offs. Our results show that Mip-NeRF models are effective at generating high-quality images of game characters on our dataset, while Pixel-NeRF performs poorly on our dataset. Despite these findings, our research demonstrates the feasibility of using NeRF-based models to create realistic character models or animations for video games, with potential applications in game development and virtual reality.