Lessons Learned During One year of Commercial Volumetric Video Production

Abstract

In June 2018, Fraunhofer Heinrich Hertz Institute (HHI) together with Studio Babelsberg, ARRI, Universum Film AG (UFA), and Interlake founded the joint venture Volucap GmbH and opened a commercial volumetric video studio on the film campus of Potsdam-Babelsberg. After a testing phase, commercial productions went live in November 2018. The core technology for volumetric video production is 3D human body reconstruction (3DHBR), developed by Fraunhofer HHI. This technology captures real persons with our novel volumetric capture system and creates naturally moving dynamic 3D models, which can then be observed from arbitrary viewpoints in a virtual or augmented environment. Thanks to the large number of test productions and new requirements from customers, several lessons have been learned during the first year of commercial activity. The processing workflow for capture and production of volumetric video has been continuously evolved such as process-dependent color grading and a new encoding scheme for volumetric video. Furthermore, specific challenges have been tackled, for instance, fast movements and relighting of actors .