Advanced Volumetric Capture and Processing

Abstract

Volumetric video is regarded worldwide as the next important development in media production, especially in the context of rapidly evolving virtual and augmented reality markets where volumetric video is becoming a key technology. Fraunhofer Heinrich Hertz Institute (HHI) has developed a novel technology for volumetric video. The 3D Human Body Reconstruction (3DHBR) technology captures real persons with our novel volumetric capture system and creates naturally moving dynamic 3D models, which can then be observed from arbitrary viewpoints in a virtual or augmented reality scene. The capture system consists of an integrated multicamera and lighting system for a full 360° acquisition. A cylindrical studio has been developed with a diameter of 6 m and consists of 32 20-MPixel cameras and 120 light-emitting-diode (LED) panels that allow for an arbitrary lit background. Hence, diffuse lighting and automatic keying are supported. The avoidance of green screen and provision of diffuse lighting offers the best possible conditions for relighting of the dynamic 3D models afterward at the design stage of the virtual reality (VR) experience. In contrast to classical character animation, facial expressions and moving clothes are reconstructed at high geometrical detail and texture quality. The complete workflow is fully automatic, requires about 12 hr/min of mesh sequence, and provides a high level of quality for immediate integration in virtual scenes. Meanwhile, a second, professional studio has been built up on the film campus of Potsdam Babelsberg. This studio is operated by VoluCap GmbH, a joint venture between Studio Babelsberg, ARRI, UFA, Interlake, and Fraunhofer Heinrich Hertz Institute (HHI).