Toward Hyper-Realistic and Interactive Social VR Experiences in Live TV Scenarios

Abstract

Social Virtual Reality (VR) allows multiple distributed users getting together in shared virtual environments to socially interact and collaborate. This article explores the applicability and potential of Social VR in the broadcast sector, focusing on a live TV show use case, by providing three main contributions: 1) a novel and lightweight social VR platform; 2) a professional piece of VR content to recreate an interactive live TV show; and 3) an analysis of the performance and user experience. The Social VR platform includes different innovative and outstanding features compared to state-of-the-art solutions. It allows a real-time integration of remote users in shared virtual environments, using realistic volumetric representations and affordable capturing systems, thus not relying on the use of synthetic avatars. It supports a seamless and rich integration of heterogeneous media formats, including 3D scenarios, dynamic volumetric representation of users and (live/stored) stereoscopic 2D and 180°/360° videos. In addition, it enables low-latency interaction between volumetric users and a video-based presenter (Chroma keying) and a dynamic control of the media playout to adapt to the session’s evolution. The article also describes the production process of an immersive an interactive TV show to demonstrate the platform’s capabilities and its potential benefits. On the one hand, the results from objective tests show the satisfactory performance of the platform. On the other hand, the promising results from user tests support the potential impact of the presented platform, opening up new opportunities in the broadcast sector.