Depth camera indoor mapping for 3D virtual radio play

Juho‐Pekka Virtanen,Matti T Vaaja,Matti Kurkela,Antero Kukko,Henrik Haggrén,Juha Hyyppä,Arttu Julin,Marika Ahlavuo,Jessica Edén Von Numers,Hannu Hyyppä,Tuomas Turppa

doi:10.1111/phor.12239

Abstract

AbstractBy combining game engines with indoor mapping, it is possible to create interactive virtual environments that represent the real world. In this paper a virtual version of an audio installation in a historic building is produced, where the user freely explores the building and experiences a set of audio clips, creating a virtual radio play. A depth camera indoor mapping system, Matterport, captured a set of staged interiors. The aim was to evaluate the quality and usability of the indoor models and to demonstrate applying them in 3D application development. The quality of the models was evaluated by comparison with laser scanning, revealing limitations with Matterport: increasing the size of the measured area lowered the level of detail and accuracy of volumetric models. The quality of textures was not affected by this limitation, maintaining the appearance of models. To obtain optimised models for mobile 3D applications, a manual revision process was applied.

Highlights

STORYTELLING AND EXPERIENCING NARRATIVES is a really fundamental cultural action, serving social, cognitive, emotional and expressive functions
While narratives are a mainstay in artistic genres, such as literature, theatre and cinema, they have been explored in the context of interactive virtual systems (Roussou, 2001)
The use of virtual environments in simulation relies on the assumption that the virtual environments triggering our senses (Allen and Madden, 1985) are perceived, and responded to, in the same manner as the real world (Arthur et al, 1997)

Summary

Introduction

STORYTELLING AND EXPERIENCING NARRATIVES is a really fundamental cultural action, serving social, cognitive, emotional and expressive functions. While narratives are a mainstay in artistic genres, such as literature, theatre and cinema, they have been explored in the context of interactive virtual systems (Roussou, 2001). Increasing the visual resemblance of the virtual environment should increase the resemblance of the reaction triggered by the virtual experience. The use of a virtual 3D environment as a pedagogical platform and interface between human sensory inputs and behavioural outputs may intensify both mental and physical learning processes (McGann, 2015). It has been found that the presence and quality of a narrative plays a significant role in the depth of the virtual experience (Gorini et al, 2011). The immersion in virtual environments appears to be formed as a combination of sensory stimulation and narrative

Objectives

Methods

Results

Discussion

Conclusion