Exploration with live stereoscopic 3D video in mixed reality environments

Abstract

This paper describes an integrated system for the real time acquisition, streaming, and display of stereoscopic 3D video in mixed reality environments. These mixed reality environments combine real-time stereoscopic video with rendered 3D environments composed of sampled and modeled 3D data from actual landscapes or structures. While live video itself is an important part of exploration, reconnaissance, and documentation, the ability to overlay this video with existing models can significantly add to the context for analysis and decision making. We describe the components of a high- resolution stereoscopic video streaming system integrated with a virtual reality visualization system viewed on high-resolution stereoscopic 3D display walls. The 3D display wall provides a field of view for visualization which is able to exceed the field of view of the human visual system, allowing for a more immersive and natural experiences, and far extending the visual canvas provided by a stereoscopic video stream. We demonstrate a live stereoscopic video feed streamed to a visualization wall and mixed in real-time with a virtual model of that location and present several different usage scenarios exploring how this new visualization technique would be beneficial.