Real-Time 3D Reconstruction for Mixed Reality Telepresence Using Multiple Depth Sensors

Abstract

AbstractIn recent years, there has been a great development in real-time three-dimensional (3D) scene reconstruction from depth sensor data, as well as the study of such data in Virtual Reality (VR) and Augmented Reality (AR) contexts. Although it has been extensively investigated and has attracted the attention of many researchers, the challenge of real-time 3D reconstruction remains a difficult research task. The majority of current techniques, target real-time 3D reconstruction for the single-view-based system rather than multi-view. In order to provide multi-view 3D reconstruction for Mixed Reality (MR) telepresence, this chapter aims to propose a multiple depth sensor capture using a marching square approach to produce a single full 3D reconstruction surface of a moving user in real-time. The chapter explains the design stage that involves setup from multiple depth sensors, surface reconstruction and merging of 3D reconstruction data for MR Telepresence. The chapter ends with results and a conclusion.KeywordsReal-time3D reconstructionMR telepresenceDepth sensor