A framework for cost-effective communication system for 3D data streaming and real-time 3D reconstruction

Abstract

The methods commonly used for people to converse despite a large geographical distance are either text messaging or video conference. A more immersive communication method which is similar to in-person conversation is more desirable. The closest form is a conversation via the holographic projection of the participants and environment. Two features needed to achieve this include 3D reconstruction of the target and streaming of 3D data. We implement 3D reconstruction and 3D streaming using cost-effective depth cameras. The focus of this work is on how to approach the idea by using such devices to create a standardized platform for the implementation of the system with the features mentioned above. Specifically, the system can capture 3D data from multiple depth sensors, reconstruct a 3D model of the human target to create an avatar, and stream the changes acquired from the sensors to the client to control the avatar in real time.