Multi-layered see-through movie in diminished reality

Abstract

This paper presents generating a multi-layered see-through movie for an auto-stereoscopic display. This work is based on Diminished Reality (DR), which is one of the research fields of Augmented Reality (AR). In the usual AR, some virtual objects are added on the real world. On the other hand, DR removes some real objects from the real world. Therefore, the background is visualized instead of the real objects (obstacles) to be removed. We use multiple color cameras and one TOF depth camera. The areas of obstacles are defined by using the depth camera based on the distance of obstacles. The background behind the obstacles is recovered by planarprojection of multiple cameras. Then, the recovered background is overlaid onto the removed obstacles. For visualizing it through the auto-stereoscopic display, the scene is divided into multiple layers such as obstacles and background. The pixels corresponding to the obstacles are not visualized or visualized semi-transparently at the center viewpoints. Therefore, we can see that the obstacles are diminished according to the viewpoints.