An optical see-through display for mutual occlusion with a real-time stereovision system

Abstract

Conventional optical see-through displays are not capable of correctly presenting the mutual occlusion of real and virtual environments, since the synthetic objects always appear as translucent ghosts floating in front of the real scene. We have been developing an optical see-through display that is capable of such mutual occlusion. Our display, named ELMO, has both the features that are necessary for mutual occlusion: a selective light-blocking mechanism and a real-time depth sensing mechanism. Firstly, our novel optics cut off incoming light selectively without themselves going out of focus, so virtual objects that should be in front of the real scene are made opaque and sharply occlude the background. This feature also ensures that the virtual objects remain in their intended colors. Secondly, the sensing mechanism, a stereovision system built into ELMO, is used to acquire a depth map of a real scene in real-time, the system can be used in unknown and dynamic environments. As a consequence, ELMO does not require particular environmental settings such as light conditions, and can be used anywhere, e.g., outdoors. We have built prototype displays and confirmed the effectiveness of the approach. In this paper, the basic concept, design of the optics, our prototype displays, and some empirical studies are presented and discussed.