Immersive Integration for Virtual and Human-Centered Environments

Abstract

Summary form only given. We envision future work and play environments that are more effectively human-centered with the user's computing interface being more closely integrated with the physical surroundings than today's conventional computer display screens and keyboards. We are working toward realizable versions of such environments, in which multiple video projectors and digital cameras enable every visible surface to be both measured in 3D and used for display. If the 3D surface positions are transmitted to a distant location, they may also enable distant collaborations to become more like working in adjacent offices connected by large windows. In one prototype, depth maps are calculated from streams of video images and the resulting 3D surface points are displayed to the user in head-tracked stereo. Another prototype allows direct painting onto movable objects - a dollhouse, for example. One long-term goal is advanced training for trauma surgeons by immersive replay of recorded procedures. More generally, we hope to demonstrate that the principal interface of a future computing environment need not be limited to a screen the size of one or two sheets of paper. Just as a useful physical environment is all around us, so too can the increasingly ubiquitous computing environment be all around us - becoming more effectively human-centered and integrated into our physical surroundings.