VConnect: perceive and interact with real world from CAVE

Abstract

The Cave Automatic Virtual Environment (CAVE) is a fully immersive Virtual Reality (VR) system. CAVE systems have been widely used in many applications, such as architectural and industrial design, medical training and surgery plan, museums and education. However, one limitation for most of the current CAVE systems is that they are separated from the real world. The user in the CAVE is not able to sense the real world around him or her. In this paper, we propose a vConnect architecture, which aims to establish real-time bidirectional information exchange between the virtual world and the real world by utilizing the advanced technologies in cloud computing, mobile communications, wireless sensor networks, and computer vision. Specifically, we address three technical challenges in the proposed vConnect architecture. First, we propose an optimal allocation scheme for the wireless sensor networks to ensure that the data streams captured by the sensors can be delivered to the cloud servers in a reliable and prompt way. Second, we optimize the allocation of the cloud resources to ensure that the data streams sent from the clients can be processed promptly by the cloud servers at a minimal resource cost. Third, we propose to use marker-based finger interactions such that the user in the CAVE can manipulate the information in a natural and intuitive way. Fourth, we implemented a vHealth prototype, a CAVE-based real-time health monitoring system, to validate the proposed vConnect architecture. We demonstrated in the vHealth prototype that the user in the CAVE can visualize and manipulate the real-time physiological data of the patient who is being monitored, and interact with the patient.