Implementation, Measurement, and Analysis of an Image-Based Virtual Environment Streaming Protocol for Wireless Mobile Devices

Abstract

Remote exploration of detailed virtual environments (VEs) on mobile devices with acceptable frame rates has become a challenging and interesting research topic. With recent advances in mobile computing devices with communication and multimedia capabilities, remote VE walk-through and real-time streaming for mobile devices have opened a new class of 3-D VE exploration-based applications such as virtual guides and malls, online gaming, training, and monitoring, just to name a few. Furthermore, emergency preparedness types of applications can benefit from mobile 3-D visualization systems to support emergency responders with critical information about the disaster area. The combination of sensor networks with VEs can provide a physical environment monitoring and measurement system with live information about the target environment that can be visualized in real time. The challenge lies in how to provide a rich and detailed 3-D VE on thin mobile devices that are known for their lack of proper resources to process large-scale 3-D geometric data. Our approach consists of moving the demanding geometry-rendering task to a dedicated remote rendering server that streams the rendering output to a client, leaving only the displaying and certain minor image-based rendering tasks to the local, less-powerful mobile hardware. In addition, wireless networks pose significant challenges to multimedia streaming due, for the most part, to the mobility-induced changes in bandwidth and errors in the wireless channel. In this paper, we focus on the implementation, measurement, and analysis of an image-based VE streaming strategy. We define a new approach to remote rendering and interactive visualization of 3-D VEs on thin mobile devices. To this end, we propose end-to-end streaming and rate control protocols, as well as buffering and scheduling mechanisms to support the requirements of bandwidth-demanding multimedia systems. The main purpose of our proposed rate control scheme is to achieve both high end-to-end throughput and low frame-rate fluctuation to adapt data traffic to the frequent changes of the bandwidth and error rate present in wireless networks. We discuss the design of our proposed solutions and report on their performance evaluation through an extensive set of simulation experiments.