RDTC Optimized Streaming for Remote Browsing in Image-Based Scene Representations

Abstract

Remote navigation in compressed image-based scene representations requires random access to arbitrary parts of the reference image data to recompose virtual views. The degree of inter-frame dependencies exploited during compression has an impact on the effort needed to access reference images and delimits the rate distortion (RD) trade-off that can be achieved. If, additionally, a given receiver hardware and a maximum available channel bitrate is taken into account, the traditional rate-distortion optimization is extended to an RDTC trade-off between rate (R), distortion (D), transmission data rate (T), and decoding complexity (C). In this work we introduce our RDTC optimized compression framework. In addition, an experimental testbed for streaming of these RDTC optimally compressed image-based scene representations is described and the impact of client side caching is investigated. Our results show a significant reduction in user perceived delay for RDTC optimized streams in such a remote browsing environment compared to RD optimized or independently encoded scene representations.