Depth map compression for real-time view-based rendering

Abstract

Realistic and interactive telepresence has been a hot research topic in recent years. Enabling telepresence using depth-based new view rendering requires the compression and transmission of video as well as dynamic depth maps from multiple cameras. The telepresence application places additional requirements on the compressed representation of depth maps, such as preservation of depth discontinuities, low complexity decoding, and amenability to real-time rendering using graphics cards. We propose an adaptation of an existing triangular mesh generation method for depth representation that can be encoded efficiently. The mesh geometry is encoded using a binary tree structure where single bit enabled flags that mark the split of triangles and the depth values at the tree nodes are differentially coded. By matching the tree traversal to the mesh rendering order, both depth map decoding and triangle strip generation for efficient rendering are achieved simultaneously. The proposed scheme also naturally lends itself to coding segmented foreground layers and providing error resilience. At similar compression ratio, new view generation using the proposed method provided similar quality as depth compression using JPEG2000. However, the new mesh based depth map representation and compression method showed a significant improvement in rendering speed when compared to using separate compression and rendering processes.