Mesh-based depth map compression and transmission for real-time view-based rendering

Abstract

Enabling telepresence using depth-based new view rendering requires the compression and transmission of dynamic depth maps and video from multiple cameras. The telepresence application places additional requirements on the compressed representation, such as preservation of depth discontinuities, low complexity decoding, and amenability to real-time rendering using graphics cards. We propose a simplified triangular mesh representation that can be encoded efficiently. The mesh geometry is encoded using a binary triangle tree structure and the depths at the tree nodes are differentially coded. By matching the tree traversal to the mesh rendering order, both depth map decoding and triangle strip generation for efficient rendering are achieved simultaneously. The proposed scheme naturally lends itself to coding segmented foreground layers. Our experiments show a significant improvement in rendering speeds at similar compression rates using the new mesh based depth map representation when compared to independent representations for compression and rendering.