A depth map representation for real-time transmission and view-based rendering of a dynamic 3D scene

Abstract

View-based 3D video streaming requires the compression and transmission of depth maps along with the video sequence. The requirements for representing these depth maps include moderately high compression, preservation of depth discontinuities, low complexity decoding, and to be in a form that is suitable for real-time rendering using graphics cards. This motivates a mesh-based representation for the depth map. We adapt a mesh generation algorithm used in real-time height field rendering for depth map representation and produce a regular mesh for each depth map. Then we propose a compression technique that encodes the triangulated depth map as a set of single bit enabled flags that mark the split of triangles, and depth values at the vertices with Huffman code. By coding the vertices in the rendering order, we achieve depth map decoding and striped triangles for OpenGL based rendering in one loop, thus greatly reducing the complexity at the rendering end.