Object-based coding for Kinect depth and color videos

Abstract

Simultaneously capturing of color and depth videos, e.g. with Kinect, favors many applications and has become very popular. Efficient representation and compression of such data is important yet challenging. In this paper, we have designed an object-based coding system to compress Kinect-like depth and color videos. Segmentation is first conducted to obtain different object planes, where a mask image is utilized to identify them. We compress depth and color images respectively using the proposed object-based coding codec, which is designed based on High Efficiency Video Coding (HEVC). The mask image is losslessly compressed by adding a new context-based mode to HEVC. To assure the alignment of object boundaries on the depth image and those on the color image, a pre-processing is conducted over the depth image. The separate coding of the different object planes for the depth image can avoid the inefficiency coding of edges blocks at object boundaries and thus bring obvious coding gain. Moreover, the attractive functionality of “content-based” coding which permits the transmission of the interested object planes rather than an entire image provides a practical way to decrease the bitrate.