EFFICIENT UNSUPERVISED CONTENT-BASED SEGMENTATION IN STEREOSCOPIC VIDEO SEQUENCES

Abstract

This paper presents an efficient technique for unsupervised content-based segmentation in stereoscopic video sequences by appropriately combined different content descriptors in a hierarchical framework. Three main modules are involved in the proposed scheme; extraction of reliable depth information, image partition into color and depth regions and a constrained fusion algorithm of color segments using information derived from the depth map. In the first module, each stereo pair is analyzed and the disparity field and depth map are estimated. Occlusion detection and compensation are also applied for improving the depth map estimation. In the following phase, color and depth regions are created using a novel complexity-reducing multiresolution implementation of the Recursive Shortest Spanning Tree algorithm (M-RSST). While depth segments provide a coarse representation of the image content, color regions describe very accurately object boundaries. For this reason, in the final phase, a new segmentation fusion algorithm is employed which projects color segments onto depth segments. Experimental results are presented which exhibit the efficiency of the proposed scheme as content-based descriptor, even in case of images with complicated visual content.