Fuzzy segmentation of video shots using hybrid color spaces and motion information

Abstract

Video segmentation can be defined as the process of partitioning video into spatio-temporal objects that are homogeneous in some feature space, with the choice of features being very important to the success of the segmentation process. Fuzzy segmentation is a semi-automatic region-growing segmentation algorithm that assigns to each element in an image a grade of membership in an object. In this paper, we propose an extension of the multi-object fuzzy segmentation algorithm to segment pre-acquired color video shots. The color features are selected from channels belonging to different color models using two different heuristics: one that uses the correlations between the color channels to maximize the amount of information used in the segmentation process, and one that chooses the color channels based on the separation of the clusters formed by the seed spels for all possible color spaces. Motion information is also incorporated into the segmentation process by making use of dense optical flow maps. We performed experiments on synthetic videos, with and without noise, as well as on some real videos. The experiments show promising results, with the segmentations of real videos produced using hybrid color spaces being more accurate than the ones produced using three other color models. We also show that our method compares favorably to a state-of-the art video segmentation algorithm.