A hierarchical approach to scene segmentation

Abstract

A scene is an important semantic unit to a user who is accessing video data and is useful in semantic-based video indexing and retrieval. However, scene segmentation from video data is not an easy task. We propose a novel hierarchical scene segmentation method which consists of three steps: initial segmentation, refinement, and adjustment. First, we detect initial scene boundaries using a continuous coherence computing model. As a coherence computing model, we use a short-term memory-based model in which the memory size and attention span size are not fixed. In our model, there are times when the removal of shots from the memory buffer may not follow a First-In-First-Out rule. After detection of initial scene boundaries, we execute the refinement process for each scene using k-means clustering algorithm to find misses not detected in scene boundary detection. The cluster-validity analysis technique is used to find the optimal number of clusters. If a new scene boundary is found within a scene by the analysis of clustering results, the scene is split into two sub-scenes. After that, we execute the adjustment process. We choose two consecutive scenes to detect whether the scene boundary is false or not. For shots of two consecutive scenes, we also perform a k-means clustering algorithm and detect desirable scene boundaries. Finally, accurate scene segmentation results are obtained and experimental results are presented.