From Low-Level Features to Semantic Classes: Spatial and Temporal Descriptors for Video Indexing

Abstract

As the quantity of publicly available multimedia material becomes larger and larger, automatic indexing becomes increasingly important in accessing multimedia databases. In this paper, a novel set of low-level descriptors is presented for the aim of content-based video classification. Concerning temporal features, we use a modified PMES descriptor for the spatial distribution of local motion and a Dominant Direction Histogram we have developed to represent the temporal distribution of camera motion. Concerning color, we present the Weighted Color Histogram we have designed in order to model color distribution. The histogram models the H parameter of the HSV color space, and we combine it with weighted means for the S and V parameters. For the selection of key-frames from which to extract the spatial descriptors we use a modified version of a simple efficient method. We then proceed to evaluate our descriptor set on a database of video shots resulting from the temporal segmentation of the archive of a real-world TV station. Results demonstrate that our approach can achieve high success rates on a wide range of semantic classes.