<title>Content-based indexing in the MPEG-1, -2, and -4 domains</title>

Abstract

Retrieval of videos from large databases using the inherent content as a key, is an important and challenging problem with many applications. The large volume of data associated with visual information presents challenges from the perspectives of storage, browsing, indexing and retrieval. The Moving Pictures Expert Group (MPEG) has addressed the issue of compression by proposing a family of video compression standards, namely MPEG 1, MPEG 2 and MPEG 4. In this paper, we propose a unified scheme for indexing the visual content in the MPEG 1, 2, and 4 domains. A video is first segmented into elemental units called shots. In the case of MPEG 1 and 2 videos containing simple camera operations (without significant object motion), we propose to generate a mosaic which is representative of the visual content of the entire shot in contrast to existing approaches where videos with both little and large motion employ one of the frames (say the first frame) of the shot as a representative key frame. In the case of MPEG 4 videos, sprites (proposed by the MPEG 4 standard) are used as the mosaic reflecting the background content of the shot. We propose a scheme for indexing the visual content by extracting features from the mosaic/key frame, which is tagged along with the temporal parameters obtained from the shot. The quantification and qualification of the color and texture information in the keyframes are obtained by using a supervised classifier. The shape information is extracted at the local and global levels using the concept of edge histograms. In addition, the shape information available from the binary alpha planes of the foreground video object in MPEG 4 is approximated by a B-Spline representation and used as a feature vector. A representation scheme has been developed which generates an XML file that contains the extracted content descriptors in accordance with the Data Description Language (DDL) of MPEG 7.