Video summarization using descriptors of motion activity: A motion activity based approach to key-frame extraction from video shots

Abstract

We describe a video summarization technique that uses motion descriptors computed in the compressed domain. It can ei- ther speed up conventional color-based video summarization tech- niques, or rapidly generate a key-frame based summary by itself. The basic hypothesis of the work is that the intensity of motion ac- tivity of a video segment is a direct indication of its ''summarizability,'' which we experimentally verify using the MPEG-7 (MPEG-7 Visual Committee Draft URL: http://www.cselt.it/mpeg/working document- s.htm official MPEG site) motion activity descriptor and the fidelity measure proposed in H. S. Chang, S. Sull, and S. U. Lee, ''Efficient video indexing scheme for content-based retrieval,'' IEEE Trans. Cir- cuits Syst. Video Technol. 9(8), (1999). Note that the compressed domain extraction of motion activity intensity is much simpler than the color-based calculations. We are thus able to quickly identify easy to summarize segments of a video sequence since they have a low intensity of motion activity. We are able to easily summarize these segments by simply choosing their first frames. We can then apply conventional color-based summarization techniques to the re- maining segments. We thus speed up color-based summarization by reducing the number of segments processed. Our results also motivate a simple and novel key-frame extraction technique that relies on a motion activity based nonuniform sampling of the frames. Our results indicate that it can either be used by itself or to speed up color-based techniques as explained earlier. © 2001 SPIE and IS&T.