Mining Semantic Structures in Movies

Abstract

‘Video data mining’ is a technique to discover useful patterns from videos. It plays an important role in efficient video management. Particularly, we concentrate on extracting useful editing patterns from movies. These editing patterns are useful for an amateur editor to produce a new, more attractive video. But, it is essential to extract editing patterns associated with their semantic contents, called ‘semantic structures’. Otherwise the amateur editor can’t determine how to use the extracted editing patterns during the process of editing a new video. In this paper, we propose two approaches to extract semantic structures from a movie, based on two different time series models of the movie. In one approach, the movie is represented as a multi-stream of metadata derived from visual and audio features in each shot. In another approach, the movie is represented as one-dimensional time series consisting of durations of target character’s appearance and disappearance. To both time series models, we apply data mining techniques. As a result, we extract the semantic structures about shot transitions and about how the target character appears on the screen and disappears from the screen.