Temporal video structuring for preservation and annotation of video content

Abstract

As books have chapters, sections, paragraphs, sentences, etc., videos have an inherent hierarchical structure as well. Chapters, scenes, shots and sub-shots are the temporal units in videos. Because manual structure extraction is time-consuming, automatic segmentation has been a research effort in the past 10 to 15 years and is a prerequisite for efficient video indexing, annotation, search and retrieval. This paper focuses on our recent research in the fields of scene, shot and sub-shot extraction and their combination into a video structure detection system. The first step is the detection of shot transitions with separate detectors for hard cuts, fades, dissolves and wipes. Then complex shots are further segmented into semantically meaningful units called sub-shots. Finally the results are employed to extract scenes. We propose to use film-grammar based on shot transition types to improve the results of scene detection. The algorithms proposed are robust to distortions and artefacts found in digitized archived video.