User-based video abstraction using visual features

Abstract

In this paper, a novel mechanism for generating user-based video abstracts has been proposed. Initially, video experts (operators) score the video frames based on the different modalities contextual information. Later, a two-dimensional matrix that stores the relativity scores of each video scene and a number of pre-defined semantic categories are calculated utilizing SIFT (Scale Invariant Feature Transform) features of the representative key-frames. Next, the users' preference levels to each of those high-level visual categories are extracted in the form of an array. By fusion of these two groups of data, the user's priorities towards the different video segments can be measured. In the next step, the pre-computed averaged scores of the frames are updated according to the realized user's interest degree into the corresponding video segments. Finally, the highest scored video frames alongside the auditory and textual content are inserted into final digest. The effectiveness of this approach has been assessed by comparison of the video summaries generated by this system against the outcome from a number of automatic and semi-automatic summarization tools that use different modalities for abstraction.