Personalized video summarization using sift

Abstract

In this paper, we propose a methodology for generating users' tailored video abstracts. First, video frames are scored by a group of video experts (operators) according to audio, visual and textual content of the video. Then, a matrix that contains the relevancy scores of each video scene into a number of pre-defined categories is computed using Scaled Invariant Feature Transform (SIFT) features, which are computed pairwise for representative keyframes and delegate images from the training collection. Later, for profiling purposes, an end-user's interest levels towards those high-level visual concepts (categories) are captured in the form of a vector. As a result of combining these two groups of data, the user's priorities in regards to different video segments can be determined. In the next stage, the initial averaged scores of the frames are updated based on the end-users' generated profiles. Eventually, the highest scored video frames alongside the auditory and textual content are inserted into final digest. The effectiveness of this approach has been evaluated by comparing the video summaries generated by this system against the results from a number of automatic and semi-automatic summarization tools that use different modalities for abstraction.